JP2018116553A - Sign recognition system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sign recognition system capable of highly precisely recognizing words written in an auxiliary sign.SOLUTION: A sign recognition system includes an image acquisition unit, a major sign recognition unit, an auxiliary sign recognition unit, a first database, a second database, a third database, a first probability calculation unit, a second probability calculation unit, a first word estimation unit, a second word estimation unit, a meaning extraction unit, a situation data acquisition unit, and a vehicle identification unit. The first word estimation unit estimates words, which may exist in a portion of an auxiliary sign, on the basis of a probability of existence of a word in the portion of the auxiliary sign which is calculated for each of plural words. If a portion which cannot be estimated by the first word estimation unit is left, the second word estimation unit uses the second database and the words which can be estimated by the first word estimation unit to estimate the words in the portion which cannot be estimated by the first word estimation unit.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to a sign recognition device.

  Patent Document 1 discloses a sign recognition device. This sign recognition device classifies auxiliary signs as much as possible by the pattern recognition method. If classification by the pattern recognition method cannot be performed, the text in the auxiliary sign is read by the text recognition method.

Special table 2014-52049 gazette

  In the conventional sign recognition device, it is difficult to recognize the word written on the auxiliary sign with high accuracy. An object of one aspect of the present disclosure is to provide a sign recognition device that can recognize a word written on an auxiliary sign with high accuracy.

  One aspect of the present disclosure includes an image acquisition unit that acquires an image representing the periphery of a vehicle, a main sign recognition unit that recognizes a type of the main sign in the image, and an auxiliary sign recognition unit that recognizes an auxiliary sign in the image. A first database that stores the degree of association between a word that can be described in the auxiliary sign and the type of the main sign for each combination of the word and the type of the main sign; and the degree of association between the words A second database stored for each combination; a third database storing the meaning of the word or the combination of words; and the word in the portion of the auxiliary sign recognized by the auxiliary sign recognition unit in the image. A first probability calculating unit for searching and calculating a probability that the word is present in the auxiliary sign portion based on the search result; Based on the probability that the word is present in the portion of the auxiliary sign calculated by the calculation unit and the degree of association between the word and the type of the sign stored in the first database, the sign recognition unit A second probability calculating unit for calculating a probability that the word is present in the auxiliary sign portion when the recognized type of the main sign is present; and the auxiliary sign portion calculated for each of the plurality of words. Based on the probability that a word exists, a first word estimation unit that estimates the word existing in the auxiliary sign part and a part that cannot be estimated by the first word estimation unit remain in the auxiliary sign part In this case, the second database and the word that can be estimated by the first word estimation unit are used for the portion that cannot be estimated by the first word estimation unit. The auxiliary sign recognition unit recognizes based on the second word estimation unit for estimating the word to be performed, the estimation results of the first word estimation unit and the second word estimation unit, and the third database. The meaning extraction unit for extracting the meaning of the auxiliary sign, the situation data acquisition unit for acquiring the situation data representing the situation of the vehicle, the meaning extracted by the meaning extraction unit, and the situation data are compared, And a vehicle determination unit that determines whether or not the vehicle is a target of a type of the main sign recognized by the main sign recognition unit.

  The sign recognition device according to one aspect of the present disclosure includes a probability that the word is present in the portion of the auxiliary sign calculated by the first probability calculation unit, the word stored in the first database, and the book Based on the degree of association with the type of sign, a word existing in the auxiliary sign is estimated. The degree of association between the word and the type of the sign stored in the first database reflects the frequency of appearance of the word with respect to the type of the sign. Therefore, the word can be estimated with higher accuracy than when only a general character recognition algorithm using a camera is used.

  Thereby, the meaning of the auxiliary sign can be extracted with high accuracy. Furthermore, since the meaning of the auxiliary sign can be extracted with high accuracy, it can be determined with high accuracy whether or not the vehicle is the target of the main sign.

  In addition, the sign recognition device according to one aspect of the present disclosure can store the contents of the second database and the words that can be estimated by the first word estimation unit even when a portion that cannot be estimated by the first word estimation unit remains. Can be used to estimate the portion of the word that cannot be estimated by the first word estimation unit.

It is a block diagram showing the structure of the sign recognition system 1 and the sign recognition apparatus 3. 3 is a block diagram illustrating a functional configuration of a control unit 7. FIG. 3A and 3B are explanatory diagrams showing the probability P (a | w), respectively. It is explanatory drawing showing a connection probability. It is a flowchart showing the process which the label | marker recognition apparatus 3 performs. It is explanatory drawing showing this label | marker 69, the auxiliary | assistant label | marker 71, and the word presence area 73. FIG. It is an explanatory view showing a method of estimating the word w _i. It is explanatory drawing showing the method of estimating the word x using a connection probability.

An embodiment of the present disclosure will be described with reference to the drawings.
<First Embodiment>
1. Configurations of the Sign Recognition System 1 and the Sign Recognition Device 3 The configurations of the sign recognition system 1 and the sign recognition device 3 will be described with reference to FIGS. As shown in FIG. 1, the sign recognition system 1 includes a sign recognition device 3 and an external server 5. The sign recognition device 3 is an in-vehicle device mounted on a vehicle. Hereinafter, the vehicle on which the sign recognition device 3 is mounted is referred to as the own vehicle. The sign recognition device 3 includes a control unit 7, a first DB (database) 9, a second DB (database) 11, and a third DB (database) 13.

  The control unit 7 is mainly configured by a known microcomputer having a CPU 15 and a semiconductor memory (hereinafter referred to as a memory 17) such as a RAM, a ROM, and a flash memory. Various functions of the control unit 7 are realized by the CPU 15 executing a program stored in a non-transitional physical recording medium. In this example, the memory 17 corresponds to a non-transitional tangible recording medium that stores a program. Also, by executing this program, a method corresponding to the program is executed. The number of microcomputers constituting the control unit 7 may be one or more.

  As shown in FIG. 2, the control unit 7 includes, as shown in FIG. 2, an image acquisition unit 19, a main sign recognition unit 21, an auxiliary sign recognition unit 23, and a first function structure realized by the CPU 15 executing a program. Probability calculation unit 25, second probability calculation unit 27, first word estimation unit 31, second word estimation unit 33, meaning extraction unit 35, situation data acquisition unit 37, and vehicle determination unit 39, a display unit 41, a selection unit 43, an information output unit 45, an update unit 47, and a motion control unit 49.

  The method of realizing these elements constituting the control unit 7 is not limited to software, and some or all of the elements may be realized using one or a plurality of hardware. For example, when the above function is realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit including a large number of logic circuits, an analog circuit, or a combination thereof.

  When the word w is present in the auxiliary sign, the first DB 9 uses the probability P (a | w) that the sign a is present as the type of the sign corresponding to the auxiliary sign, and the sign a and the word w. And remember for each combination. The probability P (a | w) corresponds to the degree of association between the word that can be described in the auxiliary sign and the type of the sign.

For example, as shown in FIG. 3A, when the word w ₁ of “bicycle” is present in the auxiliary sign, the probability P (a | w ₁ ) that the sign a is present as the main sign corresponding to the auxiliary sign is set. , Stored for each combination of various book signs a and “bicycle” (word w ₁ ).

Further, for example, as shown in FIG. 3B, when the word “w ₂ ” “stop” exists in the auxiliary sign, the probability P (a | w ₂₎ that the main sign a exists as the main sign corresponding to the auxiliary sign. ) Is stored for each combination of various main signs a and “stop” (word w ₂ ). The words w ₁ and w ₂ are examples of the word w.

  In the present specification, the word w is not necessarily limited to a word in a general sense, and broadly means a word composed of one or more characters. Characters include kanji, hiragana, katakana, numbers, romaji, symbols, and the like.

  When the second DB 11 includes a word w or a group of words w in the auxiliary sign, the probability that the word x exists in the auxiliary sign (hereinafter referred to as a connection probability) is determined as a word w or a group of words w. For each combination with the word x. For example, the word x may be located immediately after the word w, may be located immediately before the word w, or may be located away from the word w in the auxiliary sign. In addition, the word x may be located between the first word w and the second word w. The connection probability corresponds to the degree of association between words.

  For example, as shown in FIG. 4, when the word “bicycle” is present on the auxiliary sign, the second DB 11 is connected to the word “x” immediately after “bicycle” in the auxiliary sign. The probability (0.4) is stored.

  Further, for example, as shown in FIG. 4, in the second DB 11, when the word w “bicycle” is present in the auxiliary sign, the word x “excluded” is present immediately after “bicycle” in the auxiliary sign. The connection probability (0.09) is stored.

  The second DB 11 also stores the probability of the first word in the auxiliary sign. For example, as shown in FIG. 4, the probability that the first word is “bicycle” is 0.1, and the probability that the first word is “1” is 0.01.

  The third DB 13 stores the meaning of each word. In addition to the sign recognition device 3, the host vehicle is provided with a camera 51, a motion control device 53, a display device 55, a communication device 57, and an input device 59.

The camera 51 captures the vicinity of the host vehicle and creates an image. The image is an image representing the periphery of the host vehicle. The camera 51 outputs the created image to the sign recognition device 3.
The motion control device 53 controls the motion of the host vehicle. Examples of the control of the movement of the host vehicle include parking, turning left and right, speed control, changing the driving lane, and the like.

  The display device 55 is a display provided in the passenger compartment of the host vehicle. The display device 55 displays an image according to the signal from the sign recognition device 3. The communication device 57 is a device that performs wireless communication with the external server 5. The input device 59 is provided in the passenger compartment of the host vehicle and accepts an input operation of a driver. The driver corresponds to the user. The input device 59 outputs a signal corresponding to the input operation to the sign recognition device 3.

  The external server 5 is fixedly installed outside the host vehicle. The external server 5 includes a control unit 61, a server DB 63, and a communication device 65. The control unit 61 is configured around a well-known microcomputer, and executes processing to be described later. The server DB 63 stores the same contents as the first DB 9 and the second DB 11. The communication device 65 performs wireless communication with the communication device 57 of the sign recognition device 3.

2. Processing Performed by the Sign Recognition Device 3 The processing that the sign recognition device 3 repeatedly executes every predetermined time will be described with reference to FIGS. In step 1 of FIG. 5, the update unit 47 communicates with the external server 5 using the communication device 57, acquires the storage contents of the server DB 63, and updates the storage contents of the first DB 9 and the second DB 11. Update to stuff. The external server 5 updates the stored contents of the server DB 63 to the latest one using the information transmitted in step 20 described later.

In step 2, the image acquisition unit 19 uses the camera 51 to acquire an image representing the periphery of the host vehicle.
In step 3, the sign recognition unit 21 performs processing for recognizing the sign in the image acquired in step 2, and determines whether the sign is present.

  As a process for recognizing the sign, for example, a general sign recognition algorithm using a camera can be used. As a label recognition algorithm, for example, there is a technique for recognizing a label by pattern recognition. As a technique for recognizing a sign by pattern recognition, for example, SVM (support vector machine) and the like can be mentioned. In SVM, marker data and data other than markers are prepared as learning samples at the time of learning, and an identification plane that maximizes the distance to each data point is obtained in a higher-order feature space. At the time of identification, it is determined whether or not a sign is included in the input image using the identification plane. Further, the process for recognizing the present sign may be, for example, a method for recognizing the present sign on a rule basis from the image feature or a method for learning by CNN (Convolutional Neural Network).

If it is determined that the sign is present in the image acquired in step 2, the process proceeds to step 4; otherwise, the process is terminated.
In step 4, the sign recognition unit 21 recognizes the type of the sign determined to exist in step 3. Hereinafter, the recognized type is a.

  In step 5, the auxiliary sign recognition unit 23 performs processing for recognizing the auxiliary sign in the image acquired in step 2, and determines whether or not the auxiliary sign exists. In the recognition of the auxiliary sign, in addition to the shape of the auxiliary sign, the positional relationship between this sign and the auxiliary sign may be used. For example, the auxiliary sign can be recognized limited to the vicinity of the present sign or by giving priority to the vicinity of the present sign. The method for recognizing the auxiliary sign using the shape of the auxiliary sign can be the same as the method for recognizing the present sign, for example.

If it is determined that an auxiliary sign is present in the image acquired in step 2, the process proceeds to step 6; otherwise, the process ends.
FIG. 6 shows an example of the image 67 acquired in step 2, the main marker 69 determined to be recognized in step 3, and the auxiliary marker 71 determined to be recognized in step 5.

  In step 6, the auxiliary sign recognition unit 23 sets a word presence area in the image acquired in step 2. The word presence area is a partial area of the recognized auxiliary sign and is generally an area where a word is described. FIG. 6 shows an example of the word existence area 73.

In step 7, the variable i is set to 1 which is an initial value.
In step 8, the first probability calculation unit 25 searches for the word w _i in the word existence area 73. The word w _i is a word stored in the first DB 9.

In step 9, the probability P (w _i ) that the word w _i exists in the word existence area 73 is calculated based on the result of the search in step 8. For example, as illustrated in FIG. 7, when a word w _i is found in a part A _j that is a part of the word existence area 73 as a result of the search, the probability P (w that the word w _i exists in the part A _{j i} ) is calculated. The calculation of the probability P (w _i ) can be performed by a general character recognition algorithm using a camera.

In step 10, when the word w _i is present in the word presence area 73 of the auxiliary sign, the probability P (a | w _i ) that the sign a is present corresponding to the auxiliary sign is determined based on the first DB 9. The second probability calculation unit 27 calculates.

In step 11, the second probability calculation unit 27 multiplies the probability P (w _i ) calculated in step 9 by the probability P (a | w _i ) calculated in step 10 to obtain a probability P ( Calculate w _i | a). Multiplying the probability P (w _i ) and the probability P (w _i | a) corresponds to a calculation method based on them. The calculation of the probability P (w _i | a) is based on the following equation (1). Since this label a is recognized, P (a) is 1 in the formula (1).

The probability P (w _i | a) means the probability that the word w _i exists in the portion A _j in the word existence area 73 when the main marker a whose type is recognized in step 4 is present.
In step 12, it is determined whether i is equal to N or not. N is a constant. N may be the total number of words w stored in the first DB 9 or may be a value smaller than the total number. When N is smaller than the total number of words w, for example, w _{1 to} w _N have a relatively high probability of being present when the present marker a is present in the word w stored in the first DB 9. It can be the top N words.

If i is equal to N, the process proceeds to step 14; otherwise, the process proceeds to step 13.
In step 13, the value of i is incremented by 1, and the process proceeds to step 8.
In step 14, the processing at step 6-13, the probability (the i 1 to N) a plurality of word _w i calculated respectively for P | based on _(w i a), the words _{w i} which exists in the word existing region 73 Is estimated by the first word estimation unit 31.

For example, in the example shown in FIG. 7, among the probabilities P (w _i | a) calculated for the portion A _j , the value of the probability P (w _y | a) is the other probability P (w _i | a) ( here, i 1~ (y-1), greater than (y + 1) ~N), and, if it exceeds a preset threshold value, the words present in the portion _{a j} is the word _{w y} Presume that there is. y is any one of 1 to N.

If there is no probability P (w _i | a) calculated for the part A _j that exceeds a preset threshold, it is assumed that the word existing in the part A _j cannot be estimated at that time. In this case, the portion A _j is a word estimation impossible portion described later.

  In step 15, the second word estimation unit 33 determines whether or not a portion where the word cannot be estimated (hereinafter referred to as a word estimation impossible portion) remains in the word existence area 73 in the processing of step 14. To do. When the word unestimable part remains, the process proceeds to step 16, and when the word unestimable part does not remain, the process proceeds to step 21.

In step 16, the second word estimation unit 33 estimates the word x existing in the word unestimable portion using the second DB 11 and the word w _i estimated in step 14.
For example, as shown in FIG. 8, in the word existence area 73, the part A _j is a part that cannot be estimated, and the preceding and following parts A _j−2 , A _j−1 , A _{j + 1} , A _{j + 2} 14, it is assumed that the word can be estimated.

The second word estimation unit 33 adds the connection probability stored in the second DB 11 to the word w _i estimated in any of the parts A _j−2 , A _j−1 , A _{j + 1} , A _{j + 2.} By applying this, the word x existing in the word estimation impossible part is estimated.

  More specifically, the second word estimation unit 33 estimates a plurality of words x having a relatively high probability of being present in the word estimation impossible portion. For example, the second word estimation unit 33 estimates the word x having the highest probability of being present in the word non-estimable portion, the second highest word x... The Lth highest word x, respectively. L is an integer of 2 or more.

In step 17, the display unit 41, and displays on the display device 55 a word w _i estimated at the step 14, displaying a plurality of words x estimated by the step 16 to the display device 55. The driver can see the display and use the input device 59 to perform a process of selecting any one of the plurality of words x.

  In step 18, the selection unit 43 determines whether or not there is an input to the input device 59 by the driver. If there is an input, the process proceeds to step 19, and if there is no input, the present process is terminated.

  In step 19, one word x selected by the driver input is selected from the plurality of words x estimated in step 16, and it is estimated that the selected word x exists in the word unestimable portion.

In step 20, the word x selected in the step 19, the estimated in step 14, and the other words w _i in the portion other than the word predicting invalid portion, and the label a recognized in the step 4, the The information output unit 45 outputs the included information to the external server 5 using the communication device 57.

The external server 5 uses the information obtained from the sign recognition device 3 to update the storage content of the server DB 63. That is, the probability P (a | w) is updated using the information obtained from the sign recognition device 3. For example, when the content of the information obtained from the sign recognition device 3 includes the present sign a and the word w _i , the value of P (a | w _i ) is increased by the amount of the information.

Further, the external server 5 updates the connection probability using the information obtained from the sign recognition device 3. For example, the content of the information obtained from the label recognition device 3, if is that word x is present after the word w _i, the connection probability of a word x is present after the word w _i, by the amount of information increases To do.

  In step 21, the meaning extraction unit 35 extracts the meaning of the word estimated in steps 16 and 19 using the storage in the third DB 13. Furthermore, the meaning extraction unit 35 extracts the meaning of the entire auxiliary sign based on the meaning of each word.

  In step 22, the situation data acquisition unit 37 acquires situation data representing the situation of the host vehicle. The situation of the host vehicle is the time, the position of the host vehicle, the type of the host vehicle, the vehicle height of the host vehicle, the vehicle weight of the host vehicle, the vehicle width of the host vehicle, and the vehicle speed of the host vehicle.

  The status data acquisition unit 37 acquires time from a clock (not shown). The situation data acquisition unit 37 acquires the position of the host vehicle from GPS (not shown) and map data. The vehicle height of the host vehicle, the vehicle weight of the host vehicle, and the vehicle width of the host vehicle are stored in the memory 17 in advance. The situation data acquisition unit 37 acquires the vehicle speed of the host vehicle from a vehicle speed sensor (not shown).

  In step 23, the meaning of the auxiliary sign extracted in step 21 is compared with the situation data acquired in step 22, and whether the host vehicle is the target of this sign a whose type is recognized in step 4 or not. The vehicle determination unit 39 determines whether or not.

  For example, if the meaning of the auxiliary sign is a meaning that limits the time such as “from 9 o'clock to 12 o'clock”, the meaning of the auxiliary sign is compared with the time that is one of the situations of the own vehicle. It can be determined whether or not the vehicle is the target of the sign a for which the type is recognized in step 4.

  In addition, when the meaning of the auxiliary sign is to limit the position such as “the whole area of XX city”, the meaning of the auxiliary sign is compared with the position of the own vehicle which is one of the situations of the own vehicle. It can be determined whether or not the subject vehicle is the target of the sign a for which the type has been recognized in step 4.

  Further, when the meaning of the auxiliary sign is to limit the type of the vehicle such as “except for a light car”, the meaning of the auxiliary sign is compared with the type of the own vehicle which is one of the situations of the own vehicle. Thus, it can be determined whether or not the own vehicle is the target of the sign a for which the type is recognized in step 4.

  In addition, when the meaning of the auxiliary sign is a meaning that limits the vehicle height, such as “2 m or less”, the meaning of the auxiliary sign is compared with the vehicle height of the own vehicle, which is one of the situations of the own vehicle. It can be determined whether or not the subject vehicle is the target of the sign a for which the type has been recognized in step 4.

  In addition, when the meaning of the auxiliary sign is to limit the weight of the vehicle such as “2 t or less”, the meaning of the auxiliary sign is compared with the weight of the own vehicle, which is one of the situations of the own vehicle. It can be determined whether or not the subject vehicle is the target of the sign a for which the type has been recognized in step 4.

  If the meaning of the auxiliary sign is to limit the vehicle width such as “width 2 m or less”, the meaning of the auxiliary sign should be compared with the vehicle width of the own vehicle, which is one of the conditions of the own vehicle. Thus, it can be determined whether or not the host vehicle is the target of the sign a for which the type is recognized in step 4.

  If the meaning of the auxiliary sign is to limit the speed such as “safety speed 30”, the meaning of the auxiliary sign is compared with the vehicle speed of the own vehicle, which is one of the conditions of the own vehicle. It can be determined whether or not the host vehicle is the target of the sign a for which the type is recognized in step 4.

In step 24, the motion control unit 49 controls the motion of the host vehicle using the motion control device 53 according to the determination result in step 23.
The movement of the host vehicle according to the determination result in step 23 is, for example, as follows. When it is determined that the host vehicle is the target of the sign a, exercise according to the meaning of the sign is performed. For example, when this sign a means that parking is prohibited, parking is not performed at that location. If this sign a means prohibiting a right turn, do not make a right turn. On the other hand, when it is determined that the host vehicle is not the target of the sign a, exercise is not limited to the meaning of the sign a.

In step 24, according to the determination result in step 23, display, warning, etc. are performed on the display device 55 if necessary.
3. Effects produced by the sign recognition device 3 (1A) The sign recognition device 3 estimates the word w _i existing in the word existence region 73 based on the probability P (w _i | a). The probability P (w _i | a) is a probability reflecting the appearance frequency of the word w _i for the sign a. Therefore, the word w _i can be estimated with higher accuracy than when only a general character recognition algorithm using a camera is used.

  Thereby, the meaning of the auxiliary sign can be extracted with high accuracy. Furthermore, since the meaning of the auxiliary sign can be extracted with high accuracy, it can be determined with high accuracy whether or not the host vehicle is the target of the main sign a.

(1B) The sign recognition device 3 can estimate the word x of the word non-estimable part using the connection probability even when the word non-estimable part remains.
(1C) The sign recognition device 3 displays the word estimated in the auxiliary sign on the display device 55. Therefore, it can suppress that a driver overlooks an auxiliary sign.

  (1D) In step 17, the sign recognition device 3 displays a plurality of estimation results on the display device 55 that have a relatively high probability of being present in the word estimation impossible portion. Therefore, it is possible to prevent the correct estimation result from being displayed compared to the case where only one estimation result is displayed.

  (1E) The sign recognition device 3 selects any one of the plurality of estimation results displayed in step 17 according to the driver's operation on the input device 59. Therefore, the correct word can be estimated by selecting the driver.

(1F) The sign recognizing device 3 recognizes the word x selected in the step 19, the other words w _i in the portion other than the word estimation impossible portion estimated in the step 14, and the book recognized in the step 4. Information including the sign a is output to the external server 5 using the communication device 57.

  The external server 5 can update the storage content of the server DB 63 using the information obtained from the sign recognition device 3. Therefore, the stored contents of the server DB 63 can be made more appropriate. Further, even when a new word appears in the auxiliary sign, the stored contents of the server DB 63 can be made to correspond to the new word.

  (1G) The sign recognition device 3 can update the storage contents of the first DB 9 and the second DB 11 using the storage contents of the external server 5. Therefore, the storage contents of the first DB 9 and the second DB 11 can be made more appropriate. Even when a new word appears in the auxiliary sign, the stored contents of the first DB 9 and the second DB 11 can be made to correspond to the new word.

  (1H) The sign recognition device 3 uses the time, the position of the host vehicle, the type of the host vehicle, the vehicle height of the host vehicle, the vehicle weight of the host vehicle, the vehicle width of the host vehicle, and the vehicle speed of the host vehicle as the vehicle status. Including. Therefore, it is possible to more appropriately determine whether or not the host vehicle is the target of the sign a.

(1I) The sign recognition device 3 controls the movement of the host vehicle according to the determination result of the vehicle determination unit 39. Therefore, it is possible to perform an appropriate exercise with respect to the meaning of the present sign and the auxiliary sign.
<Other embodiments>
As mentioned above, although embodiment of this indication was described, this indication is not limited to the above-mentioned embodiment, and can carry out various modifications.

  (1) The sign recognition device 3 can update the storage contents of the first DB 9 and the second DB 11 using the information output in step 20. The updating method is the same as the method for updating the server DB 63. Further, the sign recognition device 3 may update the storage contents of the first DB 9 and the second DB 11 without outputting the information in the step 20.

(2) The second DB 11 may store the relationship between the vehicle information and the word x. For example, you may store the relationship between the position of the own vehicle which is an example of the own vehicle information, and the word x. For example, when the position of the host vehicle is a shopping street entrance or the like, it is possible to increase the probability of the word x that limits the type and time of the vehicle, such as “excluding specific permitted vehicles” or “9-17”. In addition, for example, when it is possible to determine that the vehicle passes under the overhead by the navigation map information, the probability of the word w _i that limits the vehicle height such as “2 m or less” can be increased.

  (3) The connection probability for the word x recognized within a certain time or within a certain space range may be set higher than in other cases. For example, when a word x that restricts the type of vehicle such as “excludes a light car” is recognized, the connection probability for the word x may be higher than in other cases.

  (4) The recognized word may be used to determine whether or not it is an auxiliary sign. For example, when a word that is not stored in the first DB 9 or the second DB 11 appears frequently in the recognized object, the probability that the object is an auxiliary sign can be lowered. Thereby, it is possible to suppress a signboard or the like that is not an auxiliary sign from being recognized as an auxiliary sign.

  (5) A plurality of functions of one constituent element in the above embodiment may be realized by a plurality of constituent elements, or a single function of one constituent element may be realized by a plurality of constituent elements. . Further, a plurality of functions possessed by a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified from the wording described in the claims are embodiments of the present disclosure.

  (6) In addition to the above-described sign recognition device, a system including the sign recognition device as a constituent element, a program for causing a computer to function as a control unit of the sign recognition device, a non-transitive semiconductor memory or the like in which the program is recorded The present disclosure can also be realized in various forms such as an actual recording medium, a sign recognition method, a vehicle control method, and a driving support method.

DESCRIPTION OF SYMBOLS 1 ... Sign recognition system, 3 ... Sign recognition apparatus, 5 ... External server, 7 ... Control part, 9 ... 1st DB, 11 ... 2nd DB, 13 ... 3rd DB, 15 ... CPU, 17 ... Memory , 19 ... Image acquisition unit, 21 ... Main sign recognition unit, 23 ... Auxiliary sign recognition unit, 25 ... First probability calculation unit, 27 ... Second probability calculation unit, 31 ... First word estimation unit, 33 ... Second word estimation unit, 35 ... semantic extraction unit, 37 ... situation data acquisition unit, 39 ... vehicle judgment unit, 41 ... display unit, 43 ... selection unit, 45 ... information output unit, 47 ... update unit, 49 ... exercise Control unit 51 ... Camera 53 ... Motion control device 55 ... Display device 57 ... Communication device 59 ... Input device 61 ... Control unit 63 ... Server DB 65 ... Communication device 67 Image, 69 ... the label, 71 ... auxiliary label, 73 ... word existing region

Claims (8)

An image acquisition unit for acquiring an image representing the periphery of the vehicle;
A sign recognition unit for recognizing the type of the sign in the image;
An auxiliary sign recognition unit for recognizing an auxiliary sign in the image;
A first database that stores the degree of association between a word that can be described in the auxiliary sign and the type of the main sign for each combination of the word and the type of the main sign;
A second database storing the degree of association between the words for each combination of the words;
A third database storing the meaning of the words or combinations of the words;
First probability calculation for searching for the word in the portion of the auxiliary sign recognized by the auxiliary sign recognition unit in the image and calculating the probability that the word exists in the auxiliary mark portion based on the search result Unit,
Based on the probability that the word is present in the portion of the auxiliary sign calculated by the first probability calculating unit, and the degree of association between the word and the type of the sign stored in the first database, A second probability calculating unit that calculates a probability that the word is present in the auxiliary sign portion when the type of the sign recognized by the sign recognition unit exists;
A first word estimation unit that estimates the word present in the auxiliary sign part based on the probability that the word exists in the auxiliary sign part calculated for each of the plurality of words;
When a portion that cannot be estimated by the first word estimation unit remains in the auxiliary sign portion, the second database and the word that can be estimated by the first word estimation unit are used. A second word estimation unit for estimating the word existing in a portion that cannot be estimated by the first word estimation unit;
A meaning extraction unit that extracts the meaning of the auxiliary sign recognized by the auxiliary sign recognition unit based on the estimation result of the first word estimation unit and the second word estimation unit and the third database;
A situation data acquisition unit for acquiring situation data representing the situation of the vehicle;
A vehicle determination unit that compares the meaning extracted by the meaning extraction unit with the situation data to determine whether or not the vehicle is a target of the type of the main sign recognized by the main sign recognition unit;
A sign recognition device comprising: The sign recognition device according to claim 1,
A sign recognition device further comprising a display unit for displaying estimation results of the first word estimation unit and the second word estimation unit. The sign recognition device according to claim 2,
The sign recognition device configured to display a plurality of estimation results of the second word estimation unit, wherein the display unit has a relatively high probability of being present in a portion that cannot be estimated by the first word estimation unit. The sign recognition device according to claim 3,
A sign recognition device further comprising a selection unit that selects one of the plurality of estimation results displayed by the display unit in accordance with a user operation. The sign recognition device according to claim 4,
A sign further comprising an information output unit for outputting information including the estimation result selected by the selection unit, another word in the auxiliary sign, and the type of the sign recognized by the sign recognition unit to the outside Recognition device. The sign recognition device according to any one of claims 1 to 5,
The sign recognition apparatus further comprising an update unit that updates the storage contents of the first database and the second database using the storage contents of an external server. The sign recognition device according to any one of claims 1 to 6,
The vehicle status is one or more selected from the group consisting of time, position of the vehicle, vehicle type, vehicle height, vehicle weight, vehicle width, and vehicle speed. Is a sign recognition device. The sign recognition device according to any one of claims 1 to 7,
A sign recognition device further comprising a motion control unit that controls motion of the vehicle according to a determination result of the vehicle determination unit.

Images