WO2011080639A1

WO2011080639A1 - An image recognition system

Info

Publication number: WO2011080639A1
Application number: PCT/IB2010/055719
Authority: WO
Inventors: Mehmet Emin Ozgul
Original assignee: Turkcell Teknoloji Arastirma Ve Gelistirme AS
Current assignee: Turkcell Teknoloji Arastirma Ve Gelistirme AS
Priority date: 2009-12-31
Filing date: 2010-12-10
Publication date: 2011-07-07
Anticipated expiration: 2012-06-30
Also published as: TR200910111A2

Abstract

The present invention relates to an image recognition system (1) which realizes an image recognition system that enables an image to be recognized by searching images such as symbol, logo, photo defined on the system from live images recorded by these phone cameras, using capability of making video call provided in 3G phones; and enables functions of sending back a live image to the mobile device by being enriched with data associated with the image recognized and sending SMS, MMS to the device or establishing communication with another device to be fulfilled.

Description

AN IMAGE RECOGNITION SYSTEM

Field of the Invention The present invention relates to processing, recognition of images using video call becoming widespread by third generation networks and an image recognition system which is easy to use and enjoyable by taking actions depending on the image recognized. Background of the Invention

Third generation networks (3N) is a name given to a version of GSM wireless communication systems and uses cellular network system. Wireless telephone calls, video calls, data transfer and many more applications in wide area are among services which are provided to mobile users in third generation networks. Third generation networks support synchronous talk and data service and higher data rates. Video call is another service started to be provided by third generation networks. Besides innovations brought by third generation networks, another improvement occurred on mobile devices and these devices got opportunities of camera, high-capability processors and storage. Upon softwares having these capabilities became widespread; an increase arose in number of image recognition applications too. Prior arts of the image recognition application are softwares which are installed into the mobile device and interactive service development media that do not have feature of image recognition and run upon keyboard, mouse and audio entries of the user are recognized.

There must be a software which is compatible with the brand and the model of the mobile device and new versions thereof must be developed, in order that the software is installed into the mobile device. Then the user must be provided with necessary medium access and the medium information must be given to her/him in order to install this software into her/his device. The user must install and set up the software into her/his device without any problem. After the software is installed without any problem, if the user wants to reach the service s/he should start the software and perform the recognition process by following instructions. In order that information about the image recognized is transmitted to the user after a successful recognition process, the mobile device makes a network connection and receives the data formatted specifically for the device. Needs for formatting special to the said mobile device vary too much and this increases costs. Thus, different softwares are developed for devices having different features and tested for all possible combinations. This increases costs.

Whereas in systems which provide interaction by keypad, mouse and audio information, another application, there are many steps to be followed in order to reach information. Although following and carrying out the said steps takes a very long time and leads to mistakes it has a common usage. This application lost its attraction especially for services wherein operating time is critical or which requires access to far too much information to be provided from a single interface.

There are certain problems and difficulties in currently existing applications which are mobile terminal-based. These are developing different softwares for devices with different features and testing them. This increases costs. Appropriate versions of the existing software must be developed. If there is no appropriate version the user cannot utilize this application. And it takes the user's time to install and set up the software compatible with the device without any problem and comprises some difficulties for the user.

The United States patent document no. US2009147990 discloses an image recognition system. In this system picture or video, which is sent by a mobile device and comprises identification information, is sent to a processing and authentication server by means of a communication network. In order to identify the location of the person or the identity of the person, the processing and authentication server searches contents such as symbol, logo or character which are included in the picture or video information that it has received and converts the contents it has found into a digital format.

Summary of the Invention

Objective of the present invention is to realize an image recognition system which enables an image to be recognized by searching images such as symbol, logo, photo that are defined on the system from live images recorded by phone cameras, using capability of making video call provided in 3G phones.

Another objective of the present invention is to realize an image recognition system which enables a live image to be sent back to the mobile device by being enriched with data associated with the image recognized. A further objective of the present invention is to realize an image recognition system which enables functions of sending SMS, MMS to the device or establishing communication with another device to be fulfilled.

Detailed Description of the Invention

"An image recognition system and method" realized to fulfill the objective of the present invention is illustrated in the accompanying figures, in which:

Figure 1 is the schematic view of the image recognition system.

Figure 2 is the flowchart of the image recognition method.

The parts illustrated in the figures are individually numbered where the numbers refer to the following:

1. Image recognition system

2. 21. Mobile device 3. Access layer

4. Image processor

5. Response processor

6. Control unit

The inventive image recognition system (1) essentially comprises: at least one mobile device (2); at least one access layer (3) which is in communication with the mobile device (2) and has call control component, audio and image access layer, audio and image transfer layer and network control component; at least one image processor (4) which consists of access layer input, picture decoder, symbol database and image mapper part; at least one response processor (5) which cooperates with the image processor (4) and at least one control unit (6) which regulates the communication between the response processor (5) and the image processor (4).

The said mobile device (2) is any mobile device which has capability of video call.

In the preferred embodiment of the invention, a transmitter mobile device (2) and a receiver mobile device (21 ) are used.

The access layer (3) has: call control component which detects the call sent by the mobile device (2), access layer which opens the related media channel, access layer which decodes the call, audio and image access layer which separates the call into its audio and image components, audio and image transfer layer which transfers the audio and the image to the related network component and network control component which controls the audio and image transfer layer.

The image processor (4) has components of: network access layer input which receives the audio and the image from the access layer (3), picture decoder which separates the audio and image information into its singular components, defined symbol database which contains defined patterns in itself, image mapper which searches for defined patterns within every picture decoded, video database which has videos, time signal which is a time signal generator, video player for playing video, network access layer output which transfers the obtained audio and image to the access layer (3).

The response processor (5) has components of other remote access which has capability of remote function call, journal record, messaging which sends e- mail/MMS/SMS, content enrichment which enriches content of the image recognized.

The symbol enrichment process of which has been completed is transferred to the image processor (4) by the control unit (6). The inventive image recognition system (1) uses an image recognition method (100) comprising the steps of:

— audio and image being separated into their components by the audio and image access layer (3) (101);

— audio and image components being transferred to the network component (102);

— audio and image information being received by the network access layer input which is located in the image processor (4) (103);

— audio and image information received being separated into its singular components by the picture decoder in the image processor (4) (104);

— defined patterns in the symbol database located in the image processor (4) being searched within every picture which is decoded by the image mapper (105);

— carrying out the process which is stated in the record in the symbol database (106); — audio and image information being transmitted to the mobile device (21) which makes video call by the access layer (3) (107).

In the preferred embodiment of the invention, the access layer (3) consists of sublayers which use 3G (Third Generation) H.324m and TDM protocols. In this protocol, video call is conveyed to the image processor (4) by ISUP (Integrated Services User Part) markup and switching system. The ISUP defines setting up, terminating voice and data calls and processes thereof on circuit-switching networks. The incoming ISUP call is received by the call control component in the ISUP access layer (3). The media channel related to the incoming call is opened by the TDM (Time Division Multiplexing) access layer (3) and access to this channel is provided. The call transmitted by the TDM access layer (3) is decoded by the H.324m access layer (3). Audio and image carried in the call are separated into their components by the audio and image access layer (3). Thus, the audio and the image can be accessed independently of each other. The components of audio and image, which are accessed independently, are transferred to the network component stated by the network transfer layer that is located in the audio and image access layer (3) controlled by the network control component in the access layer. If there is call, audio and image transfer from the access layer (3) towards the mobile device (2) these data are transmitted to the audio and image access layer by being received from the related network component via the network transfer layer which is controlled by the network control component located in the access layer (3) and located in the audio and image access layer (3). The call, audio and image are separated into audio and image components thereof in the audio and image access layer (3). Thus, the audio and the image can be accessed independently of time. The audio and the image are transferred to the mobile device (21) by the access layer (3). The audio and image information transmitted by the network layer (3) is received by the network access layer input located in the image processor (4). The audio and image information received is separated into its singular components by the picture decoder in the image processor (4). Defined patterns in the symbol database which is located in the image processor (4) are searched within every decoded picture. This search process is carried out by the image mapper which is located in the image processor (4). If one of the defined symbols exists in the picture being processed at that moment, a process which is stated in the record found in the symbol database that is owned by the image processor (4) is carried out with respect to this symbol. There is at least one process defined for each symbol in the symbol database. If a plurality of process is defined for a symbol these processes are carried out by the image processor (4) or transmitted to the response processor (5) so that the processes are carried in order of priorities. In this case, the image processor (4) cooperates with the response processor (5). If a process comprises one of: other remote access, journal record, e-mail/MMS/SMS messaging or content enrichment processes these processes are carried out by interaction of the response processor (5). Communication between the response processor (5) and the image processor (4) is provided by the communication control unit (6).

If there is a process of making request made from an outside source regarding the symbol defined in the symbol database the response processor (5) can enable a component located outside the system to decide how the process will be continued, by means of remote access component.

In the preferred embodiment of the invention, methods of remote function call such as web service/CORBA (Common Object Request Broker Architecture) are used as remote access component. Function to be fulfilled as a result of the decision given is transferred to the image processor (4) by the control unit (6). If there is process of determining the video to be shown to the user regarding the symbol recognized in the symbol database, this video is searched in the video database located in the image processor (4) and it is accessed. The video accessed is transmitted to the component of network access layer output located in the image processor (4) in accordance with the time signal generated by the time signal generator located in the image processor (4), by the video player located in the image processor (4). The network access layer output transmits these audio and video packages to the audio and video transfer layer. Audio and image received by the network access layer output are transmitted to the audio and image access layer. The audio and the image are separated into audio and image components thereof in the access layer. Thus, the audio and the image can be accessed independently of time. The audio and image access layer transmits the audio and image information to the access layer (3) with an appropriate timing. The audio and image information is transferred to the mobile device (21) making video call by the access layer (3).

If there is enrichment process of live image regarding the symbol defined in the symbol database this process is carried out by content enrichment component located in the response processor (5). The symbol content enrichment process of which has been completed is transferred to the image processor (4) by the control unit (6). Upon the symbol content enrichment process of which has been performed is transferred to the image processor (4) by the control unit (6), the video player transmits the content to the network access layer output by embedding it onto live images that it received from the picture decoder. The network access layer output transmits these audio and video packages to the audio and video transfer layer. The audio and the image are separated into audio and image components thereof in the access layer. Thus, the audio and the image can be accessed independently of time. The audio and image access layer transmits the audio and image information to the access layer with an appropriate timing. The audio and image information is transferred to the mobile device (21) making video call by the access layer (3). If a process of connection to another device is defined regarding the symbol recognized in the defined symbol database, the image processor (4) enables to make call towards the mobile device (21) by warning the access layer (3) with the address of the mobile device (21) to be connected via the control unit (6). If one of the mobile devices (2 and 21) terminates the call both calls and related sources are cleaned.

It is possible to develop a wide variety of embodiments of the inventive image recognition system (1). The invention cannot be limited to the examples described herein and it is essentially according to the claims.

Claims

1. An image recognition system (1) comprising: at least one mobile device (2); at least one access layer (3) which is in communication with the mobile device (2) and has call control component, audio and image access layer, audio and image transfer layer and network control component; at least one image processor (4) which consists of access layer input, picture decoder, symbol database and image mapper part; at least one response processor (5) which cooperates with the image processor (4) and at least one control unit (6) which regulates the communication between the response processor (5) and the image processor (4) and characterized by the steps of:

— carrying out the process which is stated in the record in the symbol database (106);

— audio and image information being transmitted to the mobile device (21) which makes video call by the access layer (3) (107).

2. An image recognition system (1) according to Claim 1 , characterized by access layer (3) having: call control component which detects the call sent by the mobile device (2), access layer which opens the related media channel, access layer which decodes the call, audio and image access layer which separates the call into its audio and image components, audio and image transfer layer which transfers the audio and the image to the related network component and network control component which controls the audio and image transfer layer.

3. An image recognition system (1) according to Claim 1, characterized by image processor (4) having components of: network access layer input which receives the audio and the image from the access layer (3), picture decoder which separates the audio and image information into its singular components, defined symbol database which contains defined patterns in itself, image mapper which searches for defined patterns within every picture decoded, video database which has videos, time signal which is a time signal generator, video player for playing video, network access layer output which transfers the obtained audio and image to the access layer (3).

4. An image recognition system (1) according to Claim 1 , characterized by response processor (5) having components of: other remote access which has capability of remote function call, journal record, messaging which sends e- mail/MMS/SMS, content enrichment which enrich content of the image recognized.

5. An image recognition system (1) according to any of the preceding claims, characterized in that it uses H.324m protocol in the access layer (3).

6. An image recognition system (1) according to any of the preceding claims, characterized in that it uses TDM protocol in the access layer (3).

7. An image recognition system (1) according to any of the preceding claims, characterized in that it uses web service as remote access component.

8. An image recognition system (1) according to any of the preceding claims, characterized in that it uses CORBA (Common Object Request Broker Architecture) as remote access component.