KR102254975B1 - Method for searching images based on tag, apparatus and system for the same - Google Patents

Abstract

The present invention relates to a tag-based image search method, an apparatus and a system for the same, and is to enable a desired image to be easily searched among chat contents existing in a chat window of a messenger program. According to the present invention, when an image is output on a chat window when a terminal performs a conversation through a running messenger program, the image is tagged based on meta information of the image and location information of the image. In addition, when a search word is input to a search box linked to the chat window, the terminal extracts an image having a tag including the input search word among the images included in the chat window and outputs it on the screen.

Description

Tag-based image search method, apparatus and system for the same {Method for searching images based on tag, apparatus and system for the same}

The present invention relates to a messenger service, and more specifically, an image input to a chat window when performing a conversation through a messenger is tagged based on location information and meta data, and a tag tagged to the image The present invention relates to a tag-based image search method for performing image search in a chat window based on, and an apparatus and system for the same.

Due to the rapid development of information and communication technology, various types of terminals such as mobile communication terminals and personal computers are being implemented to perform various functions.

Taking a mobile communication terminal as an example, the mobile communication terminal goes beyond basic voice communication functions and, in recent years, enables various functions such as data communication, taking pictures or videos using a camera, playing music or video files, playing games, and watching broadcasts. It is being implemented.

Technology development for increasing functions that can be executed in such a terminal is continuously being made, and it is continuously being made mainly on hardware improvement technology as well as software improvement technology.

On the other hand, in recent years, the frequency of use of a messenger program capable of conducting a conversation has continuously increased by transmitting and receiving messages with a plurality of users after accessing a communication network using such a terminal.

Through such a messenger program, a user creates a text message and transmits the generated text message to the other terminal to communicate with the other terminal, but recently, a captured image, a photographed image, and a stored image are used. The frequency is increasing.

In addition, in the chat window of the messenger program, chat contents such as text messages and videos exchanged with the other party are mixed according to the input time sequence. Unless the user deletes the conversation content, the conversation content exchanged with the other party remains in the conversation window.

Therefore, when a user wants to find a specific content among the content exchanged with the other party, there is an inconvenience of having to find the content while scrolling the chat window directly.

Korean Patent Registration No. 10-0932484, registered on December 09, 2009 (Name: How to insert a short message in the image of a mobile communication terminal)

In order to solve this problem, a method of providing a text search function to a chat window of a messenger program may be considered.

However, the text search function has a problem that it can only search text messages among the conversation contents of the chat window, and it cannot search for images. For this reason, in order to find a specific video in the chat window, there is still the hassle of having to scroll through the chat window one by one.

Accordingly, an object of the present invention is to provide a tag-based image search method capable of searching for a desired image through text search in a chat window of a messenger program, and an apparatus and system for the same.

Another object of the present invention is to tag an image input into a chat window based on location information and meta-data when performing a chat through a messenger, and an image within the chat window based on the tag tagged on the image. It is to provide a tag-based image search method for performing a search, and an apparatus and system for the same.

Another object of the present invention is to provide a tag-based image search method, an apparatus and a system for the same, capable of searching not only a desired text message but also an image through text search in a chat window of a messenger program.

In order to achieve the above object, the present invention provides a terminal including an output unit and a control unit. The output unit outputs the contents of the conversation that is input when conducting the conversation through the messenger program. In addition, the control unit is functionally connected to the output unit, executes a messenger program, and when an image is output to the output unit as the conversation content while performing a conversation through the running messenger program, the image is displayed based on the meta information of the image and the location information of the image. When tagging is performed and a search word is input to a search box linked to the chat window, an image having a tag including the search word is extracted from the images included in the chat window and outputted through the output unit.

In the terminal according to the present invention, the controller acquires coordinate information on the current location of the terminal, acquires the place name or building name of the current location based on the acquired coordinate information, and then the acquired place name or building name. You can use it as a tag for the video and tag it.

In the terminal according to the present invention, the control unit extracts the place name, building name, or name included in the text message located near the location where the image is displayed in the chat window, and converts the extracted place name, building name, or name into the corresponding image. You can use it as a tag for tagging.

The terminal according to the present invention may further include a storage unit for storing the tagged image, meta information of the image, and location information together under the control of the controller.

The terminal according to the present invention may further include a communication unit for transmitting and receiving information generated through a messenger program with one or more counterpart terminals by interworking with a service device through a communication network.

At this time, the control unit receives an image from the counterpart terminal through the communication unit and outputs it to the chat window, but receives the current location information of the counterpart terminal from the service device and outputs the video, current location information of the received counterpart terminal, and meta information of the output video. Can be stored in the storage unit together.

Alternatively, the control unit may transmit an image to the other terminal through a communication network and output it to a chat window, to determine the current location information of the terminal, and transmit the determined current location information to the other terminal through the service device together with the image.

The present invention also provides a service device including a service communication unit, a service control unit, and a service storage unit. The service communication unit transmits and receives various information through one or more terminals and a messenger program. The service control unit is functionally connected to the service communication unit, and when an image is received from the terminal running the chat window of the messenger program through the service communication unit, the image is tagged and stored based on the meta information of the image and the location information of the image, and the chat window When a search word is input in the search window linked to, an image having a tag including the search word is extracted from the images included in the chat window and transmitted to the terminal. In addition, the service storage unit stores the tagged image, meta information of the image, and location information together under the control of the service controller.

The present invention also provides a service system including a terminal and a service device. When performing a conversation through a running messenger program, the terminal tags the video based on the meta information of the video and the location information of the video, and when a search word is input in the search window linked to the chat window, Among the images included in the chat window, an image having a tag including a search word is extracted and displayed on the screen. In addition, the service device relays the delivery of the conversation contents according to the conversation between the terminal and the counterpart terminal through the communication network.

The present invention also provides a step of tagging an image based on meta information of the image and location information of the image when an image is output in a chat window when performing a chat through a messenger program running on the terminal, and a search word in a search box linked to the chat window. When is input, a tag-based image search method is provided that includes, by the terminal, extracting an image having a tag including a search word from among images included in a chat window and outputting it to a screen.

In the image search method according to the present invention, in the tagging step, the location information included in the tag of the image is a place name, a building name, and an image based on coordinate information on the current location of the image transmitting side. It may include a place name, a building name, or a name included in a text message located near the displayed location.

In the image search method according to the present invention, the image may include a captured image, a captured image, or an image stored in a terminal.

In the image search method according to the present invention, the tagging may further include storing, by the terminal, the image, meta information of the image, and location information.

In the image search method according to the present invention, in the storing step, the terminal may store the tagged image.

In the image search method according to the present invention, the storing includes: inputting an image to be transmitted from the terminal to the other terminal into a chat window, the terminal determining the current location information of the terminal, and the image input by the terminal, the determined terminal It may include storing the current location information and meta information of the input image together.

In the image retrieval method according to the present invention, the storing includes: receiving, by a terminal, an image from a counterpart terminal through a service device and outputting it to a chat window, receiving, by the terminal, current location information of the counterpart terminal from the service device, And storing the image output by the terminal, current location information of the received counterpart terminal, and meta information of the output image together.

In the image retrieval method according to the present invention, in the storing step, the content of a conversation including a text message and an image input to a conversation window when the terminal performs a conversation through a messenger program may be stored.

In the image search method according to the present invention, in the outputting step, when there are a plurality of extracted images, the terminal may sequentially output the extracted images to the screen according to the input time sequence.

In the image search method according to the present invention, in the outputting step, the terminal may extract an image and a text message including a search word and output it on a screen.

In the image search method according to the present invention, in the outputting step, when there are at least one extracted image and text message, the terminal may sequentially output the extracted image and text message to the screen according to the input time sequence. .

Further, the present invention provides a computer-readable recording medium in which a program for executing the above-described image retrieval method is recorded.

According to the present invention, since an image included in the chat content of a chat window can be tagged based on location information and meta information, and a search for a corresponding image in the chat window can be performed based on the tag tagged on the image, the user can You can easily search for a desired video through text search in the chat window.

Furthermore, the location information included in the video tag is included in the text message located near the center of the place name or building name based on the coordinate information of the current location of the video transmitting party, and the display location of the video displayed in the chat window. Since the location name, building name, name, etc. are included, the user can easily search for a desired image through text input for a location that can be intuitively reminiscent of the corresponding image.

In addition, according to the present invention, not only a desired text message but also an image can be searched through text search in a chat window of a messenger program.

1 is a schematic block diagram showing a messenger system according to an embodiment of the present invention.
2 is a block diagram showing a main configuration of the terminal of FIG. 1.
3 is a block diagram showing a main configuration of the service device of FIG. 1.
4 is a flowchart illustrating a tag-based image search method according to an embodiment of the present invention.
5 is a detailed flowchart of an example of the storage step of FIG. 4.
6 is a detailed flowchart of another example of the storage step of FIG. 4.
7 to 9 are exemplary screen views according to a tag-based image search method according to an embodiment of the present invention.
10 is a flowchart illustrating a tag-based image search method according to another embodiment of the present invention.
11 is an exemplary screen diagram according to a tag-based image search method according to another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in which a person of ordinary skill in the art can easily implement the present invention. However, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention in describing the operating principle of the preferred embodiment of the present invention in detail, the detailed description thereof will be omitted. This is to convey more clearly without obscuring the core of the present invention by omitting unnecessary description. In addition, terms such as first and second are used to describe various elements, and are used only for the purpose of distinguishing one element from other elements, and are not used to limit the elements.

In addition, the same reference numerals are used for portions having similar functions and functions throughout the drawings.

In addition, the terminal of the present invention can be implemented in various forms. For example, the terminal described in this specification is a smart TV as well as mobile terminals such as a smart phone, a tablet PC, a personal digital assistant (PDA), a portable multimedia player (PMP), and an MP3 player. A fixed terminal such as a (Smart TV) or a desktop computer may be used, and any device can be applied as the terminal of the present invention as long as it is a terminal capable of transmitting and receiving messages with other users through a communication network.

Hereinafter, a messenger system according to an embodiment of the present invention will be described.

1 is a schematic block diagram showing a messenger system according to an embodiment of the present invention.

Referring to FIG. 1, the messenger system 100 according to the present embodiment may include a plurality of terminals 10 and service devices 20 connected via a communication network 30.

The plurality of terminals 10 are devices that can transmit/receive messages or transmit/receive content to and from one or more counterpart terminals through a messenger program, and refer to a device that can be used by a user.

Here, the messenger program may be provided by the service device 20 and may be a program capable of providing an instant messaging service. Such a messenger program can be downloaded from the service device 20 and installed in the terminal 10, and even if the terminal 10 does not have a messenger program installed, the messenger program can be accessed by connecting to the service device 20 and using a cloud service method. You can also use it.

The terminal 10 may execute a messenger program and display a messenger program execution screen (eg, an initial screen, a chat window screen, a search window screen, etc.). Here, the messenger program may be executed according to a user's request or when a specific event occurs. For example, a messenger program of the terminal 10 may be executed when a specific event occurs, such as when a message is received from a counterpart or when specific information is received from the service device 20.

The terminal 10 according to the present embodiment transmits/receives conversation contents for conducting a conversation with one or more designated counterpart terminals through a conversation window of a running messenger program. Here, for convenience of explanation, the first terminal 10a is a terminal used by the user, and the second terminal 10b is a terminal of the other party capable of transmitting and receiving conversation contents with the user of the first terminal 10a through a messenger program. It is assumed that it is described, but is not limited thereto.

When the conversation content is displayed in the conversation window when the conversation is performed through the running messenger program, the terminal 10 stores the conversation content. In particular, when an image is output from the content of the conversation, the terminal 10 tags the image based on meta information of the image and location information of the image. When a search word is input to a search box linked to a chat window, the terminal 10 may extract an image having a tag including the search word from among the images included in the chat window and output it to the search window screen.

In addition, when a search word is input to a search box linked to a chat window, the terminal 10 may extract an image and a text message having a tag including the search word from among text messages and images included in the chat window and output it to the search window screen.

Here, the conversation content includes text messages and images exchanged between the user and the other party, and may include various contents such as various icons such as emoticons, flashicons, stickers, and voice messages.

The service device 20 relays the delivery of conversation contents according to the conversation between a plurality of terminals 10 through the communication network 30. The service device 20 serves to support transmission and reception of conversation contents between a plurality of terminals 10 through a messenger program. For example, when the content of a conversation is input by the user of the first terminal 10a, the service device 20 transmits it to one or more second terminals 10b connected through the chat window screen of the messenger program, and the second terminal 10b When the content of the conversation is received from, the entire process of transmitting the content to the first terminal 10a may be controlled.

In addition, content of conversations transmitted and received through the messenger program of the terminal 10 may be stored in the terminal 10, but is not limited thereto. For example, the service device 20 may store and manage the corresponding conversation content. That is, the service device 20 may store the contents of a conversation being transmitted/received through a current chat window according to a user's request, and provide it to the terminal 10. In other words, while the first terminal 10a and one or more second terminals 10b are connected through the chat window of the messenger program, a user input corresponding to a preset condition on the chat window screen of the first terminal 10a, for example, When a swipe motion in one direction is detected, the content of the conversation transmitted and received through the conversation window can be stored, and the stored conversation content can be provided at any time according to the user's request.

In addition, when an image is received from the terminal 10 on which the chat window of the messenger program is running, the service device 20 may tag and store the image based on meta information of the image and location information of the image. When a search word is input into a search box linked to the chat window, the service device 20 may extract an image having a tag including the search word from among the images included in the chat window and transmit the extracted image to the terminal 10.

In addition, the user of the terminal 10 may access the service device 20 to check the contents of the conversation he/she has exchanged with the other party and check this. Here, the access may include a process of going through a separate log-in procedure to the service device 20 through the terminal 10. In addition, it may include a process of automatically connecting to the service device 20 even without going through a procedure such as login. In addition, the user of the terminal 10 receives related information from the service device 20 in advance, and can immediately check this through a specific execution screen of the messenger program. When a selection input is sensed, the service device 20 may be directly connected.

In addition, conversation contents transmitted and received through the messenger program may be simultaneously stored in the terminal 10 and the service device 20, and only specific conversation contents may be stored in the service apparatus 20 according to a user's selection.

In addition, in order to provide a messenger program service to a plurality of terminals 10, the service device 20 divides subscribed users by user identifiers, and provides status, reception mode, message recipient list, and It is possible to manage subscriber environment information and manage an IM chat session, that is, a chat channel, for transferring IM chat messages between users who are connected to the Internet (On-line). In addition, the service device 20 may perform SIP (Session Initiation Protocol) call processing and MSRP (Message Session Relay Protocol) relay functions, and the like. 2 It is possible to provide a chat service between the terminals 10b.

The service device 20 may be implemented as a set of various devices to support the above-described operation. For example, a database server capable of storing and managing various related information, a file server for file processing, and a push server capable of automatically pushing information related to the terminal 10 may be implemented.

In addition to this, the service device 20 may be implemented including various servers according to a system design method. In addition, the function corresponding to the server may be implemented in one service device 20 as one module.

A state connected through a chat window according to the present embodiment described above refers to a moving path through which messages can be transmitted and received between a plurality of terminals 10, that is, a state in which a session is connected. For example, an act of creating a chat window by selecting one or more second terminals 10b to which the first terminal 10a wishes to conduct a conversation, and the first terminal 10a sending one or more second messages input from the user. It may be a concept that includes all acts of transmission to the terminal 10b.

In addition, a plurality of terminals 10 and service devices 20 according to the present embodiment transmit and receive various related information through the communication network 30. For example, the communication network 30 includes wireless communication methods such as WLAN (Wireless LAN), Wi-Fi, Wibro, Wimax, and High Speed Downlink Packet Access (HSDPA), or Ethernet, xDSL. Wired communication methods such as (ADSL, VDSL), Hybrid Fiber Coaxial Cable (HFC), Fiber to The Curb (FTTC), and Fiber To The Home (FTTH) can be used. In addition, in addition to the above-described communication method, the communication network 30 may include all types of communication methods that are widely known or will be developed in the future.

The terminal 10 according to the present embodiment will be described in detail with reference to FIGS. 1 and 2 as follows. Here, FIG. 2 is a block diagram showing the main configuration of the terminal 10 of FIG. 1.

1 and 2, the terminal 10 according to the present embodiment includes an output unit 14 and a control unit 15, and further includes a communication unit 11, an input unit 12, and a storage unit 13. It can be configured to include.

The communication unit 11 serves to support transmission and reception of related information with the service device 20 and one or more counterpart terminals through the communication network 30. At this time, the communication unit 11 interworks with the service device 20 through the communication network 30 to transmit and receive information generated through a messenger program with one or more counterpart terminals.

For example, the communication unit 11 may transmit a message input from a user through a chat window of a messenger program to a counterpart terminal connected through a chat channel of the chat window, and receive a message from the counterpart terminal. In particular, the communication unit 11 may receive a video or text message from a counterpart terminal through a messenger program, and can transmit a video or text message to one or more counterpart terminals.

In addition, the communication unit 11 may transmit various request information input from a user to the service device 20. In addition, the communication unit 11 may receive information related to a messenger program from the service device 20.

Here, the image may include a captured image, a captured image, or an image stored in the terminal 10. Of course, the image may be a moving image or a still image.

The input unit 12 transmits various information such as number and text information input from a user, various function settings, and signals input in connection with function control of the terminal 10 to the control unit 15. The input unit 12 executes a messenger program and supports input of a message from a user through the messenger program. The input unit 12 supports input of a user's preset operation (touch, drag, swipe, etc.) for inputting an image in a chat window of a messenger program. The input unit 12 supports input of a user's preset operation for inputting a textual search word in a search window of a messenger program.

Here, the user's preset operation may be input by at least one of a keypad and a touch pad that generate an input signal according to a user's touch or manipulation. That is, when the input unit 12 is configured in the form of a single touch panel (or touch screen) together with the output unit 14, the input of a user's touch on the touch panel and a drag or swipe operation in a specific direction is sensed. can do.

In addition, the input unit 12 may detect an input such as an arrow on a keypad input from a user. Alternatively, the input unit 12 may support input of a user's voice, such as “left drag” and “right drag”. Alternatively, when the input unit 12 includes a gesture input means, the input unit 12 may detect a user's preset gesture.

As described above, the input unit 12 includes at least one of a key input means such as a keyboard or a keypad, a touch input means such as a touch sensor or a touch pad, a voice input means, a gyro sensor, a geomagnetic sensor, an acceleration sensor, a proximity sensor, and a camera. It may also include a gesture input means including one or more.

In addition to this, the input unit 12 may include all types of input means that are currently under development or may be developed in the future.

The storage unit 13 may store various data generated during execution of the application program, including an application program required for functional operation of the terminal 10. The storage unit 13 may store a messenger program, and may store conversation contents transmitted and received with one or more counterpart terminals through the messenger program.

The storage unit 13 may largely include a program area and a data area. The program area stores information related to driving the terminal 10 such as an operating system (OS) that boots the terminal 10. The data area is an area in which data generated according to the use of the terminal 10 is stored, and as described above, various information generated during execution of a messenger program is stored. For example, when storing an image among conversation contents, the storage unit 13 also stores the image, meta information of the image, and location information of the image. In addition, the storage unit 13 stores a tag tagged to a corresponding image under the control of the controller 15.

The storage unit 13 is a flash memory, a hard disk, a multimedia card micro-type memory (eg, SD or XD memory, etc.), RAM, and ROM. It may be configured to include a storage medium such as.

Meanwhile, an example in which the storage unit 13 stores a messenger program or conversation contents transmitted/received through a messenger program is disclosed, but the present invention is not limited thereto. For example, the content of the conversation may be stored in the service device 20 by interlocking with the service device 20 according to a cloud service method.

The output unit 14 plays a role of outputting information about a series of operation states and operation results that occur while the terminal 10 performs a function. The output unit 14 may include a display module (not shown) and a sound source output module (not shown).

Among them, the display module (not shown) plays a role of displaying various information generated during execution of the messenger program. For example, the content of the conversation in the conversation window under the control of the control unit 15 and the information input in the search window linked to the conversation window. Search results according to the search word can be output and displayed.

Such a display module (not shown) may be implemented in the form of a single touch panel (or touch screen) together with the input unit 12 as described above, and when implemented together with the input unit 12, a user's preset operation , A drag-and-drop operation, and a variety of information generated by scrolling can be displayed.

For example, display modules (not shown) include LCD (Liquid Crystal Display), TFT-LCD (Thin Film Transistor LCD), OLED (Organic Light Emitting Diodes), Light Emitting Diodes (LED), AMOLED (Active Matrix Organic LED), and Flexible Display ( Flexible display) or 3D display

The sound source output module (not shown) serves to convert a sound source, which is an electrical signal, into an analog signal, and outputs a voice message under the control of the controller 15.

The control unit 15 performs overall control of the terminal 10, and may be an operating system (OS) and a process device that drives each component, for example, a central processing unit (CPU). .

The control unit 15 is functionally connected to each component. That is, the control unit 15 is functionally connected to the communication unit 11, the input unit 12, the storage unit 13, and the output unit 14, and controls the flow of signals for supplying power and performing functions to each component. .

The controller 15 may execute a messenger program and control execution of various functions generated through the messenger program. Basically, when a user's request or occurrence of a preset event is detected, the controller 15 controls the messenger program to be executed, and controls the messenger program execution screen to be displayed on the output unit 14.

In addition, the controller 15 may select a specific counterpart according to a user's request through a messenger program, and support an overall process for transmitting/receiving conversation contents with the selected counterpart. In addition, the controller 15 may support various operations for performing various functions provided by the messenger program.

In particular, the control unit 15 stores conversation contents exchanged with the other party through the conversation window of the messenger program in the storage unit 13, and among the conversation contents, an image may be tagged and stored. That is, when an image is output to the output unit 14 as a conversation content while performing a conversation through a running messenger program, the controller 15 may tag the image based on meta information of the image and location information of the image.

At this time, the tag assigned to the image includes a name given so that the user can intuitively and easily find the image to be searched among the images existing in the chat window through input of a search word such as text.

The reason why location information such as place name, building name, or name is included in the tag of the video is to allow the user to easily find the video. When a search is performed through a search window, a textual search word is entered in the search box of the search window to perform a search. This is because, as the search word that the user can input, words that can be easily reminiscent of the image to be searched, such as a place name, a building name, and a name, are mainly used.

For example, the meta information of the video is additional information about the video. For example, if the image is a picture taken with a digital camera, it includes information on the digital camera itself, time at the time of shooting, exposure, whether flash is used, resolution, photo size, name (name) given to the photo, etc. It may further include location information at the time of shooting acquired using the GPS function.

Accordingly, the controller 15 may tag the corresponding image using meta information of the image as a tag.

The location information included in the video tag is the name of the place or building based on the coordinate information of the current location of the person transmitting the video, and the location included in the text message located near the location where the video is displayed in the chat window. It can include people or building names.

Therefore, through the communication unit 11 or a GPS module (not shown) provided in the terminal 10, the control unit 15 may obtain coordinate information on the current location of the terminal 10. The controller 15 may obtain a place name or a building name of the current location based on the obtained coordinate information. The controller 15 may use the acquired place name or building name as a tag of a corresponding image.

Alternatively, when an image is input while performing a conversation through a chat window, in general, the user or the other party inputs a description of the image as a text message before or after inputting the image.

Accordingly, the control unit 15 may use a place name, a building name, a name, etc. included in a text message located near a location where an image is displayed in the chat window as a tag. In this case, the name may be the name of a specific object or a person's name.

In addition, when a search word is input to a search box linked to the chat window, the controller 15 extracts an image having a tag including the search word from among the images included in the chat window and outputs it through the output unit 14. Of course, the control unit 15 outputs the extracted image to the search window.

At this time, the user may input the image to the chat window, or the other party may input the image. First, when receiving an image from a counterpart terminal through the communication unit 11 and outputting it to a chat window, the controller 15 receives current location information of the counterpart terminal from the service device 20. In addition, the controller 15 may store the output image, current location information of the received counterpart terminal, and meta information of the output image together in the storage unit 13.

When an image is transmitted to a counterpart terminal through the communication unit 11 and output to a chat window, the control unit 15 determines the current location information of the terminal 10. In addition, the controller 15 may store the output image, current location information of the determined terminal 10, and meta information of the output image together in the storage unit 13. At this time, the controller 15 may transmit the identified current location information of the terminal 10 together with the image to the counterpart terminal through the service device 20.

When there are a plurality of extracted images, the controller 15 may sequentially output the extracted images to the search window according to the input time sequence. The controller 15 may output a search result equal to the number of extracted images to the search window. For example, the control unit 15 may output a search result in the form of “search for ** case for a search term”.

In addition, the controller 15 may extract a text message including a search word as well as the image searched in the search window and output it to the search window. In this case, when there are at least one extracted image and text message, the controller 15 may sequentially output the extracted image and text message to the search window according to the input time sequence. The controller 15 may output a search result such as the number of extracted images and text messages in a search window.

In addition, the controller 15 may set a search range through selection of a search menu in the search window. For example, when the user selects a search range among text messages and images through the input unit 12, the controller 15 may search for a chat window in the selected search range and output the search result to the search window.

In order to control the above operation, the controller 15 may include a messenger control module 16 and an image tagging module 17. The messenger control module 16 may perform a function of controlling overall execution of a messenger program, such as executing a messenger, generating a chat window, generating a search window, and outputting a searched video or text message. The image tagging module 17 controls a tagging process of an image displayed in a chat window, and may perform a function of controlling the entire process of storing the tagged image in the storage unit 13 together with location information and meta information.

The service device 20 according to the present embodiment will be described in detail with reference to FIGS. 1 to 3 as follows. Here, FIG. 3 is a block diagram showing the main configuration of the service device 20 of FIG. 1.

1 and 3, the service device 20 of the present invention may include a service communication unit 21, a service control unit 25, and a service storage unit 23.

The service communication unit 21 may perform a role of transmitting and receiving various information related to one or more terminals 10 and a messenger program. The service communication unit 21 may receive conversation contents transmitted/received between any one terminal 10 and another terminal 10 connected to each other. In addition, the service communication unit 21 may receive request information for various types of information from any one terminal 10 and provide the information to the corresponding terminal 10.

The service control unit 25 performs overall control of the service device 20. The service control unit 25 may perform various controls related to a messenger program.

For example, in order to provide a messenger program service to a plurality of terminals 10, the service control unit 25 classifies subscribed users by user identifiers, and provides status, reception mode, message recipient list, and subscribers for each subscribed user. It manages environmental information and manages an IM chat session, that is, a chat channel, for delivering IM chat messages between users who are connected to the Internet (On-line).

In addition, the service control unit 25 can perform SIP (Session Initiation Protocol) call processing and MSRP (Message Session Relay Protocol) relay functions, etc., through which a plurality of terminals 10, for example, the first terminal 10a and the 2 It is possible to transmit and receive messages or contents between the terminals 10b.

Meanwhile, as described above, an example in which the terminal 10 performs tagging and stores an image output on a chat window has been disclosed, but is not limited thereto. That is, the service device 20 may directly tag an image included in the conversation content and then store it. When a video search is requested through the search window according to the chat window of the terminal 10, the service device 20 extracts the video through text search based on the tag assigned to the video, and the search window of the terminal 10 Can be provided to.

That is, when the service control unit 25 receives an image transmitted from any one terminal 10 on which the messenger program is running, for example, the first terminal 10a, to any one other terminal, for example, the second terminal 10b, Images can be tagged based on the meta information of the image and the location information of the image. When a search word is input into the search box linked to the chat window of the terminal 10, the service control unit 25 extracts a text message or video having a tag including the search word among text messages or images included in the chat window, and the terminal 10 Can be provided to.

In addition, when an image transmitted from the first terminal 10a to the second terminal 10b is received, the service control unit 25 determines the current location of the first terminal 10a when transmitting the image to the second terminal 10b. After identifying the information, the determined current location information and the corresponding image may be transmitted together.

Meanwhile, since the tagging in the service control unit 25 can be performed in the same manner as the tagging of the terminal 10, a detailed description of the tagging will be omitted.

In this way, the service control unit 25 includes a messenger control module 26 and an image tagging module 27 similar to the control unit of the terminal 10 in order to provide a messenger service, image tagging, and a search environment based on tags. It can be configured. The messenger control module 26 may perform various functions for providing and controlling a messenger program between a plurality of terminals 10. The image tagging module 27 controls a tagging process of an image displayed in a chat window, and may perform a function of controlling the entire process of storing the tagged image in the service storage unit 23 together with location information and meta information. .

The service storage unit 23 stores various types of information generated while using the messenger program under the control of the service control unit 25. That is, the service storage unit 23 may store conversation information exchanged between a plurality of terminals 10. The service storage unit 23 may store the tagged image, meta information of the image, and location information of the image together. In addition, the service storage unit 23 may store conversation information corresponding to user identification information of the terminal 10.

The service storage unit 23 may be one storage in the messenger system 100 or a separate data storage device located outside the messenger system 100 and capable of transmitting and receiving data to and from the service device 20.

In addition, the service device 20 may be implemented as one or more servers operating in a server-based computing method or a cloud method. In particular, information transmitted and received through the messenger system 100 may be provided through a cloud computing function that may be permanently stored in a cloud computing device on the Internet. Here, cloud computing is a virtualized IT (Information Technology) resource, for example, hardware (server, storage, network, etc.), software by utilizing Internet technology in digital terminals such as desktops, tablet computers, laptops, netbooks, and smartphones. It refers to a technology that provides services (database, security, web server, etc.), services, and data in an on-demand manner. In the present invention, all information transmitted and received between the plurality of terminals 10 and the service device 20 is stored in a cloud computing device on the Internet, and can be transmitted anytime, anywhere.

Meanwhile, a memory mounted on the terminal 10 or the service device 20 may store information within the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

Although the present specification and drawings describe exemplary device configurations, the functional operations and implementations of the subject described in this specification may be implemented with other types of digital electronic circuits, or the structures disclosed herein and structural equivalents thereof. It may be implemented with computer software, firmware, or hardware, or may be implemented by a combination of one or more of them. Implementations of the subject matter described herein are one or more computer program products, that is, one relating to computer program instructions encoded on a tangible program storage medium for execution by or for controlling the operation of a device according to the invention. It can be implemented as the above module. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of materials that affect a machine-readable radio wave signal, or a combination of one or more of them.

A tag-based image search method in the service system 100 according to the present embodiment will be described with reference to FIGS. 1 to 4 as follows. 4 is a flowchart illustrating a tag-based image search method according to an embodiment of the present invention.

1 to 4, first in step S51, the terminal 10 executes a messenger program and outputs a chat window on the screen. That is, the terminal 10 executes a messenger program and outputs a contact list on the execution screen. When at least one conversation partner is selected from the conversation partner list according to a user's selection signal, the terminal 10 outputs a conversation window on the screen. At this time, the conversation window can be called in various ways through the input unit.

Next, in step S53, the terminal 10 performs a conversation while outputting the conversation content input through the conversation window. That is, people who participate in the chat window through the plurality of terminals 10 perform a chat with the other party while inputting the chat contents through the chat window. The service device 20 relays the contents of a conversation between a plurality of terminals 10.

Next, in step S55, the terminal 10 determines whether or not an image is displayed on the chat window while the chat is being performed. That is, the terminal 10 determines whether or not an image is output on the chat window as the chat content. Of course, the image may be provided by the first terminal 10a or the second terminal 10a.

If the image is not output as a result of the determination in step S55, the terminal 10 maintains the conversation execution state according to step S53.

On the other hand, when an image is output as a result of the determination in step S55, the terminal 10 stores the output image, meta information of the image, and location information together in step S57.

In this case, the storage according to step S57 may be performed in the manner of FIGS. 5 and 4 depending on whether the image is input from the first terminal 10a that is the user terminal or the second terminal 10b that is the counterpart terminal. Here, FIG. 5 is a detailed flowchart of an example of the storage step of FIG. 4. 6 is a detailed flowchart of another example of the storage step of FIG. 4.

First, FIG. 5 is a case in which an image is input to a chat window from the first terminal 10a.

Referring to FIG. 5, first, in step S571, the first terminal 10a inputs an image to be transmitted to the second terminal 10b into a chat window.

Subsequently, in step S573, the first terminal 10a determines its current location. That is, the terminal 10 may obtain coordinate information on the current location of the terminal 10 through a communication unit or a GPS module (not shown). The terminal 10 may acquire a place name or a building name of the current location based on the acquired coordinate information. The terminal 10 may use the acquired place name or building name as a tag of the corresponding image.

Further, in step S575, the first terminal 10a stores the input image, its own current location information, and meta information of the input image together.

6 is a case in which an image is input to a chat window from the second terminal 10b.

1 and 6, first, in step S572, the first terminal 10a receives an image from the second terminal 10b through the service device 20 and outputs it to a chat window. That is, when the second terminal 10b inputs an image to be transmitted to the first terminal 10a in the chat window, the service device 20 receives the image and transmits it to the first terminal 10a. The first terminal 10a outputs the received image to a chat window.

At this time, the image received by the first terminal 10a includes meta information of the corresponding image.

Subsequently, in step S574, the first terminal 10a receives the current location information of the second terminal 10b from the service device 20. For example, the service device 20 may receive current location information of the second terminal 10b from the second terminal 10b and transmit it to the first terminal 10a.

In step S576, the first terminal 10a stores the image output to the chat window, the current location information of the received second terminal 10b, and meta information of the output image together.

Next, in step S59, the terminal 10 tags an image based on meta information and location information of the image. In this case, tagging is to attach a keyword or tag, which is a tag for search that can represent the content of the video, and was used to provide information in the form of text that enables users to easily find the video. The location information of the video includes the name of the place or building based on the coordinate information of the current location of the person transmitting the video, the name of the place included in the text message located near the location where the video is displayed in the chat window, and the name of the building. Or it can include a name.

For example, the terminal 10 acquires coordinate information on the current location of the terminal 10, and acquires a place name or a building name of the current location based on the acquired coordinate information. In addition, the terminal 10 may tag the acquired place name or building name as a tag of the corresponding image.

Alternatively, the terminal 10 extracts a place name, a building name, or a name included in a text message located in the vicinity of the location where the image is displayed in the chat window. In addition, the terminal 10 may tag the extracted place name, building name, or name as a tag of the corresponding image. In this case, the text message located in the vicinity of the position where the image is displayed may include 1 to 5 text messages vertically based on the position where the image is displayed, but is not limited thereto.

Of course, in step S59, the terminal 10 stores the tagged image.

Meanwhile, in the method according to the present embodiment, an example in which the terminal 10 is tagged after storage has been disclosed, but is not limited thereto. For example, after the terminal 10 completes the tagging, the tagged image, meta information of the image, and location information may be stored together.

Next, in step S61, the terminal 10 determines whether or not the search window linked to the chat window is output. That is, when the user wants to search for conversation contents in the conversation window, according to the user's selection signal, the terminal 10 may call a search window linked to the conversation window and output it on the screen. The search window can be called in various ways through the input unit.

If the search window is not displayed as a result of the determination in step S61, the terminal 10 maintains the conversation execution state according to step S53.

On the other hand, when the search window is output as a result of the determination in step S61, the terminal 10 receives a search word from the user in step S63. In this case, the search word may be in the form of text.

In addition, when a search request signal is input after the search word is input, the terminal 10 extracts an image having a tag including the search word and outputs it on the screen in step S65. That is, the terminal 10 extracts an image having a tag including a search word from the stored conversation contents, and outputs the extracted image to the search window. When there are a plurality of extracted images, the terminal 10 may sequentially output the extracted images to the search window according to the input time sequence. At this time, the conversation content may be stored in the terminal 10 or the service device 20.

The tag-based image search method according to the present embodiment will be described below with reference to exemplary screens of FIGS. 7 to 9. 7 to 9 are exemplary screen views according to a tag-based image search method according to an embodiment of the present invention.

7 shows an example of a chat window 70 of a messenger program in a user terminal, that is, a first terminal.

The chat window 70 may include an upper part 71, a middle part 73, and a lower part 75. Information on the user of the counterpart terminal connected through the chat window 70, that is, the second terminal may be displayed on the upper part 71.

In the interruption 73, various types of conversation contents exchanged between the first and second terminals under the control of the service device through the conversation window 70 may be displayed.

And at the bottom 75, a content sharing screen display menu 72 for sharing various information, such as photos and music, with a second terminal, a menu 74 for transmitting emoticons, stickers, etc. to the second terminal, An input field 76 and a transmission button 79 for inputting a message may be displayed.

The user of the first terminal may input a text message, an image, etc. into the input field 76 through various menus provided at the bottom 75 of the chat window 70. The content input by the user of the first terminal may be displayed on the right side of the middle 73, and the content input by the user of the second terminal may be displayed on the left side of the middle 73.

In this way, the terminal user can exchange various types of conversation contents through the conversation window 70.

At this time, as described above, when an image is input among the conversation contents, the first terminal tags and stores the image. For example, the first terminal user disclosed an example of taking a picture at "Museum A" and inputting it into the chat window 70. In this case, the first terminal may acquire coordinate information obtained by taking a picture of “Museum A” through the GPS module, and then obtain a place name of “Museum A” in which the current location of the user is “A Museum” by using the acquired coordinate information. The first terminal may tag the picture of “Museum A” by including the name of the place of “Museum A”. Of course, the first terminal may store the tagged picture of “Museum A”, meta information of the picture, and location information together.

Alternatively, the first terminal 10 extracts the name of the place “Museum A” included in the text message located near the location where the picture of “Museum A” is displayed in the chat window. In addition, the first terminal may tag the extracted place name of "Museum A" by using it as a tag of the corresponding image. Of course, the first terminal can store the tagged picture of “Museum A”.

On the other hand, if there is a matter to be searched among the contents of the chat window 70, the user calls the search window 80 and outputs it on the screen, as shown in FIG. 8, while the corresponding chat window 70 is displayed on the screen. can do. A search box 81 through which text can be input is provided in the search box 80.

For example, when a user enters a search word in the form of text “Museum A” in the search field 81 and performs a search on the first terminal, the first terminal has a tag for “Museum A” as shown in FIG. 9. The picture of the “Museum A” in which there is a picture may be extracted from the conversation contents of the corresponding conversation window 70 and displayed on the search output window 85 of the search window 80.

In the search window 80, information about the number of searches between the search field 81 and the search output window 85 may be displayed in the summary information field 83. For example, if there is one extracted image for “Museum A”, the first terminal may display “search for one case for Museum A” in the summary information field 83.

Meanwhile, in the tag-based image search method according to the embodiment of the present invention described above, an example of searching only an image based on a tag has been disclosed, but the present invention is not limited thereto. For example, as shown in FIG. 10, the terminal may search and provide a text message along with an image matching a search word.

10 is a flowchart illustrating a tag-based image search method according to another embodiment of the present invention.

1 and 10, in the tag-based image search method according to another embodiment of the present invention, steps S51 to S63 are the same as the tag-based image search method according to the embodiment of the present invention shown in FIG. 4. Proceeds.

Therefore, it will be described in step S67 after step S63.

That is, when a search request signal is input after a search word is input, in step S67, the terminal 10 extracts an image and a text message having a tag including the search word and outputs it on the screen. That is, the terminal 10 extracts an image and a text message having a tag including a search word from the stored conversation contents, and outputs the extracted image and text message to the search window.

In this case, when there are the extracted images and text messages, the terminal 10 may sequentially output the extracted images to the search window according to the input time sequence. At this time, the conversation content may be stored in the terminal 10 or the service device 20.

A tag-based image search method according to another embodiment of the present invention will be described with reference to exemplary screens of FIGS. 7, 8 and 11 as follows. Here, FIG. 11 is an exemplary screen diagram according to a tag-based image search method according to another embodiment of the present invention.

Since the process of inputting the content of the conversation in the chat window and the search word in the search box may be performed in the same manner as in FIGS. 7 and 8, a detailed description thereof will be omitted.

Next, when the user enters “Museum A” in the search field 81 and performs a search on the first terminal, the first terminal is “Museum A” having a tag for “Museum A” as shown in FIG. 11. The text message including the picture of "" and the search word "Museum A" may be extracted from the chat contents of the corresponding chat window 70 and displayed on the search output window 85.

In the search window 80, information about the number of searches between the search field 81 and the search output window 85 may be displayed in the summary information field 83. For example, if the number of extracted images and text messages for "Museum A" is 3, the first terminal may display "Search 3 cases for Museum A" in the summary information field 83.

On the other hand, although the example of this screen shows an example of collectively summarizing and displaying the total number of search cases, the number of search cases for video and text messages may be classified and displayed in the summary information field 83. For example, if the number of extracted images and text messages for "Museum A" is 3, the first terminal may display "Search for 1 photo and 2 text messages for Museum A" in the summary information field 83. .

The image search method based on the tag of the present invention as described above may be provided in the form of a computer-readable medium suitable for storing computer program commands and data. In this case, a computer-readable medium suitable for storing computer program commands and data is, for example, a recording medium such as a hard disk, a floppy disk, and a magnetic medium such as a magnetic tape, a CD-ROM (Compact Disk Read Only Memory). , Optical Media such as DVD (Digital Video Disk), Magnetic-Optical Media such as Floptical Disk, and ROM (Read Only Memory), RAM (RAM) , Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically Erasable Programmable ROM). The processor and memory can be supplemented by special purpose logic circuitry or incorporated into it. Examples of the program instructions may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

On the other hand, the embodiments disclosed in the specification and drawings are merely presented specific examples to aid understanding, and are not intended to limit the scope of the present invention. It is obvious to those of ordinary skill in the art that other modifications based on the technical idea of the present invention may be implemented in addition to the embodiments disclosed herein.

A tag-based image search method according to the present invention, an apparatus and a system for the same, tag an image input into a chat window based on meta information and location information, and a tag assigned to an image through a text search function provided by a messenger program. By performing a text search on the chat window, not only text messages but also images can be searched and provided in the chat contents existing in the chat window.

Therefore, since the user can easily search the video included in the chat window through the text search function of the chat window provided by the messenger service, it is possible to eliminate the hassle of having to scroll through the chat window one by one to find a specific video in the chat window. .

10: terminal 11: communication unit 12: input unit
13: storage unit 14: output unit 15: storage unit
16: messenger control module 17: video tagging module 20: service device
21: service communication unit 23: service storage unit 25: service control unit
30: communication network 100: messenger system

Claims (20)

An output unit for outputting conversation contents input through a messenger program;
When functionally connected to the output unit, the messenger program is executed, and an image is output to the output unit as conversation content through the running messenger program, the image is tagged based on meta information and location information of the image. A control unit that stores in text form and, when a search word is input into the messenger program, extracts an image having a tag including the search word and outputs it through the output unit; Including,
In the chat window of the messenger program, the place name, building name, or name included in the text message located in the vicinity of the location where the image is displayed is extracted, and the extracted place name, building name, or name is used as a tag of the image to be tagged. doing
Terminal, characterized in that. The method of claim 1, wherein the control unit
Acquires coordinate information for the current location of the terminal, acquires the place name or building name of the current location based on the acquired coordinate information, and then uses the acquired place name or building name as a tag of the image to tag it.
Terminal, characterized in that. The method of claim 1, wherein the control unit
Extracting a text message including the search word together with an image having a tag including the search word and outputting it through the output unit
Terminal, characterized in that. A service communication unit for transmitting and receiving information through one or more terminals and a messenger program; And
When an image is functionally connected to the service communication unit and is received from a terminal running the messenger program through the service communication unit, the image is tagged based on the meta information and location information of the image and stored in a text format, A service control unit for extracting an image having a tag including the search word and transmitting it to the terminal when a search word input into the messenger program of the terminal is received; Including,
In the chat window of the messenger program, the place name, building name, or name included in the text message located in the vicinity of the location where the image is displayed is extracted, and the extracted place name, building name, or name is used as a tag of the image to be tagged. doing
Service device, characterized in that. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images