[go: up one dir, main page]

WO2019151548A1 - Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale - Google Patents

Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale Download PDF

Info

Publication number
WO2019151548A1
WO2019151548A1 PCT/KR2018/001365 KR2018001365W WO2019151548A1 WO 2019151548 A1 WO2019151548 A1 WO 2019151548A1 KR 2018001365 W KR2018001365 W KR 2018001365W WO 2019151548 A1 WO2019151548 A1 WO 2019151548A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
user
camera image
communication
face
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2018/001365
Other languages
English (en)
Korean (ko)
Inventor
홍석주
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Line Plus Corp
Original Assignee
Line Plus Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Line Plus Corp filed Critical Line Plus Corp
Priority to PCT/KR2018/001365 priority Critical patent/WO2019151548A1/fr
Priority to KR1020207013958A priority patent/KR102500624B1/ko
Publication of WO2019151548A1 publication Critical patent/WO2019151548A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • the description below relates to techniques provided for identifying information.
  • Examples of personal terminals include smart phones, tablets, wearable devices, notebook computers, etc. These terminals include data communication functions, voice communication functions, camera functions, multimedia player functions, and payment functions. It is equipped with various functions.
  • Korean Laid-Open Patent Publication No. 10-2003-0017043 discloses a memo based on an instant messenger, a P2P based file transfer, a text chat, a voice chat, a video.
  • communication information related to the recognized user may be provided through a camera screen.
  • An information providing method executed in a server comprising at least one processor configured to execute computer readable instructions contained in a memory, wherein the information providing method is performed by the at least one processor for each user.
  • the configuring may include storing communication information including information directly input by the user for communication or information input by another person by specifying the user and matching the user information. can do.
  • the communication information stored in the user DB may include location information at the time when the information is input.
  • the communication information when communication information stored in the user DB is input in the process of consuming a specific content, the communication information may include content reproduction information at the time when the information is input.
  • the information providing method based on the position information of the camera image by comparing the position information of the communication information stored in the user DB and the position information of the camera image by the at least one processor.
  • the method may further include obtaining user information matched with communication information including location information within a predetermined range, wherein the searching comprises: face information and communication information matched with the obtained user information in the user DB. It may include the step of searching.
  • the predetermined range may be determined in consideration of a position coordinate value of a camera of the electronic device and a direction value of the camera image with respect to the camera.
  • the electronic device may track the face position of the recognized face information on the camera image and display communication information of the specific user corresponding to the face position.
  • An information providing method executed in an electronic device wherein the electronic device includes at least one processor configured to execute computer readable instructions included in a memory, wherein the information providing method includes: information, by the at least one processor; Transmitting a camera image as a request for providing to a server; Receiving, by the at least one processor, a face location of a user recognized in the camera image and communication information related to the user from the server; And tracking, by the at least one processor, the face position on the camera image and displaying the communication information corresponding to the face position.
  • the transmitting may include transmitting location information including a position coordinate value of the camera of the electronic device and a direction value of the camera image with respect to the camera, together with the camera image to the server. Can be.
  • the communication information includes information directly input by the user for communication or information input by another person to specify the user, and includes location information at the time when the corresponding information is input, and the reception is performed.
  • the step may include the face location of a specific user recognized in the camera image among the users having communication information including location information within a predetermined range from the server based on the location information of the camera image and associated with the specific user. And receiving communication information.
  • the communication information when the communication information is input in the process of consuming a specific content, the communication information includes content reproduction information at the time when the information is input, the displaying step, the face location Correspondingly, displaying the content reproduction information together with the communication information.
  • a non-transitory computer readable recording medium having recorded thereon a program for executing the information providing method on a computer is provided.
  • An information providing system implemented in a server comprising: a memory; And at least one processor coupled to the memory, the at least one processor configured to execute computer readable instructions included in the memory, wherein the at least one processor comprises, for each user, face information of the user and communication information associated with the user; A user DB configuration unit configured to configure the user DB by matching the user information for identifying the user and storing the same in a user DB; A camera image receiver configured to receive a camera image as a request for providing information from an electronic device; And searching for face information and communication information of a user associated with the camera image in the user DB by using the camera image, and if the face information of a specific user is recognized in the camera image, And an information retrieval unit for transmitting the communication information of the specific user to the electronic device among the face location of the recognized face information and the retrieved communication information.
  • An information providing system implemented in an electronic device, comprising: a memory; And at least one processor coupled to the memory and configured to execute computer readable instructions included in the memory, wherein the at least one processor is configured to transmit a camera image to a server as an information providing request. ; An information receiver configured to receive a face position of a user recognized in the camera image and communication information related to the user from the server; And an information display unit for tracking the face position on the camera image and displaying the communication information corresponding to the face position.
  • FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
  • FIG. 2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention.
  • FIG. 3 is a block diagram illustrating an example of components that may be included in a processor of a server according to an embodiment of the present invention.
  • FIG. 4 is a flowchart illustrating an example of a method that may be performed by a server according to an embodiment of the present invention.
  • FIG. 5 is a block diagram illustrating an example of components that may be included in a processor of an electronic device according to an embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating an example of a method that an electronic device can perform according to an embodiment of the present invention.
  • Figure 7 shows an example of a user DB in an embodiment of the present invention.
  • FIG. 8 to 10 illustrate an example of an information providing process based on face recognition according to an embodiment of the present invention.
  • 11 to 12 illustrate another example of an information providing process based on face recognition according to an embodiment of the present invention.
  • 13 to 15 illustrate another example of an information providing process based on face recognition according to an embodiment of the present invention.
  • Embodiments of the present invention relate to a technique for providing information based on face recognition.
  • Embodiments including those disclosed in detail herein may provide communication information related to a recognized user by recognizing a face captured by a camera through a camera screen, thereby improving efficiency, variety, convenience, cost, and the like. Significant advantages can be achieved in terms of.
  • FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
  • the network environment of FIG. 1 illustrates an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170.
  • 1 is an example for describing the present invention, and the number of electronic devices or the number of servers is not limited as shown in FIG. 1.
  • the plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented as computer systems.
  • Examples of the plurality of electronic devices 110, 120, 130, and 140 include smart phones, mobile phones, navigation systems, computers, notebook computers, digital broadcasting terminals, personal digital assistants (PDAs), and portable multimedia players (PMPs). ), Tablet PCs, game consoles, wearable devices, wearable devices, Internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like.
  • FIG. 1 illustrates the shape of a smartphone as an example of the electronic device 110
  • the electronic device 110 may be substantially different from the network 170 using a wireless or wired communication scheme. It may mean one of various physical computer systems capable of communicating with the electronic devices 120, 130, 140 and / or the servers 150, 160.
  • the communication method is not limited and includes not only a communication method using a communication network (for example, a mobile communication network, a wired internet, a wireless internet, a broadcasting network, a satellite network, etc.) that the network 170 may include, but also a short range wireless communication between devices.
  • a communication network for example, a mobile communication network, a wired internet, a wireless internet, a broadcasting network, a satellite network, etc.
  • the network 170 may include, but also a short range wireless communication between devices.
  • the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And one or more of networks such as the Internet.
  • the network 170 may also include any one or more of network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, trees, or hierarchical networks, but It is not limited.
  • Each of the servers 150 and 160 communicates with the plurality of electronic devices 110, 120, 130, and 140 through the network 170 to provide a command, code, file, content, service, or the like. It may be implemented in devices.
  • the server 150 may be a system that provides a first service to a plurality of electronic devices 110, 120, 130, and 140 connected through the network 170, and the server 160 may also have a network ( It may be a system that provides a second service to the plurality of electronic devices 110, 120, 130, and 140 connected through the 170.
  • the server 150 is a service (for example, an information providing service, etc.) for which the corresponding application is intended through an application as a computer program installed and driven in the plurality of electronic devices 110, 120, 130, and 140. May be provided to the plurality of electronic devices 110, 120, 130, and 140 as a first service.
  • the server 160 may provide a service for distributing the file for installing and driving the above application to the plurality of electronic devices 110, 120, 130, and 140 as a second service.
  • 2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention. 2 illustrates an internal configuration of the electronic device 110 and the server 150 as an example of the electronic device.
  • the other electronic devices 120, 130, 140, or the server 160 may also have the same or similar internal configuration as the above-described electronic device 110 or the server 150.
  • the electronic device 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224.
  • the memories 211 and 221 are non-transitory computer readable recording media.
  • the memories 211 and 221 are non-transitory computer-readable recording media.
  • the memories 211 and 221 are non-transitory computer-readable recording media.
  • the non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, or the like may be included in the electronic device 110 or the server 150 as a separate persistent storage device that is distinct from the memories 211 and 221.
  • the memory 211, 221 includes an operating system and at least one program code (for example, a code installed in the electronic device 110, a browser running on the electronic device 110, or an application installed in the electronic device 110 to provide a specific service). Can be stored.
  • These software components may be loaded from a computer readable recording medium separate from the memories 211 and 221.
  • Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card, and the like.
  • software components may be loaded into the memory 211, 221 through a communication module 213, 223 that is not a computer readable recording medium.
  • At least one program is a computer program that is installed by files provided by a file distribution system (for example, the server 160 described above) through the network 170 to distribute installation files of developers or applications. It may be loaded into the memories 211 and 221 based on (for example, the above-described application).
  • a file distribution system for example, the server 160 described above
  • the network 170 to distribute installation files of developers or applications. It may be loaded into the memories 211 and 221 based on (for example, the above-described application).
  • Processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processors 212, 222 by the memory 211, 221 or the communication modules 213, 223. For example, the processors 212 and 222 may be configured to execute a command received according to a program code stored in a recording device such as the memory 211 and 221.
  • the communication modules 213 and 223 may provide a function for the electronic device 110 and the server 150 to communicate with each other through the network 170, and the electronic device 110 and / or the server 150 may be different from each other. It may provide a function for communicating with an electronic device (for example, the electronic device 120) or another server (for example, the server 160). For example, a request generated by the processor 212 of the electronic device 110 according to a program code stored in a recording device such as the memory 211 may be controlled by the server 150 through the network 170 under the control of the communication module 213. Can be delivered.
  • control signals, commands, contents, files, and the like provided according to the control of the processor 222 of the server 150 pass through the communication module 223 and the network 170 to the communication module 213 of the electronic device 110. ) May be received by the electronic device 110.
  • the control signal, command, content, file, etc. of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the content, file, etc. may be transferred to the electronic device ( 110 may be stored as a storage medium (permanent storage device described above) that may further include.
  • the input / output interface 214 may be a means for interfacing with the input / output device 215.
  • an input device may include a device such as a keyboard, a mouse, a microphone, a camera, and the like
  • an output device may include a device such as a display, a speaker, a haptic feedback device, and the like.
  • the input / output interface 214 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen.
  • the input / output device 215 may be configured as the electronic device 110 and one device.
  • the input / output interface 224 of the server 150 may be a means for interfacing with an apparatus (not shown) for input or output that may be connected to or included in the server 150.
  • a service configured using data provided by the server 150 or the electronic device 120 when the processor 212 of the electronic device 110 processes a command of a computer program loaded in the memory 211.
  • the screen or content may be displayed on the display through the input / output interface 214.
  • the electronic device 110 and the server 150 may include more components than those of FIG. 2. However, it is not necessary to clearly show most of the prior art components.
  • the electronic device 110 may be implemented to include at least some of the above-described input and output devices 215 or other components such as a transceiver, a Global Positioning System (GPS) module, a camera, various sensors, a database, and the like. It may also include more.
  • GPS Global Positioning System
  • the electronic device 110 is a smartphone, an acceleration sensor or a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, a vibrator for vibration generally included in the smartphone Various components such as may be implemented to be further included in the electronic device 110.
  • FIG. 3 is a block diagram illustrating an example of a component that a processor of a server according to an embodiment of the present invention may include, and FIG. 4 is an example of a method that the server may perform according to an embodiment of the present invention. It is a flowchart showing.
  • the server 150 serves as a platform for providing an information providing service to a plurality of electronic devices 110, 120, 130, and 140 that are clients.
  • the server 150 may provide an information providing service in association with an application installed on the electronic devices 110, 120, 130, and 140.
  • the processor 222 of the server 150 is a component as shown in FIG. 3.
  • the components of the processor 222 may be optionally included in or excluded from the processor 222.
  • the components of the processor 222 may be separated or merged to represent the functions of the processor 222.
  • the processor 222 and the components of the processor 222 may control the server 150 to perform steps S410 to S450 included in the information providing method of FIG. 4.
  • the processor 222 and the components of the processor 222 may be implemented to execute instructions according to code of an operating system included in the memory 221 and code of at least one program.
  • the components of the processor 222 may be representations of different functions of the processor 222 performed by the processor 222 according to an instruction provided by the program code stored in the server 150.
  • the user DB configuration unit 310 may be used as a functional representation of the processor 222 controlling the server 150 in accordance with the above-described command such that the server 150 configures the user DB.
  • the processor 222 may read a command required from the memory 221 loaded with a command related to the control of the server 150.
  • the read command may include a command for controlling the processor 222 to execute steps S420 to S450 to be described later.
  • the user DB configuration unit 310 matches user face information for each user with information for identifying the corresponding user (for example, device ID, etc .; hereinafter referred to as 'user information') and stores it in the user DB.
  • User DB can be configured by doing so.
  • the user DB configuration unit 310 may construct face information of each user as a DB for information recognition services for an information providing service based on face recognition.
  • the user DB configuration unit 310 may match the communication information associated with the user for each user with the user information may be stored in the user DB.
  • the communication information may be information registered directly by the user for communication, or may include information specified by the user by another person.
  • the communication information may be information that the user intends to deliver for sharing with others or information that others want to convey to the user.
  • the communication information for the user A may mean a message input by the user A to deliver to another person or a message input by another person to the user A through various communication channels such as a messenger chat room.
  • the communication information about the user A may mean a comment input by the user A with respect to the corresponding content while consuming content on the Internet such as news or video.
  • the communication information may include location information (for example, location information such as location coordinate values of user A or another person) and time information at the time when the information is input or registered, and further, the comment of a specific content
  • the content may further include content reproduction information (for example, scene information when the comment is input) when the corresponding comment is input.
  • the camera image receiver 320 may receive a camera image from the electronic device 110 as a request for providing information.
  • the camera image may include location information
  • the location information of the camera image may include location information of a camera of the electronic device 110 or location information of objects (persons or objects) included in the camera image.
  • the position information of the camera image may include a position coordinate value of the camera and a direction value of the image with respect to the camera (for example, an orientation in which the lens of the camera faces to capture the image by the camera).
  • the position information of the camera image may include position coordinate values of an object included in the image captured by the camera closest to the camera or an object closest to the center of the captured image, and a direction value of the image with respect to the camera (eg,
  • the camera may include an orientation in which the lens of the camera faces to capture an image.
  • the location information may be provided by the electronic device 110, or may be captured by the camera, the position value of the camera acquired from the electronic device 110, the focal length at the time of capturing the corresponding image, the image resolution, and the camera. It is also possible to calculate directly based on the collected images.
  • the user information acquisition unit 330 compares the location information of the camera image received in operation S430 with the location information of the communication information stored in the user DB, based on the location information of the camera image.
  • User information with communication information whose information is within a certain range can be obtained.
  • Some ranges include the position coordinate values of the camera image (e.g., the position coordinate values of the camera or the position coordinates of the object captured by the camera) and the orientation value of the camera image relative to the camera (e.g., the camera captures the image. In order to prevent the lens of the camera from facing each other).
  • the user information acquisition unit 330 may identify which users exist at a location within a predetermined range from the camera or an object in the image in the direction of the image with respect to the camera through the user DB. In other words, the user information acquirer 330 may obtain a user list that transmits or receives communication information around a location corresponding to the camera image of the electronic device 110.
  • step S450 the information retrieval unit 340 retrieves the user's face information and communication information matched with the user information obtained in step S440 in the user DB, and then receives the retrieved face information in step S430. If the at least one of the detected face information is recognized in the camera image, it is requested to provide information about the detected face information along with the position of the face in the camera image.
  • the electronic device 110 may transmit the electronic device 110.
  • the information retrieval unit 340 does not retrieve face information of all users on the user DB, but the position information is fixed from the camera image in consideration of the position coordinates of the camera image captured by the camera and the direction value of the camera image with respect to the camera.
  • the face information may be searched only for users having communication information determined to be included in the range.
  • the information retrieval unit 340 also searches for communication information in which the time information of the communication information is included within a predetermined time from the time when the camera image is received, in consideration of the time information when the camera image is received, and has such communication information.
  • the face information may be searched only for users.
  • the information retrieval unit 340 may provide a face position of a user existing at a location indicated by the camera image and communication information related to the user in response to the request for providing the information from the electronic device 110.
  • the information retrieval unit 340 may search for face information and communication information matched with the user information acquired in operation S440 in the user DB and transmit the same to the electronic device 110, and the electronic device 110. Allow the face information to be recognized in the camera image.
  • the information retrieval unit 340 When two or more pieces of communication information are retrieved with respect to face information of a user associated with a camera image, the information retrieval unit 340 is located at a location indicated by the camera image in response to a request for providing information from the electronic device 110. All or part of the two or more communication information may be provided along with the face location of the. For example, when two or more pieces of communication information are retrieved with respect to face information of a user associated with a camera image, the information retrieval unit 340 is the most recently input of the user's communication information or a doctor who has the best response on SNS. The delivery information may be selected and provided as communication information related to the user.
  • the server 150 may provide a service for identifying which user has what communication information at a location captured by the camera of the electronic device 110 based on face recognition.
  • FIG. 5 is a block diagram illustrating an example of a component that may be included in a processor of an electronic device according to an embodiment of the present invention
  • FIG. 6 is a method that may be performed by an electronic device according to an embodiment of the present invention.
  • the electronic device 110 may be configured with a computer-based information providing system.
  • the information providing system may be implemented in the form of a program that operates independently, or may be implemented in an in-app form of a specific application so as to be able to operate on the specific application.
  • Information service can be provided through interworking with 150).
  • An information providing system implemented in the electronic device 110 may perform an information providing method based on a command provided by an application installed in the electronic device 110.
  • the processor 212 of the electronic device 110 is a component, as shown in FIG. 5, a camera image transmitter 510, an information receiver 520, and information.
  • the display unit 530 may be included.
  • components of the processor 212 may optionally be included or excluded in the processor 212.
  • the components of the processor 212 may be separated or merged for representation of the functions of the processor 212.
  • the processor 212 and the components of the processor 212 may control the electronic device 110 to perform steps S610 to S640 included in the information providing method of FIG. 6.
  • the processor 212 and the components of the processor 212 may be implemented to execute instructions according to code of an operating system included in the memory 211 and code of at least one program.
  • the components of the processor 212 are performed by the processor 212 according to a command provided by a program code stored in the electronic device 110 (for example, a command provided by an application driven in the electronic device 110).
  • a command provided by a program code stored in the electronic device 110 for example, a command provided by an application driven in the electronic device 110.
  • the camera image transfer unit 510 is used as a functional representation of the processor 212 controlling the electronic device 110 according to the above-described command so that the electronic device 110 transmits the camera image to the server 150. Can be.
  • the processor 212 may read a necessary command from the memory 211 loaded with a command related to the control of the electronic device 110.
  • the read command may include a command for controlling the processor 212 to execute steps S620 to S640 to be described later.
  • the camera image transmitter 510 may transmit an image, that is, a camera image, input through the camera of the electronic device 110 to the server 150 as a request for providing information.
  • the camera image may include an image obtained by extracting only a predetermined object such as a face of a person from an image captured by the camera or an image captured by the camera. That is, the camera image transmitting unit 510 may transmit the image data captured by the camera of the electronic device 110 to the server 150 as a request for providing the information, and the predetermined image such as the face of a person is determined in the image captured by the camera. Data extracted only from the object may be transmitted to the server 150.
  • the camera image may include location information
  • the location information of the camera image may include location information of the camera of the electronic device 110 or location information of objects determined to be included in the camera image.
  • the position information of the camera image may be a position coordinate value of the electronic device 110 or a position coordinate value of the camera acquired by the electronic device 110 from GPS information, base station information, or the like.
  • the lens may include opposing orientations.
  • the position information of the camera image is the position coordinate value of the object closest to the camera among the objects included in the image captured by the camera or the object closest to the center of the captured image, and the lens of the camera to capture the image. May include opposite bearings.
  • the information receiver 520 may receive a face position and communication information of the user recognized in the camera image from the server 150.
  • the information receiver 520 may receive face information of a user existing at a location indicated by the camera image and communication information related to the user in response to the request for providing the information from the server 150.
  • the information display unit 530 provides an AR (Augmented Reality) function that allows information to be checked through the camera.
  • the information display unit 530 reflects the information received from the server 150 to the camera image and outputs the received image. Track the face position and display communication information of the corresponding user in response to the face position.
  • the information display unit 530 may detect and display the communication information transmitted from the camera image captured by the camera or the received user through face recognition.
  • the electronic device 110 may show, as AR content on the camera image, which user has what communication information in the image captured by the camera based on face recognition.
  • FIG. 7 shows an example of a user DB 700 in one embodiment of the present invention.
  • face information 720 and communication information 730 of each user may be matched with the corresponding user information 710 and stored.
  • the server 150 may receive the face image of the user for each user and store the face information 720 including the feature data extracted from the input face image in correspondence with the user information 710.
  • the server 150 receives a face image of a user for each user, constructs a face DB in which feature data extracted from the input face image is stored together with each face ID, and creates a new pseudo for any user.
  • the face ID of the user may be stored as the face information 720 together with the new communication information 730.
  • the server 150 communicates information related to the user through an interface screen interoperable with the server 150, for example, a chat room of an official account such as a fan meeting or a live broadcast, a comment registration screen such as SNS or news, etc. 730, that is, information that the user wants to share for sharing with others or information that another person wants to deliver to the user may be obtained and matched with the user information 710 and stored.
  • the communication information 730 is the object 731 for communication, content 732 input for communication, location information 733 at the time when the information is input, and the information is input or transmitted. Time information 734 may be included. As shown in FIG.
  • the communication target 731 may access a content ID for identifying a live broadcast, SNS, news, and the like, a chat room ID in a messenger, an SNS, an online community, or the like.
  • the location information 733 may include a URL that can be inputted, and the location information 733 includes a time point at which content input starts, a time point at which content input ends, a time point between start and end of content input, and contents input by a user operation. It may include location information of a predetermined specific time point, such as a transmission time point.
  • the positional information 733 is a positional information array ( ⁇ (positional coordinate value at the start of inputting content), (position of contents) representing the positional information from the time point at which content input starts to the end point as an array. A position coordinate value at the time point between the start and the end of the input), and (a position coordinate value at the time when the content input by the user's operation is transmitted) ⁇ .
  • the communication information 730 for the specific content may further include content reproduction information at the time when the corresponding information is input.
  • the video may further include and store scene information (for example, a scene corresponding to 50 seconds after playback) when the communication information 730 is input.
  • FIG. 8 to 10 illustrate an example of an information providing process based on face recognition according to an embodiment of the present invention.
  • FIG. 8 shows a chat room 800 of an official account for receiving questions from participants in a fan meeting of a celebrity group.
  • User_1 801 one of the fan meeting participants, inputs a question 803 by specifying User_3 802 belonging to a celebrity group through the chat room 800.
  • the electronic device 120 of the User_1 801 may transmit the communication information including the target to which the question 803 is input, the content input to the question 803, the current location, and the time.
  • the server 150 may match the communication information received from the electronic device 120 with the user information User_3 specified in the question 803 and store it in the user DB 700. have.
  • FIG. 9 illustrates a camera image 900 in which another user or a relay electronic device 110 at a fan meeting site captures a celebrity group.
  • the electronic device 110 transmits the camera image 900 to the server 150.
  • the server 150 checks the user_3 received a question at the fan meeting site based on the information on the user DB 700, and then the camera image (
  • the facial location 910 of User_3 may be recognized, and the recognized facial location 910 and the corresponding question content may be transmitted to the electronic device 110.
  • the electronic device 110 may track the face location 910 of User_3 in the camera image 900 and display the question content 920 received by User_3 in an area corresponding to the face location 910.
  • the electronic device 110 may recognize the face of the user who received the question in the camera image of the fan meeting site through the interworking with the server 150 and display the corresponding question content.
  • server 150 queries communication information received from electronic device 120. It may be stored in the user DB 700 by matching the user information User_1 inputted with the question 803 instead of the user information User_3 that has received 803.
  • FIG. 10 illustrates a camera image 1000 in which another user or the relay electronic device 110 at the fan meeting site captures the fan meeting participants. Similarly, the electronic device 110 transmits the camera image 1000 to the server 150, and after this, the server 150 checks User_1 who inputs a question at the fan meeting site based on the information on the user DB 700.
  • the face location 1010 of User_1 may be recognized in the camera image 1000, and the recognized face location 1010 and the corresponding question content may be transmitted to the electronic device 110.
  • the electronic device 110 may track the face location 1010 of User_1 in the camera image 1000 and display the question content 1020 input by User_1 in an area corresponding to the face location 1010.
  • the electronic device 110 may recognize the face of the user who inputs the question in the camera image of the fan meeting site through the interworking with the server 150 and display the corresponding question content.
  • 11 to 12 illustrate another example of an information providing process based on face recognition according to an embodiment of the present invention.
  • chat room 1100 of an official account for a direct transaction.
  • One user User_1 1101 moves to an appointment place for a transaction and executes location transmission 1102 through chat room 1100.
  • the electronic device 120 of the User_1 1101 may transmit communication information including the location transmission contents, the current location, and the time to the server 150, and at this time, the server 150 The communication information transmitted from the electronic device 120 may be matched with the user information User_1 transmitting the location and stored in the user DB 700.
  • FIG. 12 illustrates a camera image 1200 captured by another user who has promised a direct transaction around an appointment place.
  • the electronic device 110 transmits the camera image 1200 to the server 150.
  • the server 150 checks User_1 transmitting the location around the place based on the information on the user DB 700, and then displays the camera image.
  • the face location 1210 of the user_1 is recognized, and the recognized face location 1210 is transmitted to the electronic device 110.
  • the electronic device 110 may track the face location 1210 of User_1 in the camera image 1200 and display an indicator 1220 indicating User_1 in an area corresponding to the face location 1210.
  • the electronic device 110 may recognize the face of the user who promised the transaction in the camera image of the direct transaction appointment place through the linkage with the server 150 and show the corresponding location.
  • 13 to 15 illustrate another example of an information providing process based on face recognition according to an embodiment of the present invention.
  • the user User_2 1301 views a video from the street and inputs a comment 1302 for the video.
  • the electronic device 120 of the User_2 1301 may display the target, the comment content, and the location, time, and scene information at the time when the comment 1302 is input.
  • the communication information may be transmitted to the server 150.
  • the server 150 matches the communication information received from the electronic device 120 with the user information (User_2) inputted in the comment 1302, and the user DB. Can be stored at 700.
  • FIG. 14 illustrates a camera image 1400 in which another user captures a distance in the vicinity of where the user User_2 is located.
  • the electronic device 110 of another user transmits the camera image 1400 to the server 150 together with the information about the communication target, and the server 150 at the corresponding distance based on the information on the user DB 700.
  • the camera image 1400 recognizes the user's face position 1410 and transmits the recognized face position 1410 to the electronic device 110.
  • the electronic device 110 may track the face location 1410 of User_2 in the camera image 1400 and display the comment content 1420 input by User_2. Then, the electronic device 110 will show the scene information 1430 at the time of inputting the comment along with the comment content 1420 input by User_2 in the camera image 1400 according to the information provided by the server 150. Can be.
  • the electronic device 110 may separately include a user comment 1520 that does not recognize a user's face or does not have face information pre-registered among comments registered at a distance corresponding to the camera image 1400. It is also possible to show through region 1500 of.
  • the electronic device 110 may recognize the face of the user who inputs the comment at a peripheral distance from the camera image and display the corresponding comment content through the linkage with the server 150.
  • the camera may be driven to check other users' reactions to the content through the camera image.
  • Content owners can check the user's reaction by grouping the contents or grouping all the contents owned by them.
  • the response of another user located near the point the camera is capturing is displayed as the AR content of the camera image. New comments can be displayed and real time to see what reactions have been made in which scenes.
  • the communication information related to the user is searched through a DB stored with the location information to search for a user existing at a position corresponding to the camera image, and recognized by recognizing a face of the user found in the camera image.
  • the location of the user's face and information delivery information related to the user may be shown through a camera image.
  • the apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components.
  • the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable PLU (programmable). It can be implemented using one or more general purpose or special purpose computers, such as logic units, microprocessors, or any other device capable of executing and responding to instructions.
  • the processing device may execute an operating system (OS) and one or more software applications running on the operating system.
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • OS operating system
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include.
  • the processing device may include a plurality of processors or one processor and one controller.
  • other processing configurations are possible, such as parallel processors.
  • the software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device.
  • the software and / or data may be embodied in any type of machine, component, physical device, computer storage medium or device in order to be interpreted by or provided to the processing device or to provide instructions or data. have.
  • the software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.
  • the method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium.
  • the medium may be to continuously store a program executable by the computer, or to temporarily store for execution or download.
  • the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions.
  • examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application, a site that supplies or distributes various software, a server, or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Selon l'invention, un procédé de fourniture d'informations peut comprendre les étapes suivantes : configurer une BD d'utilisateurs en faisant correspondre, pour chaque utilisateur, des informations de visage de l'utilisateur et des informations de communication associées à l'utilisateur avec des informations d'utilisateur pour l'identification de l'utilisateur et stocker les informations mises en correspondance dans la BD d'utilisateurs ; recevoir une image de caméra provenant d'un dispositif électronique en réponse à une demande de fourniture d'informations ; et récupérer des informations de communication et des informations de visage de l'utilisateur, qui sont associées à l'image de caméra, dans la BD d'utilisateurs en utilisant l'image de caméra, et lorsque des informations de visage d'un utilisateur spécifique dans les informations de visage récupérées sont reconnues à partir de l'image de caméra, transférer, au dispositif électronique, une position de visage des informations de visage reconnues dans l'image de caméra et des informations de communication de l'utilisateur spécifique dans les informations de communication récupérées.
PCT/KR2018/001365 2018-02-01 2018-02-01 Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale Ceased WO2019151548A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/KR2018/001365 WO2019151548A1 (fr) 2018-02-01 2018-02-01 Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale
KR1020207013958A KR102500624B1 (ko) 2018-02-01 2018-02-01 얼굴 인식에 기초하여 정보를 제공하는 방법과 시스템 및 비-일시적인 컴퓨터 판독 가능한 기록 매체

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2018/001365 WO2019151548A1 (fr) 2018-02-01 2018-02-01 Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale

Publications (1)

Publication Number Publication Date
WO2019151548A1 true WO2019151548A1 (fr) 2019-08-08

Family

ID=67478326

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2018/001365 Ceased WO2019151548A1 (fr) 2018-02-01 2018-02-01 Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale

Country Status (2)

Country Link
KR (1) KR102500624B1 (fr)
WO (1) WO2019151548A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115580754A (zh) * 2022-12-06 2023-01-06 成都智元汇信息技术股份有限公司 一种地铁公共空间内容点播方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101403159B1 (ko) * 2013-02-15 2014-06-03 주식회사 엘지유플러스 객체에 관한 부가정보 제공 장치 및 방법
US20150172376A1 (en) * 2013-12-17 2015-06-18 Samsung Electronics Co., Ltd. Method for providing social network service and electronic device implementing the same
KR101534808B1 (ko) * 2013-12-30 2015-07-08 주식회사 시어스랩 얼굴 인식 기술을 이용한 전자 앨범 관리 방법 및 시스템
KR20150079804A (ko) * 2013-08-15 2015-07-08 시아오미 아이엔씨. 이미지 처리 방법, 장치, 단말기기, 프로그램 및 기록매체
KR20170068305A (ko) * 2015-12-09 2017-06-19 삼성전자주식회사 사용자 정보를 제공하는 전자 장치 및 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101403159B1 (ko) * 2013-02-15 2014-06-03 주식회사 엘지유플러스 객체에 관한 부가정보 제공 장치 및 방법
KR20150079804A (ko) * 2013-08-15 2015-07-08 시아오미 아이엔씨. 이미지 처리 방법, 장치, 단말기기, 프로그램 및 기록매체
US20150172376A1 (en) * 2013-12-17 2015-06-18 Samsung Electronics Co., Ltd. Method for providing social network service and electronic device implementing the same
KR101534808B1 (ko) * 2013-12-30 2015-07-08 주식회사 시어스랩 얼굴 인식 기술을 이용한 전자 앨범 관리 방법 및 시스템
KR20170068305A (ko) * 2015-12-09 2017-06-19 삼성전자주식회사 사용자 정보를 제공하는 전자 장치 및 방법

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115580754A (zh) * 2022-12-06 2023-01-06 成都智元汇信息技术股份有限公司 一种地铁公共空间内容点播方法及系统

Also Published As

Publication number Publication date
KR20200106026A (ko) 2020-09-10
KR102500624B1 (ko) 2023-02-16

Similar Documents

Publication Publication Date Title
CN109387217B (zh) 导航方法、计算机可读存储介质及导航服务器
WO2020096087A1 (fr) Procédé, système, et support d'enregistrement lisible par ordinateur non transitoire pour gérer un groupe de messages
CN107370657B (zh) 设备间应用程序联动方法及系统
WO2018074716A1 (fr) Procédé et système pour recommander une interrogation à l'aide d'un contexte de recherche
WO2018212470A1 (fr) Sélection de support pour fournir des informations correspondant à une demande vocale
WO2011139115A2 (fr) Procédé pour accéder à des informations sur des personnages à l'aide d'une réalité augmentée, serveur et support d'enregistrement lisible par ordinateur
WO2018030554A1 (fr) Procédé et système de fourniture de notification basée sur un message
WO2019059435A1 (fr) Procédé et système de recommandation utilisant un contenu comprenant des données de localisation
WO2018174314A1 (fr) Procédé et système de production d'une séquence vidéo d'histoire
WO2019235653A1 (fr) Procédé et système de reconnaissance de connaissance proche sur la base d'une communication sans fil à courte portée et support d'enregistrement non transitoire lisible par ordinateur
WO2015133880A1 (fr) Procédé et appareil de mise à jour d'informations publicitaires
KR102761717B1 (ko) 그룹 앨범을 이용하는 대화방에서 프로필 사진 기반 사용자 얼굴 인식과 활용 방법
WO2018128261A1 (fr) Procédé et système de partage d'informations d'activité d'utilisateur
KR20200044167A (ko) 메신저 봇을 이용하여 ＩｏＴ 기기를 제어하기 위한 방법, 시스템, 및 비-일시적인 컴퓨터 판독가능한 기록 매체
WO2019156263A1 (fr) Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de pourvoir à un salon de discussion sous une forme tridimensionnelle
JP7382733B2 (ja) イメージを位置データに変換して提供する方法とシステム、および非一時的なコンピュータ読み取り可能な記録媒体
WO2019172469A1 (fr) Procédé et système pour partager un contenu de diffusion en continu en temps réel
KR102225175B1 (ko) 영상 통화를 이용한 길안내 방법 및 시스템
WO2021125578A1 (fr) Procédé et système de reconnaissance de position reposant sur un traitement d'informations visuelles
WO2019151548A1 (fr) Procédé, système et support d'enregistrement non transitoire lisible par ordinateur permettant de fournir des informations en fonction de la reconnaissance faciale
WO2019245062A1 (fr) Procédé et système de fourniture de contenus sur la base de réactions d'utilisateur, support d'enregistrement lisible par ordinateur non transitoire
WO2019172463A1 (fr) Procédé, système et support d'enregistrement non transitoire lisible par ordinateur pour recommander une photo d'un profil
WO2019240305A1 (fr) Procédé, système et support d'enregistrement lisible par ordinateur non transitoire permettant le traitement d'un salon de discussion en fonction d'un degré d'utilisation
WO2018030553A1 (fr) Procédé et système permettant de fournir une fonction de réponse automatique sur un logiciel de messagerie
WO2018062591A1 (fr) Procédé et système de fourniture de contenu

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18903476

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18903476

Country of ref document: EP

Kind code of ref document: A1