CN101169897A - Multimedia system, face detection remote control system and method thereof - Google Patents
Multimedia system, face detection remote control system and method thereof Download PDFInfo
- Publication number
- CN101169897A CN101169897A CNA2006101365569A CN200610136556A CN101169897A CN 101169897 A CN101169897 A CN 101169897A CN A2006101365569 A CNA2006101365569 A CN A2006101365569A CN 200610136556 A CN200610136556 A CN 200610136556A CN 101169897 A CN101169897 A CN 101169897A
- Authority
- CN
- China
- Prior art keywords
- face
- remote control
- eyes
- people
- multimedia system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title abstract description 36
- 210000001508 eye Anatomy 0.000 claims description 57
- 238000004458 analytical method Methods 0.000 claims description 10
- 230000000052 comparative effect Effects 0.000 claims 11
- 230000001815 facial effect Effects 0.000 claims 11
- 230000006870 function Effects 0.000 description 21
- 230000006399 behavior Effects 0.000 description 13
- 238000010191 image analysis Methods 0.000 description 13
- 210000005252 bulbus oculi Anatomy 0.000 description 12
- 210000000887 face Anatomy 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
技术领域 technical field
本发明有关于一种使用于多媒体系统的遥控系统及其方法以及具有遥控功能的多媒体系统,且特别是有关于一种使用于多媒体系统的基于脸部检测的遥控系统及其方法以及具有基于脸部检测的遥控功能的多媒体系统。The present invention relates to a remote control system and its method used in a multimedia system and a multimedia system with a remote control function, and in particular to a face detection-based remote control system and its method used in a multimedia system and a face-based remote control system. The remote control function of the Ministry of detection is multimedia system.
背景技术 Background technique
公知的多媒体系统,例如是电视机、DVD音像播放机,大都具备遥控功能,让使用者在一定距离内,通过一台搭配多媒体系统的遥控器,以有线或是无线的方式,控制多媒体系统的运作,例如是设定多媒体系统,或是控制其播放多媒体信息,如电视广播节目、DVD影片。Known multimedia systems, such as televisions and DVD players, mostly have a remote control function, allowing users to control the multimedia system within a certain distance through a remote control that matches the multimedia system in a wired or wireless manner. Operation, such as setting the multimedia system, or controlling it to play multimedia information, such as TV broadcast programs and DVD movies.
但是,站在使用者的角度,上述公知的遥控多媒体系统的方式却因为需要一台遥控器而造成其他使用上不方便的问题。首先,因为遥控器的体积少,若要能让使用者正确的控制多媒体系统多样化的功能,必需具有许多的按键。而且,每台遥控器都具有某些不同的操作方式,加上许多的按键,操作方式往往变得复杂,令不熟悉该多媒体系统的使用者更不容易掌握,并需要花费时间学习及习惯。此外,若不小心把遥控器放在不显眼的地方,使用者需要花费时间寻找。再者,遥控器需要安装电池,需要消耗能源。若电池电力不足或电池品质不佳时,往往会造成不可预期的问题,例如是操作上的失败,更甚者,遥控器因电池内容物的漏出而毁坏掉。However, from the user's point of view, the above-mentioned known remote control method for the multimedia system causes other problems of inconvenient use because a remote control is required. First of all, because the remote controller has a small volume, it must have a lot of keys in order to allow the user to correctly control the various functions of the multimedia system. Moreover, each remote controller has some different operation methods, and with many buttons, the operation methods often become complicated, making it difficult for users who are not familiar with the multimedia system to master, and it takes time to learn and get used to. In addition, if the remote control is accidentally placed in an inconspicuous place, the user needs to spend time looking for it. Furthermore, the remote controller needs to install a battery, which consumes energy. If the power of the battery is insufficient or the quality of the battery is not good, it will often cause unexpected problems, such as failure in operation, and what's more, the remote controller will be destroyed due to the leakage of the battery content.
故此,为了使用者轻易的遥控多媒体系统或是解决上述问题,多媒体系统需要有一种有别于公知利用遥控器的遥控方式以满足使用者对方便性的需求。Therefore, in order for the user to easily control the multimedia system or solve the above problems, the multimedia system needs a remote control method different from the conventional remote control to meet the user's demand for convenience.
发明内容 Contents of the invention
有鉴于此,本发明的目的就是在提供一种使用于多媒体系统的基于脸部检测的遥控系统及其方法以及具有基于脸部检测的遥控功能的多媒体系统。多媒体系统的使用者,身体某部分的自然行为,例如脸部及眼睛,会被检测并分析并据以产生特性参数,特性参数与条件参数进行比较,比较的结果会用以决定如何遥控多媒体系统的运作。In view of this, the object of the present invention is to provide a remote control system and method based on face detection used in a multimedia system and a multimedia system with a remote control function based on face detection. For the user of the multimedia system, the natural behavior of certain parts of the body, such as the face and eyes, will be detected and analyzed to generate characteristic parameters. The characteristic parameters are compared with the condition parameters, and the comparison results are used to determine how to remotely control the multimedia system operation.
根据本发明的目的,提出一种使用于多媒体系统的基于脸部检测的遥控方法的一实施例,使用于一多媒体系统,该遥控方法包括:(a)依据该多媒体系统的一操作模式,通过一检测装置检测存在的至少人脸的一部分,并据以输出一人脸图像信号;(b)依照该人脸图像信号,分析所检测到的至少人脸的一部分,以产生相对应的至少一特性参数;(c)比较至少该特性参数与至少一条件参数,以产生一比较结果,其中该条件参数与该操作模式的至少一控制操作相关联;以及(d)依据该比较结果,产生至少一指示信号给该多媒体系统。According to the purpose of the present invention, an embodiment of a remote control method based on face detection used in a multimedia system is proposed, used in a multimedia system, the remote control method includes: (a) according to an operation mode of the multimedia system, through A detection device detects at least a part of the human face, and outputs a human face image signal accordingly; (b) according to the human face image signal, analyzes the detected at least a part of the human face to generate corresponding at least one characteristic parameters; (c) comparing at least the characteristic parameter with at least one condition parameter to generate a comparison result, wherein the condition parameter is associated with at least one control operation of the operation mode; and (d) generating at least one according to the comparison result An indication signal is sent to the multimedia system.
根据本发明的目的,提出一种使用于多媒体系统的基于脸部检测的遥控系统的一实施例,使用于一多媒体系统中,其中该多媒体系统具有一操作模式,该操作模式具有至少一控制操作。该遥控系统包括:一检测装置及一图像分析与指示产生单元。检测装置依据该操作模式,用以检测存在的至少人脸的一部分,并据以输出一人脸图像信号。图像分析与指示产生单元,与该检测装置耦接,用以依照该人脸图像信号,分析所检测到的至少人脸的一部分,以产生相对应的至少一特性参数,并比较至少该特性参数与至少一条件参数,并据以产生一比较结果,并依据该比较结果,产生至少一指示信号给该多媒体系统,其中该条件参数与该操作模式的至少该控制操作相关联。According to the object of the present invention, an embodiment of a remote control system based on face detection for use in a multimedia system is proposed, wherein the multimedia system has an operation mode with at least one control operation . The remote control system includes: a detection device and an image analysis and instruction generation unit. According to the operation mode, the detection device is used to detect at least a part of human face, and output a human face image signal accordingly. An image analysis and instruction generating unit, coupled with the detection device, for analyzing at least a part of the detected face according to the face image signal to generate at least one corresponding characteristic parameter, and comparing at least the characteristic parameter and at least one condition parameter, and generate a comparison result accordingly, and generate at least one indication signal to the multimedia system according to the comparison result, wherein the condition parameter is associated with at least the control operation of the operation mode.
根据本发明的目的,提出一具有基于脸部检测的遥控功能的多媒体系统的一实施例,其中该多媒体系统具有一操作模式,该操作模式具有至少一控制操作。该多媒体系统包括一检测装置、一图像分析与指示产生单元及一播放装置。检测装置,用以依据该操作模式,检测存在的至少人脸的一部分,并据以输出一人脸图像信号。图像分析与指示产生单元,与该检测装置耦接,用以依照该人脸图像信号,分析所检测到的至少人脸的一部分,以产生相对应的至少一特性参数,并比较至少该特性参数与至少一条件参数,以据以产生一比较结果,并依据该比较结果,产生至少一指示信号,其中该条件参数与该操作模式的至少该控制操作相关联。播放装置,用以依据接收至少该指示信号,播放多媒体信息。According to the object of the present invention, an embodiment of a multimedia system with remote control function based on face detection is proposed, wherein the multimedia system has an operation mode with at least one control operation. The multimedia system includes a detection device, an image analysis and instruction generation unit and a playback device. The detecting device is used for detecting at least a part of a human face according to the operation mode, and outputting a human face image signal accordingly. An image analysis and instruction generating unit, coupled with the detection device, for analyzing at least a part of the detected face according to the face image signal to generate at least one corresponding characteristic parameter, and comparing at least the characteristic parameter and at least one condition parameter to generate a comparison result, and generate at least one indication signal according to the comparison result, wherein the condition parameter is associated with at least the control operation of the operation mode. The playing device is used for playing multimedia information according to receiving at least the indication signal.
为让本发明的上述目的、特征、和优点能更明显易懂,下文特举较佳实施例,并配合附图,详细说明如下:In order to make the above-mentioned purposes, features, and advantages of the present invention more comprehensible, preferred embodiments are specifically cited below, together with the accompanying drawings, described in detail as follows:
附图说明 Description of drawings
图1为依据本发明的多媒体系统的一实施例的方块图,其中该媒体系统包括一基于脸部检测的遥控系统的的一实施例。FIG. 1 is a block diagram of an embodiment of a multimedia system including an embodiment of a face detection based remote control system according to the present invention.
图2表示依据本发明的具有基于脸部检测的遥控功能的多媒体系统的另一实施例。FIG. 2 shows another embodiment of a multimedia system with remote control function based on face detection according to the present invention.
图3为一基于脸部检测的遥控方法的一较佳实施例的流程图。FIG. 3 is a flowchart of a preferred embodiment of a remote control method based on face detection.
图4A为一基于脸部检测的遥控方法应用于多媒体系统的播放模式时的一实施例的流程图。FIG. 4A is a flowchart of an embodiment when a face detection-based remote control method is applied to a playback mode of a multimedia system.
图4B为图4A中,用以产生指示信号给多媒体系统的步骤440的一实施例的流程图。FIG. 4B is a flow chart of an embodiment of
图5为一示意图,表示将依据本发明的一基于脸部检测的遥控系统应用于多媒体系统的命令选项模式时的一实施例。FIG. 5 is a schematic diagram showing an embodiment of applying a remote control system based on face detection according to the present invention to a command option mode of a multimedia system.
主要元件符号说明Description of main component symbols
100、200:多媒体系统100, 200: multimedia system
110:播放装置110: Playback device
120、520:显示装置120, 520: display device
130:遥控系统130: Remote control system
140、540:检测装置140, 540: detection device
150、550:图像分析与指示产生单元150, 550: image analysis and instruction generation unit
210:系统主机210: system host
500:眼睛500: eyes
521、522:遥控选项521, 522: remote control options
580、590:箭头580, 590: Arrows
具体实施方式 Detailed ways
本发明为依据人类身体某部分的自然行为,如人的头部,包括脸或人脸任何一部分,如人脸、眼睛、下巴、嘴巴、嘴唇、鼻子、耳朵、脸颊、头发、胡须,或上述的各种组合,将的与遥控多媒体系统的操作(如指示播放或改变设定)相关联,以作为产生指示信号给多媒体系统的基础来源,从而改变多媒体系统的操作行为。如此,操作多媒体系统的方式将变得更为容易及直接,并可据该衍生智能型的控制方式,也就是说,遥控系统可依据实际检测的结果,不必使用者亲自给予指示而自动发出指示信号以调整多媒体系统的运作。以下通过不同的实施例说明本发明的各种实施方式,其中我们以人脸及眼睛的行为,将的与遥控多媒体系统的操作(如指示播放或改变设定)相关联为例子以作说明。The present invention is based on the natural behavior of a certain part of the human body, such as the human head, including the face or any part of the human face, such as the human face, eyes, chin, mouth, lips, nose, ears, cheeks, hair, beard, or the above Various combinations of these are associated with the operation of the remote control multimedia system (such as instructing to play or changing settings) as the basic source for generating instruction signals to the multimedia system, thereby changing the operation behavior of the multimedia system. In this way, the method of operating the multimedia system will become easier and more direct, and an intelligent control method can be derived based on this, that is to say, the remote control system can automatically issue instructions based on the actual detection results without the need for the user to give instructions personally signal to adjust the operation of the multimedia system. Various embodiments of the present invention are described below through different embodiments, wherein we take the behavior of human face and eyes associating with the operation of the remote control multimedia system (such as instructing to play or changing settings) as an example for illustration.
请参考图1,其为依据本发明的多媒体系统的一实施例的方块图,其中该多媒体系统包括一基于脸部检测的遥控系统的一实施例。多媒体系统100包括一播放装置110、一显示装置120以及一遥控系统130。Please refer to FIG. 1 , which is a block diagram of an embodiment of a multimedia system according to the present invention, wherein the multimedia system includes an embodiment of a remote control system based on face detection. The
多媒体系统100,具有处理多媒体数据及据以重现多媒体信息功能的系统。多媒体系统100具有至少一操作模式,例如是播放模式,操作模式具有至少一控制操作,例如是播放、暂停或停止操作。多媒体系统100又例如具有命令选项模式,命令选项模式具有至少一命令选取操作,命令选取操作用以设定或控制多媒体系统100。至少有一条件参数与该至少一控制操作相关联。The
遥控系统130用以依据人的自然行为,以脸部或至少其中一部分为基础,产生一指示信号,以改变多媒体系统100的操作行为。遥控系统130包括一检测装置140及一图像分析与指示产生单元150。遥控系统130可利用物件检测(object detection)、目标跟踪(object tracking)或脸部检测(humanface detection)等技术。The
检测装置140,为可检测人体或是提取图像的装置,例如是电荷耦合元件(CCD)、数字摄像机(camera)、网络摄像机(webcam)或数字图像提取器(video capture device),用以依据上述操作模式,通过一检测装置检测存在的人脸或至少人脸的一部分,并据以输出一人脸图像信号。检测装置140例如是具有自动对焦(auto-zooming)的功能,或是人脸特征提取的功能,或眼球跟踪定位的功能(eyeballs tracking and location)。The
图像分析与指示产生单元150,与检测装置140耦接,用以依照人脸图像信号,分析所检测到的至少人脸的一部分,以产生相对应的至少一特性参数,此外图像分析与指示产生单元150更依据该至少一特性参数及至少一条件参数,用以判断是否产生至少一指示信号给多媒体系统100。在实际操作时,图像分析与指示产生单元150比较该至少一特性参数与该操作模式的至少一控制操作相关联的条件参数,以产生一比较结果,并依据该比较结果,产生至少一指示信号。该图像分析与指示产生单元150,例如是以一数字图像处理器(digital image processor),一微处理器(microprocessor)、或单片系统(system on chip)以实现,此外,该单元150亦可包含存储器以存储数据。The image analysis and
播放装置110,例如是一DVD播放机、一数字录放像机或一无线网络多媒体播放机。播放装置110用以依据至少该指示信号,并据以处理多媒体信息,或依据该指示信号设定播放装置110。播放装置110处理及输出多媒体信号。播放装置110是一可以处理多媒体数据的处理装置,例如是一电脑系统或音像播放机或录放像机。播放装置110可以实施以使的能读取存储媒体,如光盘、硬盘或记忆卡,或是以有线或无线方式读取数字图像或声音数据,以作处理及播放。The
显示装置120,例如是平面显示器、液晶显示器、CRT显示器、投影显示器,用以依据播放装置110所输出的多媒体信号以重现图像及声音。The
此外,播放装置110与显示装置120亦可实现为一整合用以处理多媒体数据以作多媒体重现的单元,例如是桌上或笔记型电脑系统、数字电视机、模拟电视机或视频会议系统。In addition, the
请参考图2,其为依据本发明的具有基于脸部检测的遥控功能的多媒体系统的另一实施例。该多媒体系统200与图1的实施例同样具有播放装置110、显示装置120、检测装置140及图像分析与指示产生单元150。Please refer to FIG. 2 , which is another embodiment of the multimedia system with remote control function based on face detection according to the present invention. The
但该两实施例的结构上不同:播放装置110与图像分析与指示产生单元150存在于一系统主机210之中。此外,依据图2,可得多媒体系统的另一实施例,如把图2中的检测装置140改为内置在系统主机210之中。However, the structures of the two embodiments are different: the
图3为一基于脸部检测的遥控方法的一较佳实施例的流程图。遥控方法包括如下步骤:如步骤310所示,首先,设定至少一条件参数。如步骤320所示,依据该多媒体系统的一操作模式,通过一检测装置,检测存在的至少人脸的一部分,并据以输出一人脸图像信号。如步骤330所示,依据人脸图像信号,分析所检测到的至少人脸的一部分,以产生相对应的多个特性参数。如步骤340所示,对这些特性参数与该操作模式的至少一控制操作相关联的一条件参数作比较,以产生一比较结果。接着,如步骤350所示,依据该比较结果,产生至少一指示信号给多媒体系统。FIG. 3 is a flowchart of a preferred embodiment of a remote control method based on face detection. The remote control method includes the following steps: as shown in
在图3中,步骤310设定至少一条件参数于步骤320之前,只是其中一种实施方式。步骤310亦可以于步骤320执行的同时或之后执行。而在实际操作上,步骤310的条件参数可以为预设的数据,不需要在操作时才设定。而另一种实际操作方式中,多媒体系统在操作中依据操作模式的需要,自行设定或调整条件参数。另外,条件参数亦可以由使用者设定的。总之,条件参数与该多媒体系统的一操作模式的至少一控制操作相关联。In FIG. 3 , step 310 sets at least one condition parameter before
在本发明以下的实施例中,多媒体系统可具有一个至多种操作模式。以下,首先以该操作模式为一播放模式为例以公开本发明的遥控方法的一实施方式;接着,以该操作模式为一命令选项模式以公开本发明的遥控方法的另一实施方式。请注意,有关条件参数的设定,可以不同的方式实际操作;故此,以下的实施例所叙述的设定条件参数的步骤,其执行次序亦可如上述般的以各种次序或方式实际操作。In the following embodiments of the present invention, the multimedia system may have one or more operation modes. Hereinafter, firstly, an embodiment of the remote control method of the present invention is disclosed by taking the operation mode as a playback mode as an example; then, another embodiment of the remote control method of the present invention is disclosed by taking the operation mode as a command option mode. Please note that the setting of the relevant condition parameters can be actually operated in different ways; therefore, the execution sequence of the steps of setting the condition parameters described in the following embodiments can also be actually operated in various orders or ways as described above .
图4A为一基于脸部检测的遥控方法应用于多媒体系统的播放模式时的一实施例的流程图。FIG. 4A is a flowchart of an embodiment when a face detection-based remote control method is applied to a playback mode of a multimedia system.
如步骤410所示,设定条件参数包括:(1)观众人数Na、(2)延迟时间Td及(3)未离席观众阈值p%,并与播放模式的”播放”、”停止”、”暂停”操作相关联。如步骤420所示,依据播放模式,通过检测装置检测存在的观众的人脸以及至少人脸的眼睛部分,以输出一人脸图像信号。如步骤430所示,依照人脸图像信号,分析所检测到的人脸以及至少人脸的一部分,以产生相对应的特性参数包括:(1)人脸数目及(2)睁开眼数目。如步骤440所示,比较依照人脸图像信号所产生的特性参数,与播放模式的至少一控制操作相关的条件参数,以产生指示信号给多媒体系统。此外,当步骤440执行之后,该方法的实施例可以从步骤420开始,重复进行检测,继续产生指示信号给多媒体系统。As shown in
本案实施例的步骤430以特性参数(1)人脸数目Nf及(2)睁开眼数目Ne为例,而步骤440中可依据该两特性参数,实施不同的比较方式,以产生指示信号给多媒体系统。请参考图4B,其为步骤440的一实施例的流程图。为方便解释,以下以观众人数Na为5,延迟时间Td为10分钟,未离席观众阈值p%为80%,作为例子说明。Step 430 of the embodiment of this case takes the characteristic parameters (1) the number of human faces Nf and (2) the number of open eyes Ne as an example, and in
在图4B中,如步骤441所示,首先比较人脸数目Nf是否实质上大于或等于观众人数Na的一百分比,例如是判断分析所得的人脸数目Nf(例如为4)是否大于或等于(Na×p%),即(5×0.8)=4人。若是,则执行步骤442以判断睁开眼数目Ne是否实质上大于或等于该观众人数Na的另一百分比,例如是判断分析所得的睁开眼数目Ne(如8只眼睛)是否大于或等于(2Na×p%),即(2×5×0.8)=8只眼睛。若步骤442中比较结果为是,所产生的指示信号代表”播放”操作,如步骤447及460所示。In FIG. 4B , as shown in
在步骤441或442中,若结果为否,则执行步骤443,以判断所检测的人脸数目Nf是否为0。若是,则进入步骤444以判断是否已开始暂停并且已经过了延迟时间Td。若步骤443或444的判断结果为否,则所产生的指示信号代表”暂停”操作,如步骤449及460所示。In
若步骤444的结果为是(例如已暂停了11分钟),则所产生的指示信号代表”停止”操作,如步骤448及460所示。If the result of
在上述图4A及图4B依据人的自然行为,以脸部及其部分基础,以改变多媒体系统的操作行为。例如,当观众人数及其睁开眼数目符合要求时,该多媒体系统就自动开始播放,如步骤441、442、447及460所示。又例如,当有一部分观众因故暂时离席时,而且离席人数很多,该多媒体系统就自动暂停播放,如步骤443、444、449及460所示。但若暂停了一段延迟时间Td,例如10分钟以后,观众人数N及其睁开眼数目Ne还是不符合要求的话,该多媒体系统就停止播放,如步骤444、448及460所示。In FIG. 4A and FIG. 4B mentioned above, the operation behavior of the multimedia system is changed based on the face and its parts according to the natural behavior of the person. For example, when the number of viewers and the number of opened eyes meet the requirements, the multimedia system will automatically start playing, as shown in
在上述图4A及图4B所述的实施例中,使用3个条件参数及2个特性参数为例,以示如何改变多媒体系统的操作行为。然而,亦可以采用至少一个条件参数,以至少脸部的一部分或是人脸为基础,以改变多媒体系统的操作行为。例如,检测存在的观众的人脸,并判断是否有两位观众存在;若是,则产生指示信号给多媒体系统以开始播放;在此,条件参数即为观众人数Na为2,也就是判断人脸数目Nf是否大于或等于2。又例如,条件参数的观众人数Na为2,及特性参数为眼睛的睁开眼数目Ne,判断存在的观众的睁开眼数目Ne是否大于或等于条件参数的观众人数Na,以决定是否指示多媒体系统开始播放。对于其他人脸的部分或人脸部分的各种组合,亦可采上述的方式,据以作为改变多媒体系统的操作行为的基础来源,以实现控制多媒体系统的目的。In the embodiment described above in FIG. 4A and FIG. 4B , three conditional parameters and two characteristic parameters are used as an example to illustrate how to change the operation behavior of the multimedia system. However, at least one conditional parameter can also be used to change the operation behavior of the multimedia system based on at least a part of the face or a human face. For example, detect the face of the audience that exists, and judge whether there are two audiences; If so, then generate an indication signal to the multimedia system to start playing; Whether the number of faces N f is greater than or equal to 2. For another example, the number of viewers Na of the condition parameter is 2, and the characteristic parameter is the number of open eyes Ne of the eyes, and it is judged whether the number of open eyes Ne of the existing audience is greater than or equal to the number of viewers Na of the condition parameter, and then Determines whether to instruct the multimedia system to start playback. For other human face parts or various combinations of human face parts, the above method can also be used as the basic source for changing the operation behavior of the multimedia system, so as to achieve the purpose of controlling the multimedia system.
如此,上述实施例可称为智能型(intelligent)的控制方式,也就是说,遥控系统可依据实际检测的结果,不必使用者亲自给予指示,而自动发出指示信号以调整多媒体系统的运作。Thus, the above-mentioned embodiment can be called an intelligent control method, that is to say, the remote control system can automatically send an instruction signal to adjust the operation of the multimedia system according to the actual detection result without the need for the user to give instructions.
接着,以多媒体系统的操作模式为一命令选项模式以公开本发明的遥控方法的另一实施方式。请参考图5,其示意依据本发明的一基于脸部检测的遥控系统应用于多媒体系统的命令选项模式时的一实施例。Next, another implementation of the remote control method of the present invention is disclosed by taking the operation mode of the multimedia system as a command option mode. Please refer to FIG. 5 , which illustrates an embodiment of a face detection-based remote control system applied to a command option mode of a multimedia system according to the present invention.
在多媒体系统于命令选项模式时,显示装置520显示一个或多个选项,以让使用者选择,使用者只要把眼睛注视在其中一个代表选项的物件,例如是画面上的图象、方块、区域或文字的区域,若眼睛所注视的时间符合相对应的条件的话,例如眼球注视在一遥控选项上超过或约3秒时,即可实现选取该选项的目的。When the multimedia system is in the command option mode, the display device 520 displays one or more options for the user to select. The user only needs to focus on one of the objects representing the options, such as an image, a block, or an area on the screen. Or the area of text, if the time that the eye gazes on meets corresponding conditions, for example, when eyeballs gaze on a remote control option for more than or about 3 seconds, the purpose of selecting this option can be realized.
在图5中,使用者以眼睛或眼球500所表示,而显示装置520显示的选项,例如是选项521、522。In FIG. 5 , the user is represented by eyes or eyeballs 500 , and the options displayed by the display device 520 are, for example, options 521 and 522 .
请同时参考图3及图5,以便于说明本实施例的遥控方法。如步骤310所示,设定一注视时间阈值TF0作为一条件参数,使的与命令选项模式的命令选取操作相关联。如步骤320所示,检测存在的人脸以及人脸的眼睛部分,并据以输出人脸图像信号,其中可利用针对眼睛的跟踪技术,例如眼球跟踪与定位技术(eyeballs tracking and location)。如步骤330所示,依据人脸图像信号,分析所检测到的人脸以及人脸的眼睛部分,产生相对应的多个特性参数,这些特性参数包括该眼睛所注视的方向(如以球状座标示的),以及该眼睛所注视的方向的注视时间Tf。Please refer to FIG. 3 and FIG. 5 at the same time to illustrate the remote control method of this embodiment. As shown in
接着,在步骤340中,首先判断该眼睛所注视的方向是否指向显示装置520的显示画面上;若是,则判断该眼睛所注视的方向所对应到在显示画面的位置并比较该位置是否对应到一代表遥控的选项,例如是图5中的选项522。若该眼睛所注视的方向对应到一代表遥控的选项,则判断该眼睛所注视的方向的注视时间Tf是否大于或等于一注视时间阈值TF0,如约3秒,以产生一比较结果。Then, in
在步骤350中,若步骤340的比较结果为是,也就是说:该眼睛所注视的方向对应到一代表遥控的选项而且眼睛的注视时间Tf(如约3.7秒)大于或等于一注视时间阈值TF0(如约3秒),则产生一指示信号代表该遥控选项的被选取。In
在本实施例的上述次步骤340中,检测装置540及图像分析与指示产生单元550可求得自检测装置540至人脸或眼睛的方向与距离,如箭头580所示,以及眼睛所注视的方向,如箭头590所示。此外,眼睛所注视的方向,是否落在显示画面的某一选项,可通过查表(table lookup)或空间中的几何运算等方式以实现。例如,预先建立一对应表(look-up table),表达眼睛所注视的各种方向(如以角度示的)与画面上各个区域(如以数字示的)的对应关系,以便通过查表方式得到眼睛所注视的某一方向所对应的画面区域。In the above-mentioned
依据上述在命令选项模式下执行的遥控方法的实施例,可衍生出其他有关选项的多种应用如下:According to the above-mentioned embodiment of the remote control method carried out in the command option mode, various applications of other relevant options can be derived as follows:
例如,当眼球的定位在”下一”(next)或”前一”(previous)上超过注视时间阈值TF0后,指示信号即代表开始将目前播放的画面,或例如是节目或曲目,改为下一个段落或是回到前一个段落持续到下一个播放选项开始执行为止。当眼球定位的位置已经不在任何遥控选项上,即停止执行该功能。For example, when the positioning of the eyeball exceeds the fixation time threshold T F0 on "next" or "previous", the indication signal means that the currently playing picture, or such as a program or track, will be changed. Continue for the next segment or return to the previous segment until the next playback option starts to execute. When the position of eyeball positioning is no longer on any remote control option, the execution of this function will stop.
当跟踪眼球的定位在”快转”(forward)或”倒转”(backward),如图5中的选项521及522上超过或等于注视时间阈值TF0时间后,即开始快转或是倒转画面,并且当功能开始执行时眼球定位仍继续在该遥控选项上时,加快快转或是倒转的速度,持续该遥控选项的功能直到下一个遥控选项的功能开始执行为止,或是当眼球定位的位置已经不在任何遥控选项上后即停止执行该功能。When the positioning of the tracking eyeball is in "fast forward" (forward) or "reverse" (backward), as shown in the options 521 and 522 in Figure 5, after exceeding or equal to the fixation time threshold T F0 time, it will start to fast forward or reverse the picture , and when the eyeball positioning is still on the remote control option when the function starts to execute, speed up the fast forward or reverse speed, and continue the function of the remote control option until the function of the next remote control option starts to execute, or when the eyeball positioning The function stops executing when the position is no longer on any remote control option.
当跟踪眼球的定位在音量选项,如图5中的”+”号的选项上超过或等于注视时间阈值TF0时间后,即开始增加(或是减少,对应于”-“号的选项)播放音量,持续该遥控选项功能直到下一个遥控选项的功能开始执行为止,或是当眼球定位的位置已经不在任何遥控选项上后即停止执行该功能。When the positioning of the tracking eyeball exceeds or equals the fixation time threshold T F0 time on the volume option, such as the option of the "+" sign in Figure 5, it will start to increase (or decrease, corresponding to the option of the "-" sign) to play Volume, continue the function of this remote control option until the function of the next remote control option starts to execute, or stop executing this function when the position of the eyeball is no longer on any remote control option.
眼球跟踪技术亦可利用在控制菜单(menu)上不同遥控选项的选择,例如遥控选项有10个,但是显示装置的画面一次只能呈现5个遥控选项,则可利用眼球跟踪定位的技术进行遥控选项换页的操作,再者若观赏多媒体播放时不需要遥控选项的菜单出现在显示画面上,亦可利用眼球跟踪定位技术将该菜单关闭或隐藏。Eye tracking technology can also use the selection of different remote control options on the control menu (menu). For example, there are 10 remote control options, but the screen of the display device can only present 5 remote control options at a time, then the technology of eye tracking and positioning can be used for remote control The operation of option page changing, and if the menu that does not need the remote control option appears on the display screen when watching multimedia playback, the menu can also be closed or hidden by using the eye tracking positioning technology.
另外,在实际操作时,上述的遥控方法的各实施例中,如步骤330至350或相对应的步骤可以用一软件程序或程序函式加以实际操作,再配合一记录条件参数与遥控功能及其关联关系的数据库或数据结构加以实现。此外,亦可以硬体或韧体,或混合的方式来实际操作。In addition, in actual operation, in each embodiment of the above-mentioned remote control method, such as
本发明上述各个实施例所公开的使用于多媒体系统的基于脸部检测的遥控系统及其方法及多媒体系统,依据人类身体某部分的自然行为,将的与遥控多媒体系统的操作相关联,以作为产生指示信号给多媒体系统的基础来源,从而改变多媒体系统的操作行为。如此,操作多媒体系统的方式将变得更为容易及直接,并可据该衍生智能型的控制方式,也就是说,遥控系统可依据实际检测的结果,不必使用者亲自给予指示,而自动发出指示信号以调整多媒体系统的运作。The face detection-based remote control system and its method for use in multimedia systems disclosed in the above-mentioned embodiments of the present invention and the multimedia system, according to the natural behavior of a certain part of the human body, associates with the operation of the remote control multimedia system as a An indication signal is generated to an underlying source of the multimedia system, thereby changing an operational behavior of the multimedia system. In this way, the method of operating the multimedia system will become easier and more direct, and an intelligent control method can be derived based on this, that is to say, the remote control system can automatically issue instructions based on the actual detection results without the need for the user to give instructions in person. Indicating signals to adjust the operation of the multimedia system.
综上所述,本发明虽以优选实施例公开如上,然其并非用以限定本发明,任何本领域技术人员,在不脱离本发明的精神和范围的情况下,可进行更动与修改,因此本发明的保护范围以所提出的权利要求所限定的范围为准。In summary, although the present invention is disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention is therefore defined by the scope of the appended claims.
Claims (23)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2006101365569A CN101169897A (en) | 2006-10-25 | 2006-10-25 | Multimedia system, face detection remote control system and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2006101365569A CN101169897A (en) | 2006-10-25 | 2006-10-25 | Multimedia system, face detection remote control system and method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101169897A true CN101169897A (en) | 2008-04-30 |
Family
ID=39390508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2006101365569A Pending CN101169897A (en) | 2006-10-25 | 2006-10-25 | Multimedia system, face detection remote control system and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101169897A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101888495A (en) * | 2010-06-24 | 2010-11-17 | 深圳创维-Rgb电子有限公司 | Method and system for controlling working state of television, and television |
| CN102109903A (en) * | 2010-12-28 | 2011-06-29 | 中兴通讯股份有限公司 | Method and device for protecting eyesight |
| CN102724604A (en) * | 2012-06-06 | 2012-10-10 | 北京中自科技产业孵化器有限公司 | Sound processing method for video meeting |
| CN103339963A (en) * | 2011-02-01 | 2013-10-02 | Nec卡西欧移动通信株式会社 | Electronic device |
| CN103648042A (en) * | 2013-11-27 | 2014-03-19 | 乐视致新电子科技(天津)有限公司 | Character output control method and device |
| CN103974107A (en) * | 2013-01-28 | 2014-08-06 | 海尔集团公司 | Television eye movement control method and device and television |
| CN104424825A (en) * | 2013-08-30 | 2015-03-18 | 鸿富锦精密工业(深圳)有限公司 | Remote teaching method and system |
| CN104808946A (en) * | 2015-04-29 | 2015-07-29 | 天脉聚源(北京)传媒科技有限公司 | Image playing and controlling method and device |
| CN106104650A (en) * | 2014-03-12 | 2016-11-09 | 微软技术许可有限责任公司 | Remote Device Control is carried out via gaze detection |
| CN109983755A (en) * | 2016-06-30 | 2019-07-05 | 北方公司 | The image capture system focused automatically, device and method are tracked based on eyes |
-
2006
- 2006-10-25 CN CNA2006101365569A patent/CN101169897A/en active Pending
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101888495A (en) * | 2010-06-24 | 2010-11-17 | 深圳创维-Rgb电子有限公司 | Method and system for controlling working state of television, and television |
| CN102109903A (en) * | 2010-12-28 | 2011-06-29 | 中兴通讯股份有限公司 | Method and device for protecting eyesight |
| CN103339963A (en) * | 2011-02-01 | 2013-10-02 | Nec卡西欧移动通信株式会社 | Electronic device |
| US9241123B2 (en) | 2011-02-01 | 2016-01-19 | Nec Corporation | Electronic device |
| CN102724604A (en) * | 2012-06-06 | 2012-10-10 | 北京中自科技产业孵化器有限公司 | Sound processing method for video meeting |
| CN102724604B (en) * | 2012-06-06 | 2014-11-26 | 北京中自投资管理有限公司 | Sound processing method for video meeting |
| CN103974107A (en) * | 2013-01-28 | 2014-08-06 | 海尔集团公司 | Television eye movement control method and device and television |
| CN104424825A (en) * | 2013-08-30 | 2015-03-18 | 鸿富锦精密工业(深圳)有限公司 | Remote teaching method and system |
| CN103648042A (en) * | 2013-11-27 | 2014-03-19 | 乐视致新电子科技(天津)有限公司 | Character output control method and device |
| CN106104650A (en) * | 2014-03-12 | 2016-11-09 | 微软技术许可有限责任公司 | Remote Device Control is carried out via gaze detection |
| CN104808946A (en) * | 2015-04-29 | 2015-07-29 | 天脉聚源(北京)传媒科技有限公司 | Image playing and controlling method and device |
| CN109983755A (en) * | 2016-06-30 | 2019-07-05 | 北方公司 | The image capture system focused automatically, device and method are tracked based on eyes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4909854B2 (en) | Electronic device and display processing method | |
| CN101490762B (en) | Device and method for playing audio-video content | |
| JP5057918B2 (en) | Electronic device and scene type display method | |
| JP4331240B2 (en) | Electronic apparatus and image display method | |
| JP6030945B2 (en) | Viewer video display control device, viewer video display control method, and viewer video display control program | |
| US20100275164A1 (en) | Authoring apparatus | |
| CN101860704A (en) | Display device for automatically turning off image display and its implementation method | |
| US20100104146A1 (en) | Electronic apparatus and video processing method | |
| JP2009038680A (en) | Electronic device and face image display method | |
| JP4856105B2 (en) | Electronic device and display processing method | |
| TW200812376A (en) | Face-detection-based remote-control system and method and face-detection-based remote-controllable multimedia system | |
| CN101169897A (en) | Multimedia system, face detection remote control system and method thereof | |
| JP2011211481A (en) | Video/audio player | |
| US20110110564A1 (en) | Electronic apparatus and image display | |
| CN101599241A (en) | information processing equipment | |
| JP2005250322A (en) | Display device | |
| JP5330551B2 (en) | Electronic device and display processing method | |
| JP4625862B2 (en) | Authoring apparatus and authoring method | |
| JP5039020B2 (en) | Electronic device and video content information display method | |
| US8401361B2 (en) | Information recording and reproducing apparatus and method using menu of face images included in recording information | |
| CN113572986A (en) | Course recording and broadcasting guiding method and device, readable storage medium and teaching all-in-one machine | |
| JP2010263374A (en) | Recording / reproducing apparatus and recording / reproducing method | |
| JP2003333449A (en) | Composite picture output apparatus | |
| US20100204980A1 (en) | Real-time translation system with multimedia display function and method thereof | |
| CN115529482B (en) | A method for preventing screen recording and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: YINGTEWEI DIGITAL TECH CO., LTD. Free format text: FORMER OWNER: YOULI INFORMATION CO LTD Effective date: 20080711 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20080711 Address after: Taipei City, Taiwan, China Applicant after: Yingte Digital Tech Co., Ltd. Address before: Taipei City, Taiwan, China Applicant before: Youli Information Co., Ltd. |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |