JP2002176632A - Mobile phone and image transmission method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile phone and an image transmission method, with which image transmission with privacy protected can be realized. SOLUTION: The mobile phone is provided with a three-dimensional image device 4, that reads a three-dimensional face image of a face part corresponding to a changed part of a face, when expression of the face of a talker photographed by an AR camera 3 is subject to changes and generates a motion vector of the face part and the mobile phone transmits the motion vector.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a portable telephone with a video telephone function having a privacy protection function and an image transmission method.

[0002]

2. Description of the Related Art A conventional portable telephone does not have a videophone function, and transmits only a voice uttered by a caller to the other party. Therefore, the conventional mobile phone cannot transmit the video of the caller to the other party.

[0003] In a conventional video telephone, a caller and a background are photographed using a camera, and a face image and a background image of the caller are transmitted to the other party. However, in this case, the current face image of the caller is transmitted.
From the viewpoint of privacy protection, Japanese Patent Application Publication No. 219500 discloses a videophone that transmits not a current face image but a face image that has been shot and registered in advance to the other party. An information processing apparatus that combines a plurality of images and transmits a face image different from the current face image to the other party is disclosed in
No. 4,486,161.

Therefore, if the technology of the information processing device is applied to a mobile phone equipped with a camera, a mobile phone with a videophone function capable of protecting privacy can be constructed. Since the technology disclosed in the above publication is a synthesis between two-dimensional images, it requires an extremely large amount of image processing, and it is generally not practical to mount the technology on a mobile phone having a small processing capability.

[0005]

Since the conventional mobile telephone is configured as described above, if a camera is mounted,
The current face image of the caller can be transmitted to the other party. However, from the viewpoint of privacy protection, it is desirable to have a function of transmitting a face image different from the current face image to the other party. However, since a great deal of image processing is required, a mobile phone generally having a small processing capability is required. Had problems such as difficulty in mounting.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to provide a portable telephone and an image transmission method capable of realizing image transmission with privacy protected.

[0007]

According to a portable telephone according to the present invention, when there is a change in the expression of the face of a caller photographed by photographing means, a three-dimensional facial part corresponding to the changed part is obtained. A motion vector generating means for reading a face image from a registration means and generating a motion vector of the face part is provided, and the motion vector is transmitted.

The portable telephone according to the present invention analyzes the voice detected by the voice detecting means, reads out the three-dimensional face image of the face part corresponding to the analysis result from the registration means, and reads the movement of the face part. A motion vector generating means for generating a vector is provided, and the motion vector is transmitted.

A portable telephone according to the present invention transmits a three-dimensional face image of all parts of a face registered in a registration means at the beginning of a call, and thereafter transmits a motion vector.

[0010] In the portable telephone according to the present invention, when the transmitting means transmits the background image together with the three-dimensional face image,
A background image changing means for receiving the change of the background image is provided.

In the portable telephone according to the present invention, when a mode for changing the display mode in response to a caller's operation is set, the motion vector generating means sets the motion vector of the face part in response to the caller's key operation. Is generated.

In the portable telephone according to the present invention, when a mode for changing the display mode according to a change procedure registered in advance is set, the motion vector generating means generates a motion vector of a face part according to the change procedure. It is something to do.

[0013] In the portable telephone according to the present invention, the registration means registers a three-dimensional face image of the caller or a three-dimensional face image of the character, which is photographed in advance.

A portable telephone according to the present invention is adapted to generate a motion vector of a part where the shape, color or display direction of a part changes.

According to the image transmission method of the present invention, when a caller is photographed and a facial expression of the caller changes, a three-dimensional face image of a face part corresponding to the changed portion is displayed on the portable telephone. , A motion vector of the face part is generated, and the motion vector is transmitted.

According to the image transmitting method of the present invention, a voice uttered by a caller is detected and analyzed, and a three-dimensional face image of a face part corresponding to the result of the analysis is read out from a mobile phone.
A motion vector of the face part is generated, and the motion vector is transmitted.

In the image transmission method according to the present invention, a three-dimensional face image of all parts of the face registered in the mobile phone is transmitted at the beginning of a call, and thereafter, a motion vector is transmitted.

In the image transmission method according to the present invention, when a background image is transmitted together with a three-dimensional face image, a change in the background image is accepted.

According to the image transmission method of the present invention, when a mode for changing the display mode in response to a caller's operation is set, a motion vector of a face part is generated in response to a caller's key operation. It was made.

In the image transmission method according to the present invention, when a mode for changing a display mode is set according to a change procedure registered in advance, a motion vector of a face part is generated according to the change procedure. Things.

According to the image transmission method of the present invention, a three-dimensional face image of a caller or a character 3
A three-dimensional face image is registered.

According to the image transmission method of the present invention, a motion vector is generated for a part where the shape, color or display direction of a part changes.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. Embodiment 1 FIG. FIG. 1 is a block diagram showing a mobile phone according to a first embodiment of the present invention. In the figure, reference numeral 1 denotes an image registration unit for registering a three-dimensional face image in which facial parts are made into parts into a memory 2; This is the memory of the telephone. The image registration unit 1 and the memory 2 constitute a registration unit.

Reference numeral 3 denotes an AR camera (photographing means) for photographing the face of the caller. Reference numeral 4 denotes a face corresponding to the changed portion when the facial expression of the caller photographed by the AR camera 3 changes. A three-dimensional image device that reads a three-dimensional face image of the part (for example, eyes, eyebrows, nose, mouth, ear, hair, contour, etc.) from the memory 2 and generates a motion vector of the face part (motion vector generation) Means, 5 is a transmitting unit (transmitting means) for transmitting the motion vector generated by the three-dimensional image device 4. FIG. 2 is a flowchart showing an image transmission method according to the first embodiment of the present invention.

Next, the operation will be described. The caller is not the current face image of the caller, but a three-dimensional face image of the caller (for example, a beautiful face image with makeup applied) or a three-dimensional face image of the character (for example, animation) If the user wishes to transmit the face image of a person appearing in the mobile phone to the other party, the image registration unit 1 stores the three-dimensional face image (all parts of the face) in the memory 2 of the mobile phone.
(Step ST1).

Thus, when the mobile phone is connected to the other party's video phone or the like, the transmitting unit 5 starts transmitting the three-dimensional face image (all parts of the face) registered in the memory 2 ( Step ST2). By this transmission, a three-dimensional face image is displayed on the other party's videophone. When the AR camera 3 captures the face of the caller, the three-dimensional image device 4 determines whether the facial expression of the caller captured by the AR camera 3 has changed (step S).
T3). For example, it is determined whether the mouth and eyes of the caller are moving.

When there is a change in the expression of the face of the caller, the three-dimensional image device 4 changes the face parts (for example, eyes, eyebrows, nose, mouth, ear,
A three-dimensional face image of the hair, outline, etc.) is read from the memory 2 and a motion vector of the face part is generated (step ST4). For example, when the determination result of the three-dimensional image device 4 indicates the movement of the mouth, a motion vector for changing the shape of the display component of the mouth is generated so that the mouth can be opened or closed. FIG. 3 shows a state in which the mouth is open, and FIG. 4 shows a state in which the mouth is closed.

If the determination result of the three-dimensional image device 4 indicates a swing, the display direction (display angle) of the contour is changed. When the three-dimensional image device 4 generates the motion vector (Step ST)
5), and transmits the motion vector to the other party (step ST6). By transmitting the motion vector, the shape of the corresponding part of the three-dimensional face image already displayed on the other party's videophone is changed. As a result, the facial expression of the three-dimensional face image changes in accordance with the change in the facial expression of the caller captured by the AR camera.

As is apparent from the above, the first embodiment
According to the above, when there is a change in the expression of the face of the caller photographed by the AR camera 3, the three-dimensional face image of the face part corresponding to the changed portion is read from the memory 2,
Since the three-dimensional image device 4 for generating the motion vector of the face part is provided and configured to transmit the motion vector, it is possible to achieve an effect of realizing an image transmission with privacy protected. Further, since the movement of the sender is reflected on the three-dimensional model character, a sense of "sharing the place" is generated, and the communication is lubricated.

Further, in the first embodiment, since a motion vector for changing the display mode of the parts of the three-dimensional face image in which the face parts are made into parts is generated and transmitted, the two-dimensional image is transmitted. The data transmission amount can be significantly reduced as compared with the related art. As a result, an excellent effect of easily changing the facial expression of a three-dimensional face image displayed on a videophone or the like of the other party can be easily obtained even with a mobile phone generally having a small processing capacity. You.

Embodiment 2 FIG. 5 is a configuration diagram showing a mobile phone according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG. 1 denote the same or corresponding parts, and a description thereof will be omitted. Reference numeral 6 denotes a voice detection unit (voice detection means) for detecting a voice uttered by a caller, and 7 analyzes a voice detected by the voice detection unit 6 and stores a three-dimensional face image of a face part according to the analysis result. 2 is a three-dimensional image device (motion vector generation means) that generates a motion vector of the face part by reading from the image part 2.

Next, the operation will be described. In the first embodiment, when the three-dimensional image device 4 changes the facial expression of the caller, the three-dimensional image of the face part corresponding to the changed part is read from the memory 2 and the three-dimensional image is read. Although the generation of the motion vector of the face part has been described, the three-dimensional image device 7 analyzes the voice detected by the voice detection unit 6 and stores the three-dimensional face image of the face part according to the analysis result in the memory. 2, the motion vector of the face part may be generated.

For example, when the voice detection unit 6 does not detect voice, the mouth is kept closed, and when voice is detected, a motion vector that repeats opening and closing of the mouth is generated. Also, if the tone of the voice is detected, for example, in an angry voice,
Her hair stood, her eyebrows hung, her laughing voice, her eyes,
Eyebrows and mouth can make a laughing face, and sad voices can have graphic effects such as tearing, tearing, or coloring. Thus, the same effect as in the first embodiment can be obtained.

Embodiment 3 FIG. 6 is a configuration diagram showing a mobile phone according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG. 1 denote the same or corresponding parts, and a description thereof will be omitted. Reference numeral 8 denotes a background image changing unit (background image changing unit) that receives a change in the background image when the transmitting unit 5 transmits the background image together with the three-dimensional face image.

Next, the operation will be described. In the first and second embodiments, the transmission unit 5 is registered in the
Although the transmission of the three-dimensional face image has been described, the background image of the caller may be transmitted together with the three-dimensional face image. At that time, from the viewpoint of privacy protection, the background image may be changed to a background image different from the actual background image by using the background image changing unit 8. As the background image to be transmitted, for example, a wallpaper, an illustration, a photograph, a character, a moving image, a character, or the like may be registered in the memory 2 in advance.

Embodiment 4 FIG. In the above-described first to third embodiments, a case has been described in which a motion vector of a face part corresponding to a changed part of an expression is generated. However, a mode is set in which the operation of the caller is accepted to change the display mode. In this case, a motion vector of a face part may be generated according to a key operation of a caller. For example, all parts of an image such as an angry face, a crying face, a smiling face, a serious face, a troubled face, an erroneous face, a face falling in love (for example, eyes with a heart symbol) are displayed, and selection of an arbitrary part is accepted. , May be replaced with parts of the current face image, for example,
When the eye part of the current face image is operated, the eyes may be winked.

Embodiment 5 In the above-described first to third embodiments, a case has been described in which a motion vector of a face part corresponding to a change portion of an expression is generated. However, a mode in which a display mode is changed according to a previously registered change procedure is set. If there is a change procedure (for example, using a random function, the order of movement of the mouth and eyes can be determined, so opening and closing of the mouth, blinking, shaking of the hair, etc. are appropriately repeated,
The eyeball may also be moved up, down, left and right from time to time, or the mouth may be opened and closed or the hair may sway satisfactorily with a smile all the time.

[0038]

As described above, according to the present invention, when there is a change in the expression of the face of the caller photographed by the photographing means, the three-dimensional face of the face part corresponding to the changed part is obtained. A motion vector generation unit that reads an image from the registration unit and generates a motion vector of the face part;
Since the motion vector is configured to be transmitted, there is an effect that privacy-protected image transmission can be realized. Further, the data transmission amount can be significantly reduced as compared with the conventional technique for transmitting a two-dimensional image.
As a result, an excellent effect of easily changing the facial expression of a three-dimensional face image displayed on a videophone or the like of the other party can be easily obtained even with a mobile phone generally having a small processing capacity. You.

According to the present invention, the voice detected by the voice detecting means is analyzed, the three-dimensional face image of the face part corresponding to the analysis result is read out from the registration means, and the motion vector of the face part is calculated. Since the motion vector generating means for generating the motion vector is provided and the motion vector is transmitted, there is an effect that the privacy-protected image transmission can be realized.

According to the present invention, a three-dimensional face image of all parts of the face registered in the registration means is transmitted at the beginning of a call, and thereafter, a motion vector is transmitted. There is an effect that image transmission can be realized even with a telephone.

According to the present invention, when the background image is transmitted together with the three-dimensional face image by the transmitting means, the background image changing means for receiving the change of the background image is provided. There is an effect that a different background image can be transmitted.

According to the present invention, when the mode for changing the display mode by accepting the operation of the caller is set,
Since the motion vector generating means is configured to generate a motion vector of a face part according to a key operation of a caller,
There is an effect that the three-dimensional face image can be changed by the caller's intention.

According to this invention, when the mode for changing the display mode is set according to the change procedure registered in advance, the motion vector generating means generates the motion vector of the face part according to the change procedure. Therefore, there is an effect that the three-dimensional face image can be changed uniformly.

According to the present invention, since the registration means is configured to register the three-dimensional face image of the caller or the three-dimensional face image of the character photographed in advance, it is possible to protect the privacy of the caller. There is an effect that can be done.

According to the present invention, since a motion vector is generated for a part where the shape, color or display direction of a part changes, there is an effect that a real face image can be realized.

According to the present invention, the face of the caller is photographed,
When there is a change in the expression of the caller's face, a three-dimensional face image of the face part corresponding to the changed part is read from the mobile phone, and a motion vector of the face part is generated. Has been configured to send
There is an effect that image transmission with privacy protected can be realized.

According to the present invention, the voice uttered by the caller is detected and analyzed, and the face part 3 corresponding to the analysis result is detected.
Since the two-dimensional face image is read from the mobile phone, a motion vector of the face part is generated, and the motion vector is transmitted, there is an effect that privacy-protected image transmission can be realized.

According to the present invention, at the beginning of a call, a three-dimensional face image of all parts of the face registered in the mobile phone is transmitted,
After that, since it was configured to transmit the motion vector,
There is an effect that image transmission can be realized even with a mobile phone having a small processing capability.

According to the present invention, when the background image is transmitted together with the three-dimensional face image, a change in the background image is accepted, so that a background image different from the actual background image can be transmitted. There is an effect that can be done.

According to the present invention, when a mode for changing the display mode in response to a caller's operation is set,
Since the configuration is such that the motion vector of the face part is generated according to the key operation of the caller, there is an effect that the three-dimensional face image can be changed by the caller's intention.

According to the present invention, when the mode for changing the display mode according to the change procedure registered in advance is set, the motion vector of the face part is generated according to the change procedure. There is an effect that the three-dimensional face image can be changed uniformly.

According to the present invention, since the three-dimensional face image of the caller or the three-dimensional face image of the character which has been photographed in advance is registered, the privacy of the caller can be protected. is there.

According to the present invention, since a motion vector is generated for a part where the shape, color or display direction of a part changes, there is an effect that a real face image can be realized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a mobile phone according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an image transmission method according to the first embodiment of the present invention.

FIG. 3 is an explanatory diagram showing a state in which a mouth is opened.

FIG. 4 is an explanatory diagram showing a state in which a mouth is closed.

FIG. 5 is a configuration diagram showing a mobile phone according to a second embodiment of the present invention.

FIG. 6 is a configuration diagram showing a mobile phone according to a third embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 1 image registration unit (registration unit), 2 memory (registration unit), 3 AR camera (imaging unit), 4 three-dimensional image device (motion vector generation unit), 5 transmission unit (transmission unit), 6 audio detection unit (audio) Detecting means), three-dimensional image device (motion vector generating means), 8 background image changing unit (background image changing means).

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H04Q 7/38 H04B 7/26 Q H04M 1/00 109M 1/725 (72) Inventor Tateki Iwasaki Marunouchi, Chiyoda-ku, Tokyo 2-3-2, Mitsui Electric Co., Ltd. (72) Inventor Tomoki Tsubata 2-3-2, Marunouchi, Chiyoda-ku, Tokyo 2-3-2 Within Mitsui Electric Co., Ltd. (72) Daisuke Iizawa 2-chome, Marunouchi, Chiyoda-ku, Tokyo No. 2-3 F-term in Mitsubishi Electric Corporation (reference) 5B050 AA08 BA06 BA08 BA09 BA10 BA11 BA12 CA05 CA07 EA07 EA19 EA24 EA28 FA02 5C064 AA01 AB04 AC04 AC06 AC12 AC16 AD04 5D045 AB04 5K027 AA11 BB09 HH23 AH30 A04A30 DD52 DD54 FF23 FF31 HH23 KK15

Claims (16)

[Claims] A registration unit for registering a three-dimensional face image in which a face part is made into a part; a photographing unit for photographing a caller;
When there is a change in the expression of the face of the caller photographed by the photographing means, a three-dimensional face image of the face part corresponding to the changed part is read out from the registration means, and the movement of the face part is read. A mobile phone comprising: a motion vector generation unit that generates a vector; and a transmission unit that transmits the motion vector generated by the motion vector generation unit. 2. A registration unit for registering a three-dimensional face image in which facial parts are made into parts, a voice detection unit for detecting a voice uttered by a caller, and analyzing the voice detected by the voice detection unit. A three-dimensional face image of the face part corresponding to the analysis result is read from the registration means, and a motion vector generation means for generating a motion vector of the face part, and a motion vector generated by the motion vector generation means, A mobile phone comprising: a transmitting unit for transmitting. 3. The transmitting means for transmitting a three-dimensional face image of all parts of the face registered in the registering means at the beginning of a call, and thereafter transmitting a motion vector. 2. The mobile phone according to 2. 4. The apparatus according to claim 1, further comprising a background image changing unit for receiving a change of the background image when the background image is transmitted together with the three-dimensional face image by the transmitting unit. The mobile phone according to any one of the preceding claims. 5. A motion vector generation means for generating a motion vector of a face part in response to a key operation of a caller when a mode for changing a display mode by accepting a caller's operation is set. The mobile phone according to any one of claims 1 to 4, wherein the mobile phone is a mobile phone. 6. A motion vector generating means, when a mode for changing a display mode according to a previously registered change procedure is set, generates a motion vector of a face part according to the change procedure. Claim 1
The mobile phone according to any one of claims 1 to 4. 7. The registration means stores the caller's 3
The mobile phone according to any one of claims 1 to 6, wherein a three-dimensional face image or a three-dimensional face image of a character is registered. 8. A motion vector generating means, comprising:
The mobile phone according to any one of claims 1 to 7, wherein a motion vector of a portion where a color or a display direction changes is generated. 9. While registering a three-dimensional face image in which facial parts are made into parts into a mobile phone, photographing a caller, if there is a change in the expression of the caller's face, a change is made to the changed part. Reading a three-dimensional face image of the corresponding face part from the mobile phone, generating a motion vector of the face part,
An image transmission method for transmitting the motion vector. 10. While registering a three-dimensional face image in which a face part is made into a part in a mobile phone, a voice uttered by a caller is detected and analyzed, and the three-dimensional face image corresponding to the analysis result is detected.
An image transmission method for reading a three-dimensional face image from the mobile phone, generating a motion vector of the face part, and transmitting the motion vector. 11. The image according to claim 9, wherein a three-dimensional face image of all parts of the face registered in the mobile phone is transmitted at the beginning of the call, and a motion vector is transmitted thereafter. Transmission method. 12. The image transmission according to claim 9, wherein when a background image is transmitted together with the three-dimensional face image, a change of the background image is received. Method. 13. A motion vector of a face part is generated according to a key operation of a caller when a mode for changing a display mode by accepting a caller's operation is set. To any one of claims 12 to
The image transmission method described in the item. 14. A method according to claim 9, wherein when a mode for changing a display mode according to a change procedure registered in advance is set, a motion vector of a face part is generated according to the change procedure. Item 13. The image transmission method according to any one of Items 12. 15. The image according to claim 9, wherein a three-dimensional face image of the caller or a three-dimensional face image of the character, which is captured in advance, is registered. Transmission method. 16. The image transmission method according to claim 9, wherein a motion vector of a portion where a shape, a color, or a display direction of a part changes is generated.