KR20240000327A - An electronic device including a projection unit for projecting an image and providing a video call function and control method thereof - Google Patents

Abstract

Provided are an electronic device providing a video call function and a control method thereof. The electronic device comprises: a projection unit; a camera; a sensor for acquiring information on the distance from an external object; a memory for storing at least one instruction; and at least one processor connected to the projection unit, the camera, the sensor, and the memory and for controlling the electronic device. The at least one processor acquires, by executing the at least one instruction, a first user image by filming a user through the camera when a video call function is executed. The at least one processor calculates a horizontal correction angle and a vertical correction angle based on the first user image, a rotation angle of the camera, a projection angle of the projection unit, and a sensing value acquired through the sensor. The at least one processor acquires the user's front image based on the first user image, the horizontal correction angle, and the vertical correction angle. The at least one processor controls the projection unit to project a video call screen including the acquired user's front image.

Description

An electronic device including a projection unit for projecting an image and providing a video call function and control method thereof}

The present disclosure relates to an electronic device including a projection unit that projects an image and a control method thereof, and to an electronic device that provides a video call function through a projection unit and a control method thereof.

Recently, due to the development of communication technology, a video call function that allows video calls with external terminal devices has become common. In particular, video call functions are provided not only in user terminals such as smartphones, but also in various electronic devices such as TVs in the home.

When performing a video call with an external terminal device, the electronic device can capture a user video using a camera and perform a video call using the captured user video. However, in electronic devices such as projectors, the camera is not located in the direction the user is looking, but is located to the side or behind the user. Because of this, when making a video call using an electronic device such as a projector, the user's face is filmed from the side rather than the front, so there is a limitation in that the other party cannot see the user's face and make a video call. .

According to an embodiment of the present disclosure, an electronic device providing a video call function includes: a projection unit; camera; A sensor for obtaining information about the distance to an external object; a memory storing at least one instruction; and at least one processor connected to the projection unit, the camera, the sensor, and the memory, and configured to control the electronic device. When the video call function is executed by executing the at least one instruction, the at least one processor acquires a first user image by photographing the user through the camera. The at least one processor calculates a horizontal correction angle and a vertical correction angle based on the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. And, the at least one processor acquires a frontal image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle. And, the at least one processor controls the projection unit to project a video call screen including the obtained frontal image of the user.

According to an embodiment of the present disclosure, a method of controlling an electronic device including a projection unit for projecting an image and providing a video call function includes, when the video call function is executed, capturing the user through a camera to produce a first user image. acquiring; calculating a horizontal correction angle and a vertical correction angle based on the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor of the electronic device; Obtaining a frontal image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle; and projecting a video call screen including the acquired frontal image of the user through the projection unit.

According to an embodiment of the present disclosure, in a computer-readable medium including a projection unit for projecting an image and storing a program for executing a control method of an electronic device providing a video call function, the control method includes: When the call function is executed, acquiring a first user image by photographing the user through a camera; calculating a horizontal correction angle and a vertical correction angle based on the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor of the electronic device; Obtaining a frontal image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle; and projecting a video call screen including the acquired frontal image of the user through the projection unit.

1 is a diagram for explaining an electronic device that provides a video call function according to an embodiment of the present disclosure;
2A is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure;
2B is a diagram illustrating the appearance of an electronic device according to an embodiment of the present disclosure;
3 is a diagram illustrating a method of correcting a user image captured by an electronic device while performing a video call, according to an embodiment of the present disclosure;
4 is a block diagram illustrating a configuration for correcting a user's video while performing a video call, according to an embodiment of the present disclosure;
FIG. 5A is a diagram for explaining a method of calculating a first height from the floor to the center of a projection surface according to an embodiment of the present disclosure;
FIG. 5B is a diagram illustrating a method of calculating a second height from the floor to the center of the user's face according to an embodiment of the present disclosure;
6 is a diagram for explaining a method of calculating a horizontal correction angle according to an embodiment of the present disclosure;
7 is a diagram for explaining a method of calculating a vertical correction angle according to an embodiment of the present disclosure;
FIG. 8 is a diagram illustrating a method of acquiring a frontal image using a horizontal correction angle and a vertical correction angle according to an embodiment of the present disclosure;
9 is a diagram for explaining a method of providing an avatar image according to an embodiment of the present disclosure;
FIGS. 10A and 10B are diagrams for explaining a method of moving an electronic device by identifying whether it is located in a space where video calls are available, according to an embodiment of the present disclosure;
11 is a flowchart illustrating a control method of an electronic device providing a video call function according to an embodiment of the present disclosure;
12 is a block diagram specifically illustrating the configuration of an electronic device according to an embodiment of the present disclosure, and
Figure 13 is a perspective view showing the appearance of an electronic device according to an embodiment of the present disclosure.

Since these embodiments can be modified in various ways and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numbers may be used for similar components.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted.

In addition, the following examples may be modified into various other forms, and the scope of the technical idea of the present disclosure is not limited to the following examples. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to completely convey the technical idea of the present disclosure to those skilled in the art.

The terms used in this disclosure are merely used to describe specific embodiments and are not intended to limit the scope of rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

In the present disclosure, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” includes (1) at least one A, (2) at least one B, or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component).

On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

The expression “configured to” used in the present disclosure may mean, for example, “suitable for,” “having the capacity to,” depending on the situation. ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware.

Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented with at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and areas in the drawing are schematically drawn. Accordingly, the technical idea of the present invention is not limited by the relative sizes or spacing drawn in the attached drawings.

1 is a diagram for explaining an electronic device that provides a video call function according to an embodiment of the present disclosure. The electronic device 100 shown in FIG. 1 may be an image projection device such as a projector that projects an image onto a wall or screen.

According to an embodiment of the present disclosure, when the video call function is executed, the electronic device 100 may obtain a first user image by photographing the user 10 through a camera that can rotate 360 degrees. At this time, the first user image may be an image taken of the side of the user 10 when the user 10 is looking at the projection surface (particularly, the other party) 50.

Additionally, the electronic device 100 may obtain a frontal image of the user 10 looking straight ahead using the first user image. Specifically, the electronic device 100 may calculate the horizontal correction angle and the vertical correction angle based on the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. At this time, the horizontal correction angle is an angle for rotating and correcting the first user image in the horizontal direction, and the vertical correction angle is an angle for rotating and correcting the second user image in the vertical direction. The electronic device 100 may acquire a frontal image of the user 10 based on the first user image, the horizontal correction angle, and the vertical correction angle.

And, as shown in FIG. 1, the electronic device 100 may project a video call screen including the obtained frontal image of the user 10 on the projection surface 50. Because of this, the user 10 can provide a video call using the frontal image even if the electronic device 100 captures the user 10 from the side direction of the user 10, thereby providing a more natural video call screen. It becomes possible.

Additionally, the electronic device 100 may capture a second user video while performing a video call. Additionally, the electronic device 100 may correct the frontal image by analyzing the second user image in order to reflect the user's 10's natural movements (eg, head turning, lip movement, etc.) in the frontal image. Specifically, the electronic device 100 may correct the frontal image by analyzing the feature points (e.g., eyes, nose, mouth, etc.) of the second user image to obtain information about the rotation angle of the user's face or the movement of the feature point. .

As a result, the electronic device 100 can provide a natural user image by correcting the frontal image by reflecting the user's movements while making a video call.

FIG. 2A is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 2A, the electronic device 100 may include a projection unit 112, a memory 113, a sensor unit 121, a camera 122, and at least one processor 111. However, the configuration of the electronic device 100 shown in FIG. 2A is only an example, and of course, other configurations may be added or some configurations may be omitted.

The projection unit 112 is configured to project light for expressing an image to the outside. The projection unit 112 may include various detailed components such as a light source, projection lens, and reflector. The operating method and detailed configuration of the projection unit 112 will be described in detail again in the following section. In particular, while the electronic device 100 is making a video call with an external device, the projection unit 112 can project the video call screen on the projection surface.

The memory 113 may store at least one instruction for controlling the electronic device 100. In particular, the memory 120 can store data necessary for a module to correct a user's image to perform various operations while making a video call. The module for correcting the user's image while performing a video call includes a distance information acquisition module 410, a height information acquisition module 420, a correction angle calculation module 430, and a front image acquisition module as shown in FIG. 4. It may include (440), an image analysis module (450), and an image correction module (350).

Meanwhile, the memory 113 may include a non-volatile memory that can maintain stored information even when power supply is interrupted, and a volatile memory that requires continuous power supply to maintain the stored information. A module for correcting the user's video while performing a video call may be stored in non-volatile memory.

The sensor unit 121 is configured to obtain status information of the electronic device 100 or information about the surrounding environment of the electronic device 100. In particular, the sensor unit 121 may include at least one distance sensor for obtaining information about the distance between the electronic device 100 and the object. At this time, at least one distance sensor may be located around the camera 122 or around the projection unit 112, but is not limited thereto.

At least one distance sensor acquires a sensing value for obtaining information about the distance between the electronic device 100 and the center of the projection surface or information about the distance between the electronic device 100 and the center of the face of the user 10. You can obtain the sensing value to do this.

The camera 122 is placed on one side of the electronic device 100 and can capture still images or moving images. In particular, the camera 122 may be placed on the side of the cylindrical electronic device 100, as shown in FIG. 2B. Additionally, the camera 122 may be rotatable 360 degrees. Accordingly, the camera 122 can capture a user located around the electronic device 100.

At least one processor 111 may be electrically connected to the memory 113 and control the overall functions and operations of the electronic device 100. When an event for performing a video call or an event for correcting a user's video during a video call is detected, the at least one processor 111 corrects the user's video while performing a video call stored in the non-volatile memory. The module may load data for performing various operations into volatile memory. At least one processor 111 may perform various operations using various modules based on data loaded into volatile memory. Here, loading refers to an operation of loading and storing data stored in non-volatile memory in volatile memory so that at least one processor 111 can access it.

In particular, the at least one processor 111 executes at least one instruction stored in the memory 113, and when the video call function is executed, the at least one processor 111 captures the user through the camera 122 to obtain a first user image. At least one processor 111 performs horizontal correction angle and vertical correction based on the first user image, the rotation angle of the camera 122, the projection angle of the projection unit 112, and the sensing value obtained through the sensor unit 121. Calculate the angle. At least one processor 111 obtains a front image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle. At least one processor 111 controls the projection unit 112 to project a video call screen including the acquired frontal image of the user.

In particular, at least one processor 111 may obtain information about the first distance between the electronic device 100 and the center of the projection surface based on the first sensing value obtained through the sensor unit 121. In addition, at least one processor 111 may obtain information about the second distance between the electronic device 100 and the user's face based on the second sensing value obtained through the sensor unit 121. Additionally, at least one processor 111 may calculate a horizontal correction angle based on information about the first distance, information about the second distance, and the first rotation angle of the camera 122. At this time, the first rotation angle of the camera 122 is the angle by which the camera 122 is rotated in the left and right directions based on the center of the projection surface.

In addition, at least one processor 111 provides information about the first height from the floor to the center of the projection surface based on the projection angle of the projection unit 112 and the second sensing value obtained through the sensor unit 121. It can be obtained. And, the at least one processor 111 provides information about the second height from the floor to the user's face based on the second rotation angle of the camera 122 and the second sensing value obtained through the sensor unit 121. It can be obtained. At this time, the second rotation angle of the camera 122 is the angle at which the camera 122 is rotated in the vertical direction with respect to the floor. And, the at least one processor 111 generates information about a third distance between the user's face and the center of the projection surface based on information about the first distance, information about the second distance, and the second rotation angle of the camera. It can be obtained. And, at least one processor 111 may calculate the vertical correction angle based on information about the first height, information about the second height, and information about the third distance.

Additionally, the at least one processor 111 rotates the first user image in a first direction based on the horizontal correction angle and rotates the first user image in the second direction based on the vertical correction angle to produce the user's frontal image. It can be obtained.

Additionally, at least one processor 111 may obtain a frontal image of the user by inputting the first user image, the horizontal correction angle, and the vertical correction angle into the learned neural network model.

Additionally, at least one processor 111 may generate an avatar image based on the user's frontal image and control the projection unit 112 to project a video call screen including the generated avatar image.

In addition, at least one processor 111 enables the electronic device 100 to make a video call based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor unit 121. It is possible to identify whether it is located in an available space. Additionally, when the electronic device 100 is not located in a space where video calls are available, at least one processor 111 may control the driver 120 to move to a space where video calls are available.

Additionally, at least one processor 111 may acquire a second user image using the camera 122 while performing a video call. Additionally, at least one processor 111 may calculate the rotation angle of the user's face based on the second user image. Additionally, at least one processor 111 may rotate the user's frontal image based on the calculated rotation angle of the user's face.

Additionally, when a plurality of users are included in the first user image, the at least one processor 111 may identify a speaker among the plurality of users and calculate a horizontal correction angle and a vertical correction angle based on the identified speaker. .

Meanwhile, the electronic device 100 may further include a driver 120, as shown in FIG. 2B. The driving unit 120 is a component for moving the electronic device 100 and may include a motor or wheels. Additionally, the electronic device 100 can adjust the projection angle of the projection unit 112 by rotating the body of the electronic device 100, and can rotate the camera 122 disposed on the body of the electronic device 100 by 360 degrees. You can adjust the rotation angle of the camera.

FIG. 3 is a diagram illustrating a method of correcting a user image captured by an electronic device while performing a video call, according to an embodiment of the present disclosure.

First, the electronic device 100 can execute a video call function (S310). Specifically, when a video call request is received from the outside or a user command to run a video call application is input, the electronic device 100 may execute the video call function.

The electronic device 100 may acquire a first user image by photographing the user through the camera 122 (S320). At this time, the first user image is an image captured when the user looks at the projection screen (the other party on the video call screen). As an example, the electronic device 100 may acquire a first user image by photographing the user when a message telling the user to look at the projection screen (e.g., please look straight ahead) is output before starting a video call. As an example, the electronic device 100 may detect a greeting message (for example, hello) from the other party in a video call, and acquire a first user image by filming the user when the greeting message is performed.

The electronic device 100 may acquire a frontal image of the user (S330). Specifically, the electronic device 100 adjusts the horizontal correction angle and vertical correction angle based on the first user image, the rotation angle of the camera 122, the projection angle of the projection unit 112, and the sensing value obtained through the sensor unit 121. The correction angle can be calculated. Additionally, the electronic device 100 may acquire the user's frontal image based on the first user image, the horizontal correction angle, and the vertical correction angle. This will be described in detail later with reference to FIGS. 4 to 8. The electronic device 100 may control the projection unit 112 to project a video call screen including the acquired frontal image of the user.

While performing a video call, the electronic device 100 may acquire a second user image (S340).

The electronic device 100 may analyze the second user image (S350). Specifically, the electronic device 100 analyzes the movement of the user's feature points (e.g., eyes, nose, mouth, etc.) included in the second user image to obtain information about the user's face rotation, lip movement, and eye movement. You can.

The electronic device 100 may correct the user's frontal image based on the analysis result (S360). Specifically, the electronic device 100 rotates the user's frontal image based on information about the user's face rotation and corrects the user's frontal image according to changes in the user's face based on information about lip movements or eye movements. can do.

As described above, through the image correction process, the electronic device 100 can make a video call with an external user through a frontal image even if the user is not photographed from the front.

Hereinafter, a method for correcting a user's video during a video call will be described in more detail with reference to FIGS. 4 to 8. Figure 4 is a block diagram illustrating a configuration for correcting a user's video while performing a video call, according to an embodiment of the present disclosure. As shown in FIG. 4, a distance information acquisition module 410, a height information acquisition module 420, a correction angle calculation module 430, a front image acquisition module 440, an image analysis module 450, and an image auxiliary module. It may include (460).

Meanwhile, the electronic device 100 may acquire the first user image. At this time, the first user image may be an image taken of the side of the user 10 when the user 10 is looking at the projection surface (particularly, the other party) 50. In particular, when capturing a first user image, the electronic device 100 may rotate the camera 122 so that the face of the user 10 is located at the center of the first user image.

The distance information acquisition module 410 may acquire distance information between the electronic device 100 and the object based on sensing values obtained from at least one distance sensor included in the sensor unit 121. Specifically, the distance information acquisition module 410 provides information about the first distance (a ₁ ) between the electronic device 100 and the center of the projection surface based on the first sensing value acquired through at least one distance sensor. It can be obtained. In addition, the distance information acquisition module 410 acquires information about the second distance (a ₂ ) between the electronic device 100 and the user's face based on the second sensing value acquired through at least one distance sensor. You can.

The height information acquisition module 420 may calculate the first height (h ₁ ) from the floor to the center of the projection surface and the second height (h ₃ ) from the floor to the user's face. Specifically, the height information acquisition module 420 may acquire the projection angle α of the projection unit 110 from the ground surface of the electronic device 100 through the rotation angle of the body of the electronic device 100. At this time, the projection angle (α) of the projection unit 110 is between the line segment (b1) having the shortest distance between the projection unit 110 and the projection surface 50 and the line segment between the projection unit 110 and the center of the projection surface. It may be the angle a1. Information about the first distance _a1 can be obtained using the second sensing value obtained through at least one distance sensor. And, the height information acquisition module 420 calculates the first height from the floor to the center of the projection surface 50 based on the information about the projection angle (α) and the first distance (a ₁ ) of the projection unit 110. h ₁ ) can be obtained. Referring to FIG. 5A, the height information acquisition module 420 can calculate c ₁ through Equation 1 below.

Meanwhile, the height (h ₂ ) from the floor to the camera 122 of the electronic device may be pre-stored.

Accordingly, the height information acquisition module 420 can calculate the first height (h ₁ ) from the floor to the center of the projection surface 50 using Equation 2 below.

In addition, the height information acquisition module 420 may acquire a second rotation angle (β) of the camera rotated upward with respect to the floor, based on the second sensing value obtained through at least one distance sensor. Information about the second distance (a ₂ ) can be obtained. And, _the height information acquisition module 420 calculates a second height ( h ₃ ) can be obtained. Referring to FIG. 5B, the height information acquisition module 420 can calculate c ₂ through Equation 3 below.

Meanwhile, the height (h ₂ ) from the floor to the camera 122 of the electronic device may be pre-stored.

Accordingly, the height information acquisition module 420 may calculate the first height (h ₁ ) from the floor to the face of the user 10 using Equation 4 below.

The correction angle calculation module 430 may acquire the horizontal correction angle and the vertical correction angle based on the distance information acquired by the distance information acquisition module 410 and the height information obtained by the height information acquisition module 420. .

Specifically, the correction angle calculation module 430 acquires the first distance (a ₁ ) and the second distance (a ₂ ) obtained by the distance information acquisition module 410 and the first rotation angle (γ) of the camera. You can. At this time, the first rotation angle γ of the camera may mean an angle at which the camera 122 is rotated based on the direction in which the camera 122 faces the center of the projection surface.

Referring to FIG. 6, the correction angle calculation device 430 can calculate the horizontal correction angle A using Equation 5 and Equation 6 below. At this time, _a3 may be the third distance between the face of the user 10 and the center of the projection surface 50.

In addition, the correction angle calculation module 430 calculates the vertical correction angle B using the first height (h1), the second height (h ₃ ), and the third distance (a3) obtained by the height information acquisition module 420. It can be calculated.

Referring to FIG. 7, the correction angle calculation device 430 can calculate the vertical correction angle B using Equation 7 and Equation 8 below.

The front image acquisition module 440 may acquire the user's front image using the first user image, the horizontal correction angle (A), and the vertical correction angle (B). At this time, the frontal image may be an image captured by a camera located at the center of the projection surface (or the area where the other person is displayed) when the user is looking at the center of the projection surface (or the other party).

Specifically, the front image acquisition module 440 rotates the first user image in the first direction based on the horizontal correction angle (A) and rotates the first user image in the second direction based on the vertical correction angle (B). You can obtain a frontal image of the user by rotating it. As an example, the front image acquisition module 440 may acquire a three-dimensional user image by performing 3D modeling on the first user image. Then, the front image acquisition module 440 rotates the 3D user image in the first direction based on the horizontal correction angle (A) and rotates the 3D user image in the second direction based on the vertical correction angle (B). You can obtain a 3D frontal image by rotating it. Additionally, the front image acquisition module 440 may acquire a two-dimensional front image of the user based on the three-dimensional front image. As another example, the front image acquisition module 440 rotates the first user image in the first direction based on the horizontal correction angle (A), and rotates the first user image in the second direction based on the vertical correction angle (B). You can obtain a frontal image of the user by rotating in this direction. At this time, the front image acquisition module 440 may generate a portion of the rotated front image that was not captured using a previously stored user image.

Additionally, the front image acquisition module 440 may rotate the first user image (or three-dimensional user image) based on the direction in which the electronic device 100 captured the user. In particular, the front image acquisition module 440 may rotate the first user image (or three-dimensional user image) in a direction opposite to the direction in which the electronic device 100 captured the user. For example, as shown on the left side of FIG. 8, when the electronic device 100 photographs the user from the lower right direction, the front image acquisition module 440 moves the first image to the left by the horizontal correction angle A. The user image 810 may be rotated, and the first user image 810 may be rotated downward by the vertical correction angle B. Accordingly, the front image acquisition module 440 can acquire the front image 820, as shown on the right side of FIG. 8.

Additionally, the front image acquisition module 440 may acquire the user's front image by inputting the first user image, the horizontal correction angle (A), and the vertical correction angle (B) into the learned neural network model. At this time, the neural network model may be an artificial intelligence model learned to output the user's frontal image by inputting the user image, horizontal correction angle, and vertical correction angle.

Additionally, the front image acquisition module 440 may determine whether additional correction is necessary by comparing the previously stored front image of the user with the acquired front image. For example, the frontal image acquisition module 440 operates when the feature point location (e.g., eyes, nose, mouth) of the pre-stored user's frontal image is different from the feature point location (e.g., eyes, nose, mouth) of the acquired frontal image. , the front image acquisition module 440 may determine that additional correction is necessary. In this case, the front image acquisition module 440 may re-acquire the user's front image by rephotographing the first user image.

Additionally, the front image acquisition module 440 can project a video call screen including the acquired front image onto the projection surface through the projection unit 112, and transmit the acquired front image to the terminal of the other party in the video call. At this time, the front image acquisition module 440 may change the front image acquired according to the user input into an avatar image. For example, as shown in FIG. 9, the front image acquisition module 440 changes the front image 820 into an avatar image 920 and sends a video call screen including the changed avatar image 920 to the projection unit. It is possible to project on a projection surface through (112) and transmit the obtained frontal image to the terminal of the other party in the video call.

The video analysis module 450 may analyze the second user video captured while making a video call. At this time, the image analysis module 450 may analyze the movement of feature points (e.g., eyes, nose, mouth) included in the second user image to obtain information about the rotation of the user's face or movement of lips, eyes, etc. there is. For example, when a plurality of feature points included in the second user image all move in one direction, the image analysis module 450 detects the rotation of the user's face and adjusts the rotation angle of the user's face according to the movement distance of the feature point. It can be calculated. Additionally, when only some feature points (eg, feature points related to the mouth) among the plurality of feature points move, the image analysis module 450 may obtain information about the user's lip movement.

The image correction module 460 may correct the user's frontal image based on the analysis result of the image analysis module 450. Specifically, the image correction module 460 may correct the user's frontal image based on information about the rotation of the user's face or movements of the lips and eyes, etc. acquired by the image analysis module 450. For example, if it is analyzed that the user has rotated his face 20 degrees to the left, the image correction module 460 may correct the user's frontal image to rotate it 20 degrees to the left. For example, if it is analyzed that the user's lip movement exists, the image correction module 460 may correct the user's frontal image to reflect the user's lip movement.

That is, the video correction module 460 can provide the user with a more natural video call experience by correcting the frontal image by reflecting the user's actual movements while making a video call.

Meanwhile, in the above-described embodiment, the electronic device 100 horizontally displays the first user image, the rotation angle of the camera 122, the projection angle of the projection unit 112, and the sensing value obtained through the sensor unit 121. Although it has been described as calculating the correction angle and the vertical correction angle and acquiring the user's frontal image based on the first user image, the horizontal correction angle, and the vertical correction angle, this is only an example, and the electronic device 100 After pre-registering the user's frontal image or printing a message guiding the user to look at the camera 122 of the electronic device 100 before starting a video call (or before accepting the video call), the user's frontal image is captured to capture the user's message. Frontal images can be provided.

In addition, the electronic device 100 determines whether the electronic device 100 is located in a space where a video call is possible based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. can be identified. Additionally, when the electronic device 100 is not located in a space where video calls are available, the electronic device 100 may control the driving unit to move to a space where video calls are available.

FIGS. 10A and 10B are diagrams for explaining a method of moving an electronic device by identifying whether it is located in a space where video calls can be made, according to an embodiment of the present disclosure.

Specifically, when the electronic device 100 performs a video call function, the electronic device 100 uses the first user image acquired at the current location, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. It is possible to identify whether the electronic device 100 is located in a space where video communication is possible based on at least one of the following. More specifically, whether the electronic device 100 can correct the first user image to a frontal image based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. It is possible to determine whether the first area is currently a space where video calls are possible. In particular, the user's face included in the first user image is less than a critical ratio (e.g., 40%) compared to all user faces, the rotation angle of the camera or the projection angle of the projection unit is outside the critical range, or the electronic If the distance between the device 100 and the projection surface 50 or the distance between the electronic device 100 and the user 10 is greater than or equal to the threshold, the electronic device 100 is currently located in a space where the electronic device 100 can make video calls. It can be judged that it was not done.

Additionally, when the electronic device 100 is not located in a space where video calls are available, the electronic device 100 may control the driver 120 to move to a space where video calls are available. At this time, the electronic device 100 may move to a space where a video call is possible based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor. For example, if the user's face included in the first user image is in the left rear area, the electronic device 100 may control the driver 120 to move to the right front. Additionally, the electronic device 100 may control the driver 120 so that the rotation angle or projection angle of the camera is within a critical range. Additionally, the electronic device 100 may control the driver 120 so that the projection surface 50 and the user are located within a critical distance from the electronic device 100.

For example, as shown in FIG. 10A, when it is determined that the video call is outside the space 1000, the electronic device 100 displays the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the information obtained through the sensor. The space 1000 where video calls can be made can be determined based on at least one of the sensing values. And, as shown in FIG. 10B, the electronic device 100 may control the driver 120 so that the electronic device 100 is located in the space 1000 where video communication is possible.

Meanwhile, in the above-described embodiment, it has been described that the first user image captured by the camera 122 includes one user, but this is only an example, and a plurality of users may be included in the first user image. .

In particular, when a plurality of users are included in a user image, the electronic device 100 can identify the speaker who is speaking among the plurality of users. Additionally, the electronic device 100 may calculate the horizontal correction angle and vertical correction angle based on the identified speaker. Specifically, the electronic device 100 may identify a speaker based on lip movements among the faces of a plurality of users included in the first image. Additionally, the electronic device 100 may rotate the camera 122 so that the identified speaker is located in the central area of the image. The electronic device 100 may acquire a first user image based on the speaker, and the first distance (a ₁ ), the second distance (a ₂ ), the rotation angle of the camera 122, and the projection unit based on the speaker. The horizontal correction angle and the vertical correction angle can be calculated by calculating the projection angle of 112 and the first to third heights (h ₁ to h ₃ ).

FIG. 11 is a flowchart illustrating a method of controlling an electronic device that provides a video call function, according to an embodiment of the present disclosure.

First, the electronic device 100 executes the video call function (S1110).

The electronic device 100 acquires a first user image by photographing the user through the camera 122 (S1120). At this time, the first user image may be an image taken of the side of the user 10 when the user 10 is looking at the projection surface (particularly, the other party) 50.

The electronic device 100 performs horizontal correction angle and vertical correction based on the first user image, the rotation angle of the camera 122, the projection angle of the projection unit 112, and the sensing value obtained through the sensor of the electronic device 100. Calculate the angle (S1130).

Specifically, the electronic device 100 may obtain information about the first distance between the electronic device 100 and the center of the projection surface based on the first sensing value obtained through at least one distance sensor. Additionally, the electronic device 100 may obtain information about the second distance between the electronic device 100 and the user's face based on the second sensing value obtained through at least one distance sensor. Additionally, the electronic device 100 may calculate the horizontal correction angle based on information about the first distance, information about the second distance, and the first rotation angle of the camera 112.

In addition, the electronic device 100 may acquire information about the first height from the floor to the center of the projection surface based on the projection angle of the projection unit 112 and the second sensing value obtained through at least one distance sensor. You can. In addition, the electronic device 100 may obtain information about the second height from the floor to the user's face based on the second rotation angle of the camera 122 and the second sensing value obtained through at least one sensor. there is. And, the electronic device 100 provides information about the third distance between the user's face and the center of the projection surface based on the information about the first distance, the information about the second distance, and the second rotation angle of the camera 122. can be obtained. Additionally, the electronic device 100 may calculate the vertical correction angle based on information about the first height, information about the second height, and information about the third distance.

Then, the electronic device 100 acquires the user's front image based on the first user image, the horizontal correction angle, and the vertical correction angle (S1140). Specifically, the electronic device 100 rotates the first user image in the first direction based on the horizontal correction angle and rotates the first user image in the second direction based on the vertical correction angle to display the user's frontal image. It can be obtained. Alternatively, the electronic device 100 may acquire the user's frontal image by inputting the first user image, the horizontal correction angle, and the vertical correction angle into the learned neural network model.

Then, the electronic device 100 projects a video call screen including the acquired frontal image of the user through the projection unit 112 (S1150). At this time, the electronic device 100 may generate an avatar image based on the user's frontal image and may project a video call screen including the generated avatar image.

Additionally, the electronic device 100 may acquire a second user image using the camera 122 while performing a video call. Additionally, the electronic device 100 may calculate the rotation angle of the user's face based on the second user image. Additionally, the electronic device 100 may rotate the user's frontal image based on the calculated rotation angle of the user's face.

In addition, the electronic device 100 is based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit 112, and the sensing value obtained through the sensor unit 121. You can identify whether you are located in a space where video calls are possible. Additionally, when the electronic device 100 is not located in a space where video calls are available, the electronic device 100 may control the driver 120 to move to a space where video calls are available.

Additionally, when a plurality of users are included in the first user image, the electronic device 100 may identify the speaker among the plurality of users. Additionally, the electronic device 100 may calculate the horizontal correction angle and vertical correction angle based on the identified speaker.

FIG. 12 is a block diagram specifically illustrating the configuration of an electronic device according to an embodiment of the present disclosure. Referring to FIG. 12, the electronic device 100 includes a processor 111, a projection unit 112, a memory 113, a communication interface 114, an operation interface 115, an input/output interface 116, and a speaker 117. , it may include at least one of a microphone 118, a power supply unit 119, a driver unit 120, or a sensor unit 121. Meanwhile, the configuration shown in FIG. 12 is merely an example of various embodiments, and some configurations may be omitted and new configurations may be added. Meanwhile, the content already described in FIG. 2 will be omitted.

The processor 111 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, it is not limited to this, and is not limited to the central processing unit ( central processing unit (CPU), micro controller unit (MCU), micro processing unit (MPU), controller, application processor (AP), graphics-processing unit (GPU), or communication processor (CP)), ARM (advanced reduced instruction set computer (RISC) machines) processor, or may be defined by the corresponding term. In addition, the processor 111 is a SoC (System) with a built-in processing algorithm. on Chip), may be implemented in the form of LSI (large scale integration), or may be implemented in the form of FPGA (Field Programmable Gate Array). In addition, the processor 111 stores computer executable instructions stored in the memory 113. You can perform various functions by executing instructions.

The projection unit 112 is a component that projects an image to the outside. According to various embodiments of the present disclosure, the projection unit 112 uses various projection methods (for example, a cathode-ray tube (CRT) method, a liquid crystal display (LCD) method, a digital light processing (DLP) method, and a laser method. etc.) can be implemented. For example, the CRT method is basically the same as the CRT monitor. The CRT method magnifies the image with a lens in front of the cathode ray tube (CRT) and displays the image on the screen. Depending on the number of cathode ray tubes, it is divided into a one-tube type and a three-tube type. In the case of the three-tube type, the red, green, and blue cathode ray tubes can be implemented separately.

As another example, the LCD method displays images by transmitting light from a light source through liquid crystal. The LCD method is divided into a single-panel type and a three-panel type. In the case of the three-panel type, the light from the light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only light of a specific color and passes the rest) and then passes through the liquid crystal. Afterwards, the light can gather in one place again.

As another example, the DLP method is a method of displaying images using a DMD (Digital Micromirror Device) chip. The DLP projection unit may include a light source, color wheel, DMD chip, projection lens, etc. Light output from a light source can acquire color as it passes through a rotating color wheel. The light that passes through the color wheel is input to the DMD chip. The DMD chip contains numerous micro-mirrors and reflects the light input to the DMD chip. The projection lens can play the role of enlarging the light reflected from the DMD chip to the image size.

As another example, laser methods include DPSS (Diode Pumped Solid State) lasers and galvanometers. A laser that outputs various colors uses a laser whose optical axes are overlapped using a special mirror after installing three DPSS lasers for each RGB color. Galvanometers contain mirrors and high-power motors that move the mirrors at high speeds. For example, a galvanometer can rotate a mirror at up to 40 KHz/sec. The galvanometer is mounted according to the scanning direction. Since projectors generally scan flat, the galvanometer can also be arranged divided into x and y axes.

Meanwhile, the projection unit 112 may include various types of light sources. For example, the projection unit 112 may include at least one light source among a lamp, LED, and laser.

The projection unit 112 can output images in 4:3 screen ratio, 5:4 screen ratio, and 16:9 wide screen ratio depending on the purpose of the electronic device 100 or user settings, and can output images in WVGA (854*480) depending on the screen ratio. ), SVGA(800*600), ), images can be output at various resolutions, such as

Meanwhile, the projection unit 112 can perform various functions to adjust the output image under the control of the processor 111. For example, the projection unit 112 can perform functions such as zoom, keystone, quick corner (4 corners) keystone, and lens shift.

Specifically, the projection unit 112 can enlarge or reduce the image depending on the distance from the screen (projection distance). That is, the zoom function can be performed depending on the distance from the screen. At this time, the zoom function may include a hardware method that adjusts the screen size by moving the lens and a software method that adjusts the screen size by cropping the image, etc. Meanwhile, when the zoom function is performed, the focus of the image needs to be adjusted. For example, methods for controlling focus include manual focus methods, electric methods, etc. The manual focus method refers to a method of focusing manually, and the electric method refers to a method in which the projector automatically focuses using a built-in motor when the zoom function is performed. When performing a zoom function, the projection unit 112 can provide a digital zoom function through software and an optical zoom function that performs the zoom function by moving the lens through the driving unit 120.

Additionally, the projection unit 112 may perform a keystone correction function. If the height is not appropriate for front projection, the screen may be distorted upward or downward. The keystone correction function refers to the function to correct a distorted screen. For example, if distortion occurs in the left and right directions of the screen, it can be corrected using horizontal keystone, and if distortion occurs in the vertical direction, it can be corrected using vertical keystone. The quick corner (4 corners) keystone correction function is a function that corrects the screen when the center area of the screen is normal but the corner areas are unbalanced. The lens shift function is a function that moves the screen as is when the screen is off-screen.

Meanwhile, the projection unit 112 can automatically analyze the surrounding environment and projection environment without user input and provide zoom/keystone/focus functions. Specifically, the projection unit 112 displays the distance between the electronic device 100 and the screen detected through a sensor (depth camera, distance sensor, infrared sensor, illuminance sensor, etc.) and the space where the electronic device 100 is currently located. Zoom/keystone/focus functions can be automatically provided based on information about the camera, surrounding light amount, etc.

Additionally, the projection unit 112 may provide a lighting function using a light source. In particular, the projection unit 112 can provide a lighting function by outputting a light source using LED. According to various embodiments, the projection unit 112 may include one LED, and according to other embodiments, the electronic device 100 may include a plurality of LEDs. Meanwhile, the projection unit 112 may output a light source using a surface-emitting LED depending on the implementation example. Here, a surface-emitting LED may refer to an LED having a structure in which an optical sheet is disposed on the upper side of the LED so that the light source is output evenly distributed. Specifically, when a light source is output through an LED, the light source can be evenly distributed through the optical sheet, and the light source dispersed through the optical sheet can be incident on the display panel.

Meanwhile, the projection unit 112 may provide the user with a dimming function to adjust the intensity of the light source. Specifically, when a user input for adjusting the intensity of the light source is received from the user through the operation interface 115 (e.g., a touch display button or dial), the projection unit 112 displays the light source corresponding to the received user input. The LED can be controlled to output an intensity of

Additionally, the projection unit 112 may provide a dimming function based on content analyzed by the processor 111 without user input. Specifically, the projection unit 112 may control the LED to output the intensity of the light source based on information about the currently provided content (eg, content type, content brightness, etc.).

Meanwhile, the projection unit 112 can control the color temperature under the control of the processor 111. Here, the processor 111 can control the color temperature based on content. Specifically, when content is identified to be output, the processor 111 may obtain color information for each frame of the content for which output has been determined. Additionally, the processor 111 may control the color temperature based on the obtained color information for each frame. Here, the processor 111 may obtain at least one main color of the frame based on color information for each frame. Additionally, the processor 111 may adjust the color temperature based on at least one acquired main color. For example, the color temperature that the processor 111 can adjust may be divided into warm type or cold type. Here, it is assumed that the frame to be output (hereinafter referred to as output frame) includes a scene where a fire occurs. The processor 111 may identify (or obtain) that the main color is red based on color information included in the current output frame. Additionally, the processor 111 can identify the color temperature corresponding to the identified main color (red). Here, the color temperature corresponding to red may be warm type. Meanwhile, the processor 111 may use an artificial intelligence model to obtain color information or the main color of the frame. According to various embodiments, the artificial intelligence model may be stored in the electronic device 100 (eg, memory 113). According to another embodiment, the artificial intelligence model may be stored in an external server capable of communicating with the electronic device 100.

The memory 113 is implemented as internal memory such as ROM (e.g., electrically erasable programmable read-only memory (EEPROM)) and RAM included in the processor 111, or is implemented by the processor 111 and the It may also be implemented as a separate memory. In this case, the memory 113 may be implemented as a memory embedded in the electronic device 100 or as a memory detachable from the electronic device 100 depending on the data storage purpose. For example, in the case of data for driving the electronic device 100, it is stored in the memory embedded in the electronic device 100, and in the case of data for the expansion function of the electronic device 100, it is detachable from the electronic device 100. It can be stored in available memory.

Meanwhile, in the case of memory embedded in the electronic device 100, volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory ( Examples: one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g. NAND flash or NOR flash, etc.) ), a hard drive, or a solid state drive (SSD), and in the case of a memory that is removable from the electronic device 100, a memory card (e.g., compact flash (CF), SD ( secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g. It can be implemented in a form such as USB memory).

The memory 113 may store at least one command related to the electronic device 100. Additionally, an operating system (O/S) for driving the electronic device 100 may be stored in the memory 113. Additionally, the memory 113 may store various software programs or applications for operating the electronic device 100 according to various embodiments of the present disclosure. Additionally, the memory 113 may include a semiconductor memory such as flash memory or a magnetic storage medium such as a hard disk.

Specifically, the memory 113 may store various software modules for operating the electronic device 100 according to various embodiments of the present disclosure, and the processor 111 executes various software modules stored in the memory 113. Thus, the operation of the electronic device 100 can be controlled. That is, the memory 113 is accessed by the processor 111, and reading/writing/modifying/deleting/updating data by the processor 111 can be performed.

Meanwhile, in the present disclosure, the term memory 113 refers to a storage unit, a ROM (not shown) within the processor 111, a RAM (not shown), or a memory card (not shown) mounted on the electronic device 100 (e.g. , micro SD card, memory stick).

The communication interface 114 is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface 114 may include a wireless communication module or a wired communication module. Here, each communication module may be implemented in the form of at least one hardware chip.

The wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules.

The Wi-Fi module and Bluetooth module can communicate using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information such as SSID (service set identifier) and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this.

The infrared communication module performs communication based on infrared communication (IrDA, infrared data association) technology, which transmits data wirelessly over a short distance using infrared rays that lie between visible light and millimeter waves.

In addition to the communication methods described above, other communication modules include zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G. It may include at least one communication chip that performs communication according to various wireless communication standards such as (5th Generation).

The wired communication module may be a module that communicates with an external device by wire. For example, the wired communication module may include at least one of a local area network (LAN) module, an Ethernet module, a pair cable, a coaxial cable, an optical fiber cable, or an ultra wide-band (UWB) module.

The manipulation interface 115 may include various types of input devices. For example, the operating interface 115 may include physical buttons. At this time, the physical button may include a function key, a direction key (for example, a 4-way key), or a dial button. According to various embodiments, a physical button may be implemented as a plurality of keys. According to another embodiment, the physical button may be implemented as one key. Here, when the physical button is implemented as one key, the electronic device 100 may receive a user input in which one key is pressed for more than a threshold time. When a user input in which one key is pressed for more than a threshold time is received, the processor 111 may perform a function corresponding to the user input. For example, the processor 111 may provide a lighting function based on user input.

Additionally, the manipulation interface 115 may receive user input using a non-contact method. When receiving user input through a contact method, physical force must be transmitted to the electronic device 100. Accordingly, a method for controlling the electronic device 100 regardless of physical force may be needed. Specifically, the manipulation interface 115 may receive a user gesture and perform an operation corresponding to the received user gesture. Here, the manipulation interface 115 may receive the user's gesture through a sensor (eg, an image sensor or an infrared sensor).

Additionally, the manipulation interface 115 may receive user input using a touch method. For example, the manipulation interface 115 may receive user input through a touch sensor. According to various embodiments, the touch method may be implemented as a non-contact method. For example, the touch sensor can determine whether the user's body approaches within a threshold distance. Here, the touch sensor can identify user input even when the user does not contact the touch sensor. Meanwhile, according to another implementation example, the touch sensor may identify a user input in which the user touches the touch sensor.

Meanwhile, the electronic device 100 may receive user input in various ways other than the operation interface 115 described above. In various embodiments, the electronic device 100 may receive user input through an external remote control device. Here, the external remote control device is a remote control device corresponding to the electronic device 100 (e.g., a dedicated control device for the electronic device 100) or a user's portable communication device (e.g., a smartphone or wearable device). It can be. Here, an application for controlling the electronic device 100 may be stored in the user's portable communication device. The portable communication device may obtain user input through a stored application and transmit the obtained user input to the electronic device 100. The electronic device 100 may receive user input from a portable communication device and perform an operation corresponding to the user's control command.

Meanwhile, the electronic device 100 may receive user input using voice recognition. According to various embodiments, the electronic device 100 may receive a user's voice through a microphone included in the electronic device 100. According to another embodiment, the electronic device 100 may receive a user's voice from a microphone or an external device. Specifically, the external device can acquire the user's voice through the external device's microphone and transmit the acquired user's voice to the electronic device 100. The user's voice transmitted from an external device may be audio data or digital data converted from audio data (for example, audio data converted to the frequency domain, etc.). Here, the electronic device 100 may perform an operation corresponding to the received user voice. Specifically, the electronic device 100 may receive audio data corresponding to the user's voice through a microphone. Also, the electronic device 100 can convert the received audio data into digital data. Additionally, the electronic device 100 can convert the converted digital data into text data using the STT (Speech To Text) function. According to various embodiments, the Speech To Text (STT) function may be performed directly on the electronic device 100,

According to another embodiment, the Speech To Text (STT) function may be performed on an external server. The electronic device 100 may transmit digital data to an external server. The external server can convert digital data into text data and obtain control command data based on the converted text data. The external server may transmit control command data (here, text data may also be included) to the electronic device 100. The electronic device 100 may perform an operation corresponding to the user's voice based on the acquired control command data.

Meanwhile, the electronic device 100 may provide a voice recognition function using a single assistant (or an artificial intelligence assistant, for example, BixbyTM, etc.), but this is only a variety of embodiments and can provide a voice recognition function through a plurality of assistants. Voice recognition function can be provided. At this time, the electronic device 100 may provide a voice recognition function by selecting one of a plurality of assistants based on a trigger word corresponding to the assistance or a specific key present on the remote control.

Meanwhile, the electronic device 100 may receive user input using screen interaction. Screen interaction may refer to a function of identifying whether a predetermined event occurs through an image projected by the electronic device 100 on a screen (or projection surface) and obtaining user input based on the predetermined event. Here, the predetermined event may mean an event in which a predetermined object is identified at a specific location (for example, a location where a UI for receiving user input is projected). Here, the predetermined object may include at least one of a user's body part (eg, a finger), a pointing stick, or a laser point. When a predetermined object is identified at a location corresponding to the projected UI, the electronic device 100 may identify that a user input for selecting the projected UI has been received. For example, the electronic device 100 may project a guide image to display a UI on the screen. And, the electronic device 100 can identify whether the user selects the projected UI. Specifically, if a predetermined event is identified at the location of the projected UI, the electronic device 100 may identify that the user has selected the projected UI. Here, the projected UI may include at least one item. Here, the electronic device 100 may perform spatial analysis to identify whether a predetermined event is located at the location of the projected UI. Here, the electronic device 100 may perform spatial analysis through sensors (eg, image sensors, infrared sensors, depth cameras, distance sensors, etc.). The electronic device 100 may identify whether a predetermined event occurs at a specific location (a location where the UI is projected) by performing spatial analysis. Additionally, if it is identified that a predetermined event occurs at a specific location (a location where the UI is projected), the electronic device 100 may identify that a user input for selecting the UI corresponding to the specific location has been received.

The input/output interface 116 is configured to input and output at least one of an audio signal and an image signal. The input/output interface 116 can receive at least one of audio and image signals from an external device and output control commands to the external device.

Depending on the implementation, the input/output interface 116 may be implemented as an interface that inputs and outputs only audio signals and an interface that inputs and outputs only image signals, or as a single interface that inputs and outputs both audio signals and image signals.

Meanwhile, in various embodiments of the present disclosure, the input/output interface 116 includes High Definition Multimedia Interface (HDMI), Mobile High-Definition Link (MHL), Universal Serial Bus (USB), USB C-type, Display Port (DP), It can be implemented with at least one wired input/output interface among Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (Dsubminiature), and DVI (Digital Visual Interface). According to various embodiments, the wired input/output interface may be implemented as an interface that inputs and outputs only audio signals and an interface that inputs and outputs only image signals, or may be implemented as a single interface that inputs and outputs both audio signals and image signals.

Additionally, the electronic device 100 may receive data through a wired input/output interface, but this is only a variety of embodiments, and power may be supplied through a wired input/output interface. For example, the electronic device 100 can receive power from an external battery through USB C-type or from an outlet through a power adapter. As another example, the electronic device 100 may receive power from an external device (eg, a laptop or monitor, etc.) through DP.

Meanwhile, audio signals can be input through a wired input/output interface, and image signals can be input through a wireless input/output interface (or communication interface). Alternatively, the audio signal may be input through a wireless input/output interface (or communication interface), and the image signal may be input through a wired input/output interface.

The speaker 117 is configured to output an audio signal. In particular, the speaker 117 may include an audio output mixer, an audio signal processor, and a sound output module. The audio output mixer can synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may synthesize an analog audio signal and another analog audio signal (eg, an analog audio signal received from an external source) into at least one analog audio signal. The sound output module may include a speaker or an output terminal. According to various embodiments, the sound output module may include a plurality of speakers. In this case, the sound output module may be disposed inside the main body, and the sound emitted by covering at least a portion of the diaphragm of the sound output module may be transmitted through a sound conduit ( It can be transmitted outside the body by passing through a waveguide. The sound output module includes a plurality of sound output units, and the plurality of sound output units are arranged symmetrically on the exterior of the main body, so that sound can be radiated in all directions, that is, in all directions of 360 degrees.

The microphone 118 is configured to receive a user's voice or other sounds and convert them into audio data. The microphone 118 can receive the user's voice when activated. For example, the microphone 118 may be formed integrally with the electronic device 100, such as on the top, front, or side surfaces. The microphone 118 includes a microphone that collects user voice in analog form, an amplifier circuit that amplifies the collected user voice, an A/D conversion circuit that samples the amplified user voice and converts it into a digital signal, and noise components from the converted digital signal. It may include various configurations such as a filter circuit to remove .

The power supply unit 119 may receive power from the outside and supply power to various components of the electronic device 100. The power supply unit 119 according to various embodiments of the present disclosure may receive power through various methods. In various embodiments, the power supply unit 119 may receive power using the connector 130 as shown in FIG. 1. Additionally, the power supply unit 119 can receive power using a 220V DC power cord. However, the electronic device 100 is not limited to this, and may receive power using a USB power cord or a wireless charging method.

Additionally, the power supply unit 119 may receive power using an internal battery or an external battery. The power supply unit 119 according to various embodiments of the present disclosure may receive power through an internal battery. As an example, the power unit 119 can charge the power of the internal battery using at least one of a 220V DC power cord, a USB power cord, and a USB C-Type power cord, and receive power through the charged internal battery. . Additionally, the power supply unit 119 according to various embodiments of the present disclosure may receive power through an external battery. For example, when the electronic device 100 and an external battery are connected through various wired communication methods such as a USB power cord, USB C-Type power cord, and socket groove, the power unit 119 receives power through the external battery. You can. That is, the power supply unit 119 can receive power directly from an external battery, or charge the internal battery through an external battery and receive power from the charged internal battery.

The power supply unit 119 according to the present disclosure may receive power using at least one of the plurality of power supply methods described above.

Meanwhile, with regard to power consumption, the electronic device 100 may have power consumption below a preset value (eg, 43W) due to socket type and other standards. At this time, the electronic device 100 may vary power consumption to reduce power consumption when using the battery. That is, the electronic device 100 can vary power consumption based on the power supply method and power usage amount.

The driver 120 may drive at least one hardware component included in the electronic device 100. The driving unit 120 may generate physical force and transmit it to at least one hardware component included in the electronic device 100.

Here, the driver 120 is driven to move the hardware component included in the electronic device 100 (e.g., move the electronic device 100) or rotate the component (e.g., rotate the projection lens). It can generate power.

The driving unit 120 can adjust the projection direction (or projection angle) of the projection unit 122. Additionally, the driver 120 can move the position of the electronic device 100. Here, the driver 120 may control the moving member 109 to move the electronic device 100. For example, the driving unit 120 may control the moving member 109 using a motor and wheels.

The sensor unit 121 may include at least one sensor. Specifically, the sensor unit 121 may include at least one of a tilt sensor that senses the tilt of the electronic device 100 and an image sensor that captures an image. Here, the tilt sensor may be an acceleration sensor or a gyro sensor, and the image sensor may be a camera or a depth camera. Meanwhile, the tilt sensor can be described as a motion sensor. Additionally, the sensor unit 121 may include various sensors in addition to a tilt sensor or an image sensor. For example, the sensor unit 121 may include an illumination sensor and a distance sensor. The distance sensor may be Time of Flight (ToF). Additionally, the sensor unit 121 may include a LiDAR sensor.

Meanwhile, the electronic device 100 can control lighting functions in conjunction with external devices. Specifically, the electronic device 100 may receive lighting information from an external device. Here, the lighting information may include at least one of brightness information or color temperature information set in an external device. Here, the external device is a device connected to the same network as the electronic device 100 (for example, an IoT device included in the same home/work network) or a device that is not in the same network as the electronic device 100 but communicates with the electronic device 100. It may refer to a capable device (for example, a remote control server). For example, assume that an external lighting device (IoT device) included in the same network as the electronic device 100 is outputting red light at a brightness of 50. An external lighting device (IoT device) may directly or indirectly transmit lighting information (for example, information indicating that red lighting is output at a brightness of 50) to the electronic device 100. Here, the electronic device 100 may control the output of the light source based on lighting information received from an external lighting device. For example, if lighting information received from an external lighting device includes information about outputting red lighting at a brightness of 50, the electronic device 100 may output red lighting at a brightness of 50.

Meanwhile, the electronic device 100 can control the lighting function based on biometric information. Specifically, the processor 111 may obtain the user's biometric information. Here, the biometric information may include at least one of the user's body temperature, heart rate, blood pressure, respiration, and electrocardiogram. Here, biometric information may include various information in addition to the information described above. As an example, the electronic device 100 may include a sensor for measuring biometric information. The processor 111 may acquire the user's biometric information through a sensor and control the output of the light source based on the acquired biometric information. As another example, the processor 111 may receive biometric information from an external device through the input/output interface 116. Here, the external device may refer to the user's portable communication device (eg, a smartphone or wearable device). The processor 111 may obtain the user's biometric information from an external device and control the output of the light source based on the obtained biometric information. Meanwhile, depending on the implementation example, the electronic device 100 may identify whether the user is sleeping, and if the user is identified as sleeping (or preparing to sleep), the processor 111 may identify the user based on the user's biometric information. This allows you to control the output of the light source.

Meanwhile, the electronic device 100 according to various embodiments of the present disclosure may provide various smart functions.

Specifically, the electronic device 100 is connected to a portable terminal device for controlling the electronic device 100, and a screen output from the electronic device 100 can be controlled through user input input from the portable terminal device. As an example, the mobile terminal device may be implemented as a smartphone including a touch display, and the electronic device 100 receives and outputs screen data provided by the mobile terminal device and inputs the screen data from the mobile terminal device. The screen output from the electronic device 100 may be controlled according to user input.

The electronic device 100 can share content or music provided by the mobile terminal device by connecting to the mobile terminal device through various communication methods such as Miracast, Airplay, wireless DEX, and Remote PC method.

Additionally, the mobile terminal device and the electronic device 100 may be connected using various connection methods. In various embodiments, the mobile terminal device may search for the electronic device 100 to perform a wireless connection, or the electronic device 100 may search for the mobile terminal device to perform a wireless connection. Additionally, the electronic device 100 can output content provided by the mobile terminal device.

In various embodiments, when a mobile terminal device is positioned near the electronic device 100 while specific content or music is being output on the mobile terminal device, a preset gesture is detected through the display of the mobile terminal device (e.g., motion tap view). , the electronic device 100 can output content or music being output on the portable terminal device.

In various embodiments, while specific content or music is being output on the mobile terminal device, the mobile terminal device approaches the electronic device 100 by a preset distance or less (e.g., non-contact tap view) or the mobile terminal device contacts the electronic device 100. When touched twice at a short interval (eg, contact tab view), the electronic device 100 can output content or music being output on the mobile terminal device.

In the above-described embodiment, it has been described that the same screen as the screen provided in the mobile terminal device is provided in the electronic device 100, but the present disclosure is not limited to this. That is, when a connection is established between the portable terminal device and the electronic device 100, the first screen provided by the portable terminal device is output, and the electronic device 100 displays a first screen provided by a different portable terminal device from the first screen. A second screen may be output. As an example, the first screen may be a screen provided by a first application installed on the mobile terminal device, and the second screen may be a screen provided by a second application installed on the mobile terminal device. For example, the first screen and the second screen may be different screens provided by an application installed on the mobile terminal device. Additionally, as an example, the first screen may be a screen that includes a remote control-type UI for controlling the second screen.

The electronic device 100 according to the present disclosure can output a standby screen. For example, when the electronic device 100 is not connected to an external device or when no input is received from the external device for a preset time, the electronic device 100 may output a standby screen. The conditions for the electronic device 100 to output the standby screen are not limited to the above-described example, and the standby screen may be output under various conditions.

The electronic device 100 may output a standby screen in the form of a blue screen, but the present disclosure is not limited to this. As an example, the electronic device 100 may obtain an atypical object by extracting only the shape of a specific object from data received from an external device, and output a standby screen including the obtained atypical object.

Meanwhile, the electronic device 100 may further include a display (not shown).

The display (not shown) may be implemented as various types of displays such as LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, PDP (Plasma Display Panel), etc. The display (not shown) may also include a driving circuit and a backlight unit that can be implemented in the form of a-si TFT (amorphous silicon thin film transistor), LTPS (low temperature poly silicon) TFT, OTFT (organic TFT), etc. there is. Meanwhile, the display (not shown) may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display, etc. Additionally, according to various embodiments of the present disclosure, a display (not shown) may include a bezel housing the display panel as well as a display panel that outputs an image. In particular, according to various embodiments of the present disclosure, the bezel may include a touch sensor (not shown) to detect user interaction.

Meanwhile, the electronic device 100 may further include a shutter unit (not shown).

The shutter unit (not shown) may include at least one of a shutter, a fixing member, a rail, or a body.

Here, the shutter may block the light output from the projection unit 112. Here, the fixing member can fix the position of the shutter. Here, the rail may be a path for moving the shutter and the fixing member. Here, the body may be configured to include a shutter and a fixing member.

Figure 13 is a perspective view showing the appearance of an electronic device according to an embodiment of the present disclosure. Referring to the embodiment 1310 of FIG. 13 , the electronic device 100 may include a moving member 109. The moving member 109 may refer to a member for moving from a first position to a second position in the space where the electronic device 100 is placed. The electronic device 100 may use the force generated by the driving unit 120 to control the moving member 109 so that the electronic device 100 moves. At this time, the moving member 109 may include a motor or wheels.

The embodiment 1320 of FIG. 13 is a view of the electronic device 100 of the embodiment 1310 viewed from another direction.

Meanwhile, methods according to various embodiments of the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Methods according to various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media that can be read by a machine (e.g., a computer). The device stores information stored from the storage medium. A device capable of calling a command and operating according to the called command may include an electronic device (eg, a TV) according to the disclosed embodiments.

Meanwhile, a storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is semi-permanently stored in a storage medium and temporary storage media. It does not distinguish between cases where it is stored as . For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

When the instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or using other components under the control of the processor. Instructions may contain code generated or executed by a compiler or interpreter.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field to which the disclosure pertains without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

111: at least one processor 112: projection unit
113: memory 121: sensor unit
122: Camera

Claims (18)

In an electronic device that provides a video call function,
projection unit;
camera;
A sensor for obtaining information about the distance to an external object;
a memory storing at least one instruction; and
At least one processor connected to the projection unit, the camera, the sensor, and the memory, and configured to control the electronic device,
The at least one processor executes the at least one instruction,
When the video call function is executed, the first user video is acquired by filming the user through the camera,
Calculating a horizontal correction angle and a vertical correction angle based on the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor,
Obtaining a frontal image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle,
An electronic device that controls the projection unit to project a video call screen including the acquired frontal image of the user. According to paragraph 1,
The at least one processor,
Obtaining information about a first distance between the electronic device and the center of the projection surface based on the first sensing value obtained through the sensor,
Obtaining information about a second distance between the electronic device and the user's face based on the second sensing value obtained through the sensor,
An electronic device that calculates the horizontal correction angle based on the information about the first distance, the information about the second distance, and the first rotation angle of the camera. According to paragraph 2,
The at least one processor,
Obtaining information about a first height from the floor to the center of the projection surface based on the projection angle of the projection unit and the second sensing value obtained through the sensor,
Obtaining information about a second height from the floor to the user's face based on the second rotation angle of the camera and the second sensing value obtained through the sensor,
Obtain information about a third distance between the user's face and the center of the projection surface based on the information about the first distance, the information about the second distance, and the second rotation angle of the camera,
An electronic device that calculates the vertical correction angle based on the information about the first height, the information about the second height, and the information about the third distance. According to paragraph 1,
The at least one processor,
An electronic device that rotates the first user image in a first direction based on the horizontal correction angle and rotates the first user image in a second direction based on the vertical correction angle to obtain a frontal image of the user. According to paragraph 1,
The at least one processor,
An electronic device that acquires a frontal image of the user by inputting the first user image, the horizontal correction angle, and the vertical correction angle into a learned neural network model. According to paragraph 1,
The at least one processor,
Generating an avatar image based on the user's frontal image,
An electronic device that controls the projection unit to project a video call screen including the generated avatar image. According to paragraph 1,
It further includes a driving unit for moving the electronic device,
The at least one processor,
Identifying whether the electronic device is located in a space where video communication is possible based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor,
An electronic device that controls the driving unit to move to a space where video calls are available when the electronic device is not located in a space where video calls are available. According to paragraph 1,
The at least one processor,
Obtaining a second user video using the camera while performing the video call,
Calculating a rotation angle of the user's face based on the second user image,
An electronic device that rotates a frontal image of the user based on the calculated rotation angle of the user's face. According to paragraph 1,
The at least one processor,
When a plurality of users are included in the first user image, a speaker among the plurality of users is identified,
An electronic device that calculates a horizontal correction angle and a vertical correction angle based on the identified speaker. In the control method of an electronic device that includes a projection unit that projects an image and provides a video call function,
When the video call function is executed, acquiring a first user image by photographing the user through a camera;
calculating a horizontal correction angle and a vertical correction angle based on the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor of the electronic device;
Obtaining a frontal image of the user based on the first user image, the horizontal correction angle, and the vertical correction angle;
A control method comprising: projecting a video call screen including the acquired frontal image of the user through the projection unit. According to clause 10,
The calculating step is,
Obtaining information about a first distance between the electronic device and the center of the projection surface based on the first sensing value obtained through the sensor;
Obtaining information about a second distance between the electronic device and the user's face based on a second sensing value obtained through the sensor; and
A control method comprising: calculating the horizontal correction angle based on information about the first distance, information about the second distance, and a first rotation angle of the camera. According to clause 11,
The calculating step is,
Obtaining information about a first height from the floor to the center of the projection surface based on the projection angle of the projection unit and the second sensing value obtained through the sensor;
Obtaining information about a second height from the floor to the user's face based on a second rotation angle of the camera and a second sensing value obtained through the sensor;
Obtaining information about a third distance between the user's face and the center of the projection surface based on the information about the first distance, the information about the second distance, and the second rotation angle of the camera; and
A control method including; calculating the vertical correction angle based on the information about the first height, the information about the second height, and the information about the third distance. According to clause 10,
The step of acquiring the frontal image is,
A control method for obtaining a frontal image of the user by rotating the first user image in a first direction based on the horizontal correction angle and rotating the first user image in a second direction based on the vertical correction angle. According to clause 10,
The step of acquiring the frontal image is,
A control method for obtaining a frontal image of the user by inputting the first user image, the horizontal correction angle, and the vertical correction angle into a learned neural network model. According to clause 10,
It includes: generating an avatar image based on the user's frontal image,
The projection step is,
A control method for projecting a video call screen including the generated avatar image. According to clause 10,
The electronic device further includes a driving unit for moving the electronic device,
The control method is,
Identifying whether the electronic device is located in a space where video communication is possible based on at least one of the first user image, the rotation angle of the camera, the projection angle of the projection unit, and the sensing value obtained through the sensor; and
When the electronic device is not located in a space where video calls are available, controlling the driving unit to move to the space where video calls are available. According to clause 10,
The control method is,
acquiring a second user image using the camera while performing the video call;
calculating a rotation angle of the user's face based on the second user image; and
A control method comprising: rotating the front image of the user based on the calculated rotation angle of the user's face. According to clause 10,
The control method is,
When a plurality of users are included in the first user image, it further includes identifying a speaker among the plurality of users,
The calculating step is,
A control method for calculating a horizontal correction angle and a vertical correction angle based on the identified speaker.

Images