TWI899841B - Wearable apparatus and integrating method of virtual and real images - Google Patents

Abstract

A wearable apparatus and a method for integrating virtual and real images. Wearable apparatus includes an image capture device, a projection module and a processor. The processor is configured to: identify a physical display area in the captured image, the physical display area corresponds to the size of the display screen of the display, and the external device includes the display; define one or more virtual display areas based on the location of the physical display area position; and project the interface of one or more applications to one or more virtual display areas through the projection module according to the relevant parameters of one or more applications of the external device. The virtual image projected by the projection module corresponds to the virtual display areas. Therefore, user experience could be improved.

Description

Wearable device and virtual and real screen integration method

The present invention relates to an image processing technology, and more particularly to a wearable device and a method for integrating virtual and real images.

If you need to view and edit multiple files simultaneously, existing devices can display content on a single screen or multiple screens. While connecting multiple screens can increase the number of available screens, these screens take up a lot of physical space and have certain costs and limitations in scalability.

This invention provides a wearable device and virtual reality screen integration method. By integrating the physical screen with the virtual window, data switching during work can be made smoother.

The wearable device of an embodiment of the present invention includes (but is not limited to) an image capture device, a projection module, and a processor. The image capture device is used to capture images. The projection module is used to project virtual images. The processor is coupled to the image capture device and the projection module. The processor is configured to: identify a physical display area in the captured image, where the physical display area corresponds to the size of a display screen of a display, and the external device includes a display; define the positions of one or more virtual display areas based on the position of the physical display area; and project one or more application interfaces onto the one or more virtual display areas via the projection module based on parameters related to one or more applications in the external device. The virtual images correspond to the virtual display areas.

The virtual-reality integration method of an embodiment of the present invention is applicable to a wearable device comprising an image capture device, a projection module, and a processor. The virtual-reality integration method includes (but is not limited to) the following steps: identifying a physical display area in a captured image by a processor, where the physical display area corresponds to the size of a display screen of a display, and the external device includes such a display; defining the positions of one or more virtual display areas by the processor based on the position of the physical display area; and projecting one or more application interfaces onto the one or more virtual display areas via the projection module based on parameters related to one or more applications in the external device. The virtual images projected by the projection module correspond to the virtual display areas.

Based on the above, the wearable device and virtual-reality integration method of the present invention can define a virtual display area based on the physical display area in the captured image and project the application interface onto the virtual display area based on the application's parameters. This allows for automatic projection of appropriate content and a suitable size ratio onto the virtual display area, based on the application's properties and user usage habits.

To make the above features and advantages of the present invention more clearly understood, the following examples are given and described in detail with reference to the accompanying drawings.

1: System Integration

10: Wearable devices

11: Image capture device

12: Projection Module

13, 33: Communication transceiver

14, 34: Storage

15, 35: Processor

30: External devices

121: Light guide plate

E: Eyes

M1, M2: Marking points

S410~S430, S810~S830, S910~S920, S1110~S1120: Steps

DA: Display Area

C1: Capture image

VDA1, VDA1’, VDA2, VDA3: Virtual Display Area

VS: Virtual Screen

PDA: Physical Display Area

MA: Main Application

SA: Sub-application

SA1~SA3, SA1-1~SA1-6, SA2-1~SA2-6, SA3-1~SA3-6: interface

Figure 1 is a block diagram of components of an integrated system according to one embodiment of the present invention.

Figure 2A is a schematic diagram of a wearable device according to an embodiment of the present invention.

Figure 2B is a schematic diagram illustrating projection imaging according to an embodiment of the present invention.

Figure 3A is a schematic diagram of an integrated system according to one embodiment of the present invention.

Figure 3B is a schematic diagram of an integrated system in an application scenario according to an embodiment of the present invention.

Figure 4 is a flow chart of a virtual-real integration method according to an embodiment of the present invention.

Figure 5 is a schematic diagram illustrating the identification of a physical display area according to an embodiment of the present invention.

Figure 6A is a schematic diagram illustrating a physical display area and a virtual display area according to an embodiment of the present invention.

Figure 6B is a schematic diagram illustrating a physical display area and a virtual display area according to another embodiment of the present invention.

Figure 7A is a schematic diagram illustrating window folding according to an embodiment of the present invention.

Figure 7B is a schematic diagram illustrating window expansion according to an embodiment of the present invention.

Figure 8 is a flowchart illustrating sorting according to an embodiment of the present invention.

Figure 9 is a flowchart of the sorting decision according to one embodiment of the present invention.

Figure 10 is a schematic diagram illustrating window allocation based on usage history according to an embodiment of the present invention.

Figure 11 is a flowchart of location allocation according to an embodiment of the present invention.

Figure 12 is a schematic diagram illustrating window allocation based on application attributes according to an embodiment of the present invention.

FIG1 is a block diagram of an integrated system 1 according to an embodiment of the present invention. Referring to FIG1 , the integrated system 1 includes (but is not limited to) a wearable device 10 and an external device 30.

The wearable device 10 can be a head-mounted display, smart glasses, an augmented reality (AR) or mixed reality (MR) wearable device, a head-up display, a smartphone, a tablet computer, or other devices.

The wearable device 10 includes (but is not limited to) an image capture device 11, a projection module 12, a communication transceiver 13, a memory 14, and a processor 15.

The image capture device 11 can be a camera, a camcorder, a monitor, or a circuit with image capture functionality, and is used to capture images within a specified field of view (FOV). For example, Figure 2A is a schematic diagram of a wearable device 10 according to one embodiment of the present invention. Referring to Figure 2A , the wearable device 10 is in the form of glasses (but is not limited to this). Furthermore, the image capture device 11 is located on the front side of the wearable device 10. When a user wears the wearable device 10, the image capture device 11 is used to capture images facing forward from the user. However, the placement and field of view of the image capture device 11 are not limited to those shown in Figure 2A .

The projection module 12 can be a video playback device using digital light processing (DLP), liquid crystal display (LCD), light emitting diode (LED), or other projection display technologies. In one embodiment, the projection module 12 is used to project one or more images, such as virtual images, augmented reality images, photos, or videos.

Figure 2B is a schematic diagram illustrating projection imaging according to an embodiment of the present invention. Referring to Figures 2A and 2B , the projection module 12 is disposed on the front side of the wearable device 10. The light guide plate 121 is, for example, a freeform optical lens. The light beam emitted by the projection module 12 is reflected by the light guide plate 121 toward the eye E, where it is imaged. Furthermore, the eye E can still see the real world. However, the imaging method of the projection module 12 is not limited to the example shown in Figure 2B , and other projection/imaging methods are possible.

Referring to Figure 1 , the communication transceiver 13 can support communication transceiver circuits/transmission interfaces such as Bluetooth, Wi-Fi, mobile networks, optical networks, Universal Serial Bus (USB), Thunderbolt, or other communication technologies. In one embodiment, the communication transceiver 13 is used to receive signals from an external device or transmit signals to an external device 30. In some embodiments, the communication transceiver 13 is used to connect to the external device 30 and transmit or receive data accordingly.

The memory 14 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, a traditional hard disk drive (HDD), a solid-state drive (SSD), or similar device. In one embodiment, the memory 14 is used to store program code, software modules, configurations, data (e.g., images, usage history, or application properties), or files.

The processor 15 is coupled to the image capture device 11, the projection module 12, the communication transceiver 13, and the memory 14. The processor 15 may be a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a visual processing unit (VPU), a tensor processing unit (TPU), or a neural-network processing unit (NPU), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other similar components or a combination of the above components. In one embodiment, the processor 15 is used to execute all or part of the operations of the wearable device 10 and can load and execute one or more software modules, files and/or data stored in the memory 14.

The external device 30 may be a smartphone, tablet computer, wearable device, laptop computer, desktop computer, server, smart home appliance, smart assistant device, in-vehicle system, conference phone, home game console, personal computer, artificial intelligence personal computer (AI PC), or other electronic device.

The external device 30 includes (but is not limited to) a display 32, a communication transceiver 33, a memory 34, and a processor 35.

Display 32 may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED), a mini-LED display, or other displays. In one embodiment, display 32 is configured to display one or more images on its panel/screen.

The implementation and functions of the communication transceiver 33, memory 34, and processor 35 can be found in the aforementioned descriptions of the communication transceiver 13, memory 14, and processor 15, respectively, and will not be further elaborated here.

The processor 35 is coupled to the display 32, the communication transceiver 33, and the memory 34. In one embodiment, the processor 35 is used to execute all or part of the operations of the external device 30 and can load and execute one or more software modules, files, and/or data stored in the memory 34.

FIG3A is a schematic diagram of an integrated system 1 according to an embodiment of the present invention. Referring to FIG3A , the wearable device 10 is connected to the external device 30 via a transmission cable, such as a USB transmission interface. Alternatively, the wearable device 10 can be connected to the external device 30 via wireless transmission.

FIG3B is a schematic diagram of an integrated system according to an embodiment of the present invention in an application scenario. Referring to FIG3B , a user wears a wearable device 10, and an external device 30 is mounted on a table. In this application scenario, the user wears the wearable device 10 while using the external device 30. However, the application scenario shown in FIG3B is merely an example, and other application scenarios are possible.

Figure 4 is a flow chart of a virtual-reality integration method according to an embodiment of the present invention. Referring to Figure 4 , processor 15 identifies a physical display area in a captured image (step S410). Specifically, image capture device 11 captures and obtains the captured image. For example, in Figure 3B , wearable device 10 captures the image facing forward. Figure 5 is a schematic diagram illustrating the identification of a physical display area according to an embodiment of the present invention. Referring to Figure 5 , captured image C1 is captured and displayed on display 32.

The physical display area corresponds to the area of the display screen of the display 32 (hereinafter referred to as the display area DA). The physical display area corresponds to the size of the display screen of the display 32 of the external device 30. The panel of the display 32 emits light and displays images accordingly. Therefore, the size of the display screen of the display 32 is approximately equal to or equal to the size of the panel. As shown in Figure 5, the position of the physical display area corresponds to the display area DA of the display 32.

In one embodiment, the processor 15 may identify two or more markers in the captured image. The markers may be physical objects (e.g., paper, plastic, or stickers) with one or more text, symbols, patterns, shapes, and/or colors printed, embedded, or otherwise artificially added. Alternatively, the markers may consist of text, symbols, patterns, shapes, and/or colors already present on the physical object.

Taking Figure 3B as an example, assume that the markers M1 and M2 are square stickers attached to the external device 30. Referring to Figure 5 , in the captured image C1, the markers M1 and M2 are located at the upper left and lower right corners of the display 32.

Marker point recognition can be based on object detection technology. For example, the external device 30 can apply a neural network-based algorithm (e.g., YOLO (You only look once), Region Based Convolutional Neural Networks (R-CNN), or Fast R-CNN) or a feature matching-based algorithm (e.g., Histogram of Oriented Gradient (HOG), Scale-Invariant Feature Transform (SIFT), Harm, or Speeded Up Robust Features (SURF)) to achieve object detection. The processor 15 can determine whether the object in the captured image is a preset marker point.

The processor 15 can determine the physical display area based on the positions of two or more marker points. The imaginary lines connecting these marker points can form the shape/outline of the physical display area. For example, in Figure 5 , the horizontal imaginary line of marker point M1 intersects the vertical imaginary line of marker point M2, and the horizontal imaginary line of marker point M2 intersects the vertical imaginary line of marker point M1. These horizontal and vertical imaginary lines of marker points M1 and M2 form the shape/outline of the display area DA. The shape/outline of the display area DA can serve as the shape/outline of the physical display area.

In another embodiment, more marker points may be set. These marker points are located at the outline of the physical display area.

In one embodiment, the processor 15 can identify the panel of the display 32 based on the aforementioned object detection technology and determine the physical display area based on the shape/outline of the panel. In other words, the shape/outline of the panel is used as the shape/outline of the physical display area.

In one embodiment, the processor 15 may receive user input via an input device (not shown), such as a handheld controller, a mouse, or a keyboard. This user input may define the shape/outline of the physical display area.

Referring to Figure 4 , the processor 15 defines the location of the virtual display area based on the location of the physical display area (step S420 ). Specifically, the virtual display area is the imaging area of the projection module 12 . That is, when the user wears the wearable device 10 , the image (e.g., virtual screen) projected by the projection module 12 appears on the virtual display area. Areas outside the virtual display area can be disabled, stopped, or not imaged, or imaged by changing transparency, displaying color blocks, or other methods.

In one embodiment, the processor 15 may define one or more virtual display areas to be located at least one of the right side, left side, bottom side, and top side of the physical display area. In other words, the one or more virtual display areas are located to the right, left, bottom, or top of the physical display area.

For example, Figure 6A is a schematic diagram illustrating a physical display area PDA and virtual display areas VDA1-VDA3 according to an embodiment of the present invention. Referring to Figure 6A , the physical display area PDA covers the display area DA of the display 32. Virtual display areas VDA1-VDA3 are located on the right, top, and left sides of the physical display area PDA, respectively. Virtual display areas VDA1-VDA3 are all horizontally shaped rectangles. That is, their horizontal length is greater than their vertical length. Virtual display areas VDA1-VDA3 do not overlap with the physical display area PDA, serving as auxiliary displays. However, in other embodiments, the virtual display area may partially overlap with the physical display area.

For example, Figure 6B is a schematic diagram illustrating a physical display area PDA and virtual display areas VDA1'-VDA3 according to another embodiment of the present invention. Referring to Figure 6B , virtual display areas VDA1'-VDA3 are located on the right, top, and left sides of the physical display area PDA, respectively. Virtual display area VDA1' is an upright rectangle. That is, its horizontal length is shorter than its vertical length.

However, the shape and size of the virtual display area shown in Figures 6A and 6B are merely examples and can be adjusted by the user according to actual needs.

Referring to Figure 4 , the processor 15 projects the application interface of the external device 30 onto the virtual display area via the projection module 12 based on the relevant parameters of the application (step S430). Specifically, the external device 30 executes one or more applications. The applications may be document editors, communication or conferencing programs, design programs, entertainment programs, browser programs, or other types of programs. The operating system may present the application user interface in a windowed, full-screen, or widget format. In addition to displaying the application interface on the display 32 of the external device 30, embodiments of the present invention can also project a virtual screen containing the application interface via the projection module 12. When a user wears the wearable device 10 and uses the external device 30, the virtual display area can serve as an extended image/screen of the display area of the monitor.

For example, Figure 7A is a schematic diagram illustrating window collapse according to an embodiment of the present invention. Referring to Figure 7A , the display 32 displays multiple windowed applications. These applications include a main application MA and a secondary application SA. The application running in the foreground is the main application MA. The application running in the background is the secondary application SA. The display 32 displays the interfaces of the main application MA and the secondary applications SA. In collapsed mode, the interface of the main application MA is stacked on top of the interfaces of one or more secondary applications SA. In other words, the interfaces of the main application MA and the one or more secondary applications are all presented within the physical display area of the PDA.

Figure 7B is a schematic diagram illustrating window expansion according to an embodiment of the present invention. Referring to Figure 7B , in expanded mode, the interfaces of one or more secondary applications SA are presented in virtual display areas VDA1'-VDA3 via virtual screens VS. In some application scenarios, secondary applications SA with no application attributes or usage history related to the primary application MA are located in the physical display area PDA and are covered by the primary application MA. Application attributes and usage history will be described in subsequent embodiments.

In one embodiment, the processor 15 can receive an opening and closing operation. For example, a user operation is received through an input device, and the opening and closing operation performed by the user is used to execute an expanded mode or a collapsed mode. Based on the opening and closing operation, the processor 15 can project the interface of one or more applications to one or more virtual display areas through the projection module 12. As shown in Figure 7B, the opening and closing operation corresponds to the expanded mode, so the interface of the application SA is presented in the virtual display areas VDA1'~VDA3 respectively. On the other hand, the processor 15 can stop projecting the interface of one or more applications to one or more virtual display areas through the projection module 12 based on the opening and closing operation. As shown in FIG7A , the opening and closing operation corresponds to the collapsed mode. The projection module 12 stops projecting, and only the display 32 displays the image. Therefore, the interface of the secondary application SA is displayed in the display area DA.

In one embodiment, in a marked augmented reality imaging, the processor 15 can determine the posture of the wearable device 10 based on the image features of the display area DA and place the image in the virtual display area.

In another embodiment, in markerless augmented reality imaging, the processor 15 may use time, accelerometer, satellite positioning, and/or compass information to locate itself and use the image capture device 11 to overlay the image on the virtual display area.

In one embodiment, processor 15 may determine the size and shape of one or more virtual display areas based on the display properties of the application interface. Display properties may include size, aspect ratio, and mode. For example, if an application interface is 1280*800 pixels with a 16:10 aspect ratio, the virtual display area is a rectangle with a 16:10 aspect ratio, and its size can be scaled to fit the 1280*800 pixel aspect ratio. For example, in Figure 7B , the interface of the secondary application SA on the right side of the image is 800*1280 pixels with a 10:16 aspect ratio. Therefore, virtual display area VDA1′ is an upright rectangle. However, the virtual display areas VDA2 and VDA3 are horizontal rectangles for displaying the secondary application SA located at the top or left of the screen.

In one embodiment, application-related parameters include usage history, multiple applications, and multiple virtual display areas. Figure 8 is a flowchart illustrating sorting according to one embodiment of the present invention. Referring to Figure 8 , the processor 15 may assign priorities to the virtual display areas (step S810). Taking Figure 7B as an example, based on the right-side priority principle, virtual display area VDA1' has the highest priority, virtual display area VDA2 has the second priority, and virtual display area VDA3 has the third priority.

Referring to FIG. 8 , the processor 15 may sort the multiple applications based on their usage history to generate a sorting result (step S820 ). In one embodiment, the usage history includes at least one of the following: the switching frequency of the multiple applications; the simultaneous display time of the multiple applications; and the usage of the multiple applications.

The switching frequency is the number of times one of two applications switches to the foreground and the other switches to the background/non-foreground. For example, during a statistical period (e.g., an hour, a day, or the period from the last boot to the last shutdown), the processor 35 counts the number of times application A switches to application B and application B switches to application A.

The concurrent display time is the cumulative time that two applications are simultaneously running in the foreground. For example, during a statistical period (e.g., three hours, half a day, or the period from the last boot to the last shutdown), processor 35 calculates the cumulative time that applications C and D are concurrently displayed.

Usage is the time that any application is in use. For example, the time between when the interface (e.g., window) of application E is opened and when it is closed.

FIG9 is a flowchart of a ranking decision according to an embodiment of the present invention. Referring to FIG9 , the processor 15 may perform a weighted operation on at least one of the switching frequency, the concurrent display time, and the usage amount according to a corresponding weight (step S910). Specifically, the processor 15 assigns corresponding weights to the switching frequency, the concurrent display time, and the usage amount. For example, the switching frequency has a weight of 60%, the concurrent display time has a weight of 30%, and the usage amount has a weight of 10%. The corresponding relationship between the application and the usage history is:

The weighted value for the web browser is 20*60%+43.8*30%+13*10%=26.44, the weighted value for the email program is 15*60%+36*30%+11*10%=20.9, and the weighted value for the music player is 10*60%+24*30%+24*30%=14.24. However, these weights can be changed based on actual needs.

Referring to Figure 9 , the processor 15 may determine the ranking result based on the weighted calculation value (step S920). Specifically, the larger the weighted calculation value, the higher the ranking or the higher the priority; the smaller the weighted calculation value, the lower the ranking or the less important/less priority.

Referring to FIG. 8 , the processor 15 may project the interfaces of multiple application programs onto virtual display areas corresponding to their priorities based on the ranking results (step S830 ). Specifically, the higher the ranking or the higher the priority, the higher the corresponding priority; the lower the ranking or the less important the application, the lower the corresponding priority.

FIG10 is a schematic diagram illustrating window allocation according to a usage history according to an embodiment of the present invention. Referring to FIG10 , as shown in Table (1), the web browser has the highest corresponding priority, the mail program has the second highest corresponding priority, and the music player has the lowest corresponding priority. The web browser interface SA1 is projected via the virtual screen VS into the virtual display area VDA1′ having the highest priority, the mail program interface SA2 is projected via the virtual screen VS into the virtual display area VDA2 having the second priority, and the music player interface SA3 is projected via the virtual screen VS into the virtual display area VDA3 having the third priority. The interfaces of other applications can be displayed in the display area DA or one of the virtual display areas VDA1'-VDA3 and stacked below the main application MA or interfaces SA1-SA3.

In one embodiment, application-related parameters include application attributes, multiple applications are provided, and multiple virtual display areas are provided. Figure 11 is a flowchart of location allocation according to one embodiment of the present invention. Referring to Figure 11 , the processor 15 may allocate multiple applications to multiple groups based on their application attributes (step S1110). Specifically, the application attributes relate to the application's functionality. For example, the application attributes may be editing, communication, design, entertainment, or browser. Each category corresponds to a group. In other words, multiple applications within a group have the same or similar application attributes.

Tables (2) to (6) show the correspondence between multiple application attributes, applications, and display modes:

Word, Excel, PowerPoint, and Notes are assigned to one group; Outlook, Line, Messages, Teams, and Zoom are assigned to another group, and so on.

Referring to Figure 11 , the processor 15 can project the interfaces of one or more applications onto the virtual display area corresponding to the group via the projection module 12 (step S1120 ). Specifically, one or more group/application attributes correspond to a virtual display area. The projection module 12 projects each application onto the corresponding virtual display area.

For example, Figure 12 is a schematic diagram illustrating window allocation based on application attributes according to one embodiment of the present invention. Referring to Figure 12 , applications with the editing and browser attributes correspond to virtual display area VDA1, applications with the design and entertainment attributes correspond to virtual display area VDA2, and applications with the communication attribute correspond to virtual display area VDA3.

For example, the Excel interface SA1-1, Word interface SA1-2, PDF viewer interface SA1-3, web browser interface SA1-4, memo interface SA1-5, and PowerPoint interface SA1-6 are displayed in virtual display area VDA1 via the virtual screen VS. The After Effects interface SA2-1, Sketch interface SA2-2, Illustrator interface SA2-3, Premiere interface SA2-4, ProtoPie interface SA2-5, and Photoshop interface SA2-6 are displayed in virtual display area VDA2 via the virtual screen VS. The Line interface SA3-1, Messenger interface SA3-2, Meet interface SA3-3, Outlook interface SA3-4, Teams interface SA3-5, and Zoom interface SA3-6 are presented on the virtual display area VDA3 via the virtual screen VS. Furthermore, the display mode (e.g., portrait or landscape) of these application interfaces SA1-1 through SA1-6, SA2-1 through SA2-6, and SA3-1 through SA3-6 can maintain their previous mode/status.

In one embodiment, the processor 15 can identify the interface of the main application in the captured image. For example, based on object detection technology or usage records provided by the external device 30, the main application running in the foreground of the external device 30 is known. Then, based on the relevant parameters of one or more secondary applications, the processor 15 projects the interfaces of one or more secondary applications to one or more virtual display areas through the projection module 12. Taking the usage history as an example of the relevant parameters, please refer to Figure 10. If the main application MA is Figma, then the application located on the virtual display VDA1' is the application with the highest ranking corresponding to the usage history of Figma. For example, Chrome and Figma are often used, Chrome and Figma are often used at the same time, and/or Chrome is used the most. Taking application attributes as an example, refer to Figure 12. Virtual display areas VDA1 through VDA3 each display a secondary application SA associated with the primary application MA, with the positions of these secondary application SAs assigned based on their application attributes. Alternatively, virtual display areas VDA1 through VDA3 can display only secondary application SAs with the same application attributes as the primary application MA.

In summary, in the wearable device and virtual-reality screen integration method of the present embodiment, the position of the projection module's virtual display area is defined based on the display's physical display area, and the application interface is projected onto the virtual display area based on the usage history or application properties of the application executed by the external device.

The present invention provides at least the following features: By integrating the physical screen with the virtual window, data switching during work is made smoother. By learning the content of the main screen and the usage habits of other applications, the virtual window's content and corresponding screen size ratio are automatically switched based on parameter weights.

Although the present invention has been disclosed above through embodiments, they are not intended to limit the present invention. Anyone with ordinary skill in the art may make minor modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the scope of the attached patent application.

S410~S430: Steps

Claims (16)

A wearable device includes: an image capture device for acquiring a captured image; a projection module for projecting a virtual screen; and a processor coupled to the image capture device and the projection module and configured to: identify a physical display area in the captured image, wherein the physical display area corresponds to the size of a display screen of a display, and an external device includes the display; define the position of at least one virtual display area according to the position of the physical display area, wherein the virtual screen corresponds to the virtual display area; and project the interface of the at least one application program on the at least one virtual display program through the projection module according to relevant parameters of the at least one application program of the external device. The processor comprises a display area, wherein the parameters related to the at least one application include a usage history, the usage history including at least one of the following: a switching frequency of the applications; a time period during which the applications are displayed simultaneously; and a usage amount of the applications. The at least one application includes a plurality of applications, and the at least one virtual display area includes a plurality of virtual display areas. The processor is further configured to: assign a priority to each of the virtual display areas; sort the applications according to the usage history to generate a sorting result; and project, via the projection module, the interfaces of the applications on the virtual display areas corresponding to the priorities according to the sorting result. The wearable device of claim 1, wherein the processor is further configured to: identify at least two marker points in the captured image; and determine the physical display area based on the positions of the at least two marker points. The wearable device of claim 1, wherein the processor is further configured to: define the at least one virtual display area to be located at least one of the right side, the left side, and the top side of the physical display area. The wearable device of claim 1, wherein the processor is further configured to: determine the size and shape of the at least one virtual display area based on display properties of the at least one application interface. The wearable device of claim 1, wherein the processor is further configured to: perform a weighted calculation on at least one of the switching frequency, the simultaneous display time, and the usage amount according to a corresponding weight; and determine the ranking result according to a value obtained from the weighted calculation. The wearable device of claim 1, wherein the parameters associated with the at least one application include an application attribute, the at least one application includes a plurality of applications, the at least one virtual display area includes a plurality of virtual display areas, and the processor is further configured to: assign the applications to a plurality of groups based on the application attributes, wherein each group corresponds to a virtual display area; and project, via the projection module, the interfaces of the applications onto a virtual display area corresponding to a group. The wearable device of claim 1, wherein the at least one application includes a main application and at least one secondary application, and the processor is further configured to: recognize an interface of the main application in the captured image; and project the interface of the at least one primary application onto the at least one virtual display area via the projection module based on parameters related to the at least one secondary application. The wearable device of claim 1, wherein the at least one application includes a main application and at least one secondary application, the display displays the interface of the main application, and the processor is further configured to: receive an opening and closing operation; and project the interface of the at least one application onto the at least one virtual display area via the projection module in response to the opening and closing operation; or stop projecting the interface of the at least one application onto the at least one virtual display area via the projection module in response to the opening and closing operation. A virtual and real image integration method is applicable to a wearable device, the wearable device including an image capture device, a projection module, and a processor. The virtual and real image integration method includes: identifying a physical display area in a captured image obtained by the image capture device through the processor, wherein the physical display area corresponds to the size of a display screen of a display, and an external device includes the display; The processor defines a position of at least one virtual display area according to the position of the physical display area, wherein a virtual screen projected by the projection module corresponds to the virtual display area; and the processor projects an interface of the at least one application program to the at least one virtual display area through the projection module according to relevant parameters of the at least one application program of the external device, wherein the relevant parameters of the at least one application program are The data includes a usage history, the usage history including at least one of the following: a switching frequency of the applications; a simultaneous display time of the applications; and a usage amount of the applications, the at least one application includes a plurality of applications, the at least one virtual display area includes a plurality of virtual display areas, and the at least one application is projected according to the relevant parameters of the at least one application of the external device. The step of displaying the application interface of the program to the at least one virtual display area includes: assigning a priority to each of the virtual display areas by the processor; sorting the applications by the processor based on the usage history of the applications to generate a sorting result; and projecting the application interfaces of the application programs by the projection module on the virtual display area corresponding to the priority based on the sorting result. The virtual-reality image integration method of claim 9, wherein the step of identifying the physical display area in the captured image comprises: identifying at least two marker points in the captured image by the processor; and determining the physical display area by the processor based on the positions of the at least two marker points. The virtual and real screen integration method of claim 9, wherein the step of defining the position of the at least one virtual display area based on the position of the physical display area includes: defining, by the processor, that the at least one virtual display area is located at least one of the right side, the left side, and the top side of the physical display area. The virtual and real screen integration method of claim 9 further comprises: determining, by the processor, the size and shape of the at least one virtual display area based on the display properties of the at least one application interface. The virtual and real screen integration method of claim 9, wherein the step of projecting the application interfaces onto a virtual display area corresponding to the priority using the projection module based on the ranking result comprises: performing a weighted calculation on at least one of the switching frequency, the simultaneous display time, and the usage amount based on a corresponding weight; and determining the ranking result based on the value of the weighted calculation using the processor. The method for integrating virtual and real screens as described in claim 9, wherein the parameters related to the at least one application include an application attribute, the at least one application includes a plurality of applications, the at least one virtual display area includes a plurality of virtual display areas, and the step of projecting the interface of the at least one application onto the at least one virtual display area based on the parameters related to the at least one application of the external device comprises: assigning, by the processor, the applications into a plurality of groups based on the application attributes, wherein each group corresponds to the virtual display area; and projecting, by the projection module, the interfaces of the applications onto a virtual display area corresponding to the group. The method for integrating virtual and real images as described in claim 9, wherein the at least one application includes a main application and at least one secondary application, and the step of projecting the interface of the at least one application onto the at least one virtual display area based on parameters related to the at least one application from the external device comprises: identifying the interface of the main application in the captured image by the processor; and projecting the interface of the at least one primary application onto the at least one virtual display area by the projection module based on the parameters related to the at least one secondary application by the processor. The method for integrating virtual and real screens as described in claim 9, wherein the at least one application includes a main application and at least one secondary application, the display displays the interface of the main application, and the method further comprises: receiving an opening and closing operation via the processor; and projecting the interface of the at least one application onto the at least one virtual display area via the projection module in response to the opening and closing operation; or ceasing to project the interface of the at least one application onto the at least one virtual display area via the projection module in response to the opening and closing operation.