JP2005031510A - Information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve user operability and convenience in an information processing apparatus for displaying a moving image.
Based on image data prepared in a storage device, a moving image is displayed in an image display area of a display screen independently of drawing processing of a UI (user interface) display component by an operating system. When displaying the UI display component, if the display region of the display component and the image display region of the moving image overlap with each other, a portion corresponding to the display region of the display component in the image data constituting the moving image is displayed. The removed image data is drawn in the image display area. This prevents the UI display component from being erased by the drawing update for moving image display.
[Selection] Figure 1

Description

  In the information processing apparatus having a moving image display function, the display area of the display component is displayed when the display area of the display component for user interface and the display area of the moving image interfere with each other on the screen. The present invention relates to a technique for avoiding moving image display.

  An operating system (hereinafter simply referred to as “OS”) installed in a computer device provides various functions necessary for application processing, but pointing in the form of a GUI (Graphical User Interface) that enhances user operability. Intuitive icon selection and menu operations can be performed on the device.

  For example, when a user activates an application for displaying a moving image and displays a desired moving image data in the entire display area and performs some operation, a GUI component such as a menu or a window is rendered. The display area of the moving image is partially hidden by the drawing area, but when the GUI component returns to the non-display state, the display part that has been hidden until then returns to the original state. For such processing, the display and non-display of the GUI component and the display range are managed by the function of the OS, and redrawing is performed for the overlapping range between the drawing region of the GUI component and the display region of the moving image.

  By the way, when displaying a moving image independently of drawing of a GUI component under an OS in an apparatus or the like equipped with an OS that does not have a moving image display function as its own function, for example, a GUI component Since the display area is overwritten by the drawing of the moving image, the operability may be adversely affected.

  FIG. 5 shows an example of a display screen. “Edit” and “File” items are displayed on the menu bar of the application, and a pull-down menu (Cut) displayed when the user selects “Edit”. , Copy, Paste) shows a state in which a part of the image is not visible due to the drawing of the moving image (see the image display area “GA”).

  Of GUI parts such as menus drawn by the OS, the range that overlaps with the moving image display area is rewritten by image updating (new drawing) for displaying moving images. The range necessary for the operation is hidden behind, making it difficult for the user to operate. Therefore, in order to prevent such inconvenience, it is sufficient to pause the video display during menu operation. However, for convenience, the video display should not be interrupted even if GUI parts are drawn. Is preferred. In addition, there is a method of shifting the display positions so that the display range does not overlap, but such a process is not always possible and the process becomes complicated.

  Therefore, an object of the present invention is to improve user operability and convenience in an information processing apparatus that displays moving images.

  In order to solve the above-described problem, the present invention is based on the image data prepared in the storage device, and the moving image is displayed in the image display area of the display screen independently of the drawing processing of the display component for the user interface by the operating system. When displaying the image and displaying the display component, when the display area of the display component and the image display area of the moving image overlap each other (interference between areas), Image data other than the portion corresponding to the display area of the display component is configured to be drawn in the image display area.

  Therefore, according to the present invention, in the display state in which the display component for user interface (GUI component or the like) is drawn on the display screen, the moving image can be displayed in the image display area excluding the display range of the display component. it can.

  According to the present invention, the user interface display component on the display screen is not overwritten by an image drawn thereafter, and it is not necessary to take measures such as stopping the moving image display. Therefore, user operability and convenience can be improved. Even when the OS does not have a moving image display function as its own function, it is possible to display a moving image without any trouble independently of drawing of a menu or the like by the OS.

  In addition, a mask area composed of setting data for defining display and non-display of image data is provided for each display position in the image display area, and the mask area is updated according to the display area of the display component for user interface. Thus, a moving image can be drawn while avoiding the display area of the display component.

  The processing speed is increased by providing a device for designating the memory address related to the setting data of the mask area, the address of the image data as the copy source, and the address of the copy destination and transferring the image data to the video memory area. Or the processing load on the central processing unit can be reduced. Alternatively, without using such an apparatus, the image data is transferred to the video memory area while referring to the setting data of the mask area by a program executed by the central processing unit without changing the hardware configuration. It can be handled by program processing.

  In an information processing apparatus capable of displaying a moving image, the present invention provides a display device for a user interface when there is interference on the display screen between the display area of the user interface display component and the display area of the moving image. It is an object of the present invention to prevent the overlapping range of both display areas from being overwritten by a moving image and to continue moving image display while the user interface display component is normally displayed.

  In the application of the present invention, it is assumed that the information processing apparatus is equipped with an OS having a function of drawing a display component for a user interface on the display screen of the display apparatus and can display a moving image. For example, it can be used for a portable computer, a PDA (Personal Digital Assistance), a mobile communication terminal device, a video device, an imaging device, and the like.

  In displaying a moving image, when displaying image data in an image display area of a display screen independently of drawing processing of a display component such as a menu by the OS based on image data prepared in a storage device, It is necessary to cope with the case where the display ranges overlap between the display component and the moving image. For example, when the OS does not have a moving image display function as its own function, and a display component such as a menu is displayed by a user operation during the moving image display, the moving image display Therefore, it is necessary to prevent the display component from being erased by updating the image (otherwise, the user may not be able to perform menu operations or the like).

  In the present invention, when displaying a display component such as a menu, if the display area of the display component and the image display area of the moving image overlap each other, the display data of the display component is given priority and image data constituting the moving image is displayed. And a means for drawing image data other than the portion corresponding to the display area of the display component in the image display area. That is, the image data related to the moving image is displayed in the image display area related to the moving image by displaying the image in the range excluding the display area of the display component such as the menu that is the overlapping range. Even in such a case, the moving image display can be continued in the other image display range.

  FIG. 1 is a diagram for explaining an image processing example according to the present invention.

  The image shown at the left end shows one image “G” constituting the moving image. For example, MPEG (Moving Picture Experts Group) 4 decode data and the like are prepared in the storage device. Then, the image data to be transferred is written to, for example, a video memory area (or VRAM area), and then transferred to a display device for image display.

  The areas MA and MB shown in the center of the figure both indicate mask areas. Of these, the upper area MA indicates a mask area for displaying the entire area of the image “G”, and the lower area MB indicates an area excluding a part of the image “G”. The mask area | region for making it show is shown.

  The mask area is composed of setting data (mask data) for defining display and non-display of an image for each display position with respect to the image display area. For example, “1” is displayed using binary data. When “0” is defined as non-display, 8 pixels can be represented by 1 byte.

  In the mask areas MA and MB, the parts shown in satin represent the range in which pixels are to be displayed. In the mask area MA shown in this example, the pattern (mask pattern) made of the mask data is rectangular and uses MA. As shown in the upper right figure (A), the data of the image G is displayed over the entire area (the range indicated by the checkered pattern in the figure conceptually indicates the non-display range in the VRAM).

  In the mask area MB, a portion indicated by a white rectangular range “L” (see a broken line frame) is not displayed, and a satin portion excluding this range represents a range in which pixels are to be displayed. Therefore, the mask pattern in this case has a shape lacking a part (the upper left part in this example), and as shown in the lower right figure (B), a part “K” of the image G corresponding to the part is not displayed. It becomes.

  As described above, by setting a mask area composed of setting data for defining display and non-display of an image for each display position, an image display of the entire area or a partial area of the image G is performed. Alternatively, non-display processing can be performed (for example, logical product operation of image data and mask data).

  The range “L” corresponds to the display range of a display component such as a menu, and the non-display portion “K” is an area hidden on the screen by the display component. The procedure for displaying a moving image in a display range excluding a specific display range using such a mask pattern is generally as follows.

(1) When displaying a display part such as a menu, the display area is detected. (2) If an overlap occurs between the display area of the menu and the image display area of the moving image, the mask area The mask area is updated by processing the portion corresponding to the display area of the display component (see “L” above) as a non-display area. (3) Of the image data related to the moving image, updated at (2) Only the portion to be displayed is drawn on the display screen according to the set data of the mask area.

  In this way, when displaying a display component such as a menu, the mask area is updated each time, and a moving image can be displayed in a range designated using the mask area.

  Examples of the configuration according to the present invention include the following examples.

(I) A form using a memory transfer device related to image data (II) A form in which image data is transferred to a memory by a central processing unit (CPU).

  First, in the form (I), an apparatus or the like having a DMA (Direct Memory Access) transfer function is used to reduce the processing load on the CPU and increase the processing speed. For example, when image data drawing processing is performed in the image display area on the display screen corresponding to the mask area, the memory address related to the setting data of the mask area, the address of the image data that is the copy source, and the address of the copy destination And an apparatus for transferring the image data to the video memory area of the storage device.

  FIG. 2 shows a main part of such a basic configuration example of the information processing apparatus 1, and illustrations of various devices (cameras, audio processing devices, etc.) other than that are omitted.

  The information processing apparatus 1 includes a CPU 2, a storage device (memory) 3, a display device 4, and an input device (or operation device) 5.

  The CPU 2 reads the OS and application programs on the storage device 3 and executes them sequentially.

  A device (semiconductor memory such as SDRAM) having a short access time is used for the storage device 3 for storing programs and data.

  In this example, the display device 4 is a liquid crystal display device (LCD), and includes a controller (LCD controller) 4a and an image display unit (LCD panel) 4b. The controller 4a reads data in the video memory area assigned to a specific address on the storage device 3 at a predetermined timing and transfers it to the image display unit 4b. The transferred data is displayed as an image on the image display unit 4b.

  Examples of the input device 5 include a pen tap device, a keyboard, a dial operation device, and the like, and can transmit a user operation command to the CPU 2.

  The apparatus 6 (hereinafter referred to as “graphic engine”) having the DMA transfer function described above has a function of rectangularly transferring image data from a specific address on the storage device 3 to another address in response to an instruction from the CPU 2. Have. However, unlike the normal DMA, after transferring one line in the horizontal direction, the transfer destination address can be skipped to the beginning of the drawing position of the next line. That is, taking FIG. 1A as an example, in the image G, after performing line transfer for 1H from the start position of the Nth horizontal line, jump from the end position to the start position of the next horizontal line. From there, the image data of the rectangular area can be transferred, such as transferring one horizontal line. In addition, when performing rectangular transfer, an address of mask data can be specified, and a function of masking and transferring a specific portion of the rectangular area is provided. Note that this is required when only a specific range in the image G is hidden as described with reference to FIG.

  The above apparatus is, for example, under the control of an OS having a GUI function, and an application program can use GUI parts (display parts) such as menus and windows by calling an OS service routine. . As described above, if the area on the video memory is rewritten at a specific timing, a moving image can be displayed on the display screen. However, in the following, the OS installed in the apparatus displays the moving image as its own function. Assume that it has no function.

  FIG. 3 is a conceptual diagram illustrating a software layer on a hardware layer as a lower layer.

  Only the graphic engine 6 is shown in the hardware layer, and a device driver 7, an OS 8, and an application (program) 9 are shown in this order.

  The graphic engine 6 has processing functions such as color space conversion (color space conversion) and a video scaler in addition to the DMA transfer described above. The present invention relates to a function as a memory data transfer device. .

  In the following, assuming that the application 9 for displaying a moving image is operating, when the OS 8 that does not support moving image display displays a GUI component such as a menu, the display of the GUI component and the moving image The screen mediation operation for rendering processing not competing due to interference with display will be described with reference to FIG. In the display screen examples shown in FIGS. (A) and (B), FIG. (A) shows the state where the menu screen is not operated, and (B) shows the pull-down menu displayed by the user operation. Indicates the state.

  First, when the application 9 is initialized, mask data corresponding to a moving image display area is created at an appropriate timing. As for the mask area created here, the entire area where the moving image is to be displayed is designated as indicated by the area “GA” in FIG. When the graphic engine 6 is sequentially activated from the application 9 at a timing when an image is desired to be displayed, the data in the range designated by the mask data specified at the time of the transfer is stored in the video memory among the image data prepared on the memory. A moving image is displayed after being transferred to a rectangle. In other words, in FIG. (A), a moving image is displayed over the entire image display area, and the application 9 is operating in a normal mode (a state where there is no interference with the image display area due to a pull-down menu or the like). Represents.

  Next, when the user (operator) performs, for example, a menu operation using the input device 5 while the application 9 is displaying a moving image, the application 9 calls a service routine of the OS 8. , Entrusts menu processing to OS8. The OS 8 normally has a function for replacing the service routine (for example, replacing an event handler). If a patch (additional processing portion) is previously assigned to a service routine that displays a menu, this patch portion is called when the menu processing is performed. The service routine for drawing the menu includes information such as the position and size for drawing the menu. For example, in the case of a rectangular display component, the upper left position and the lower right position on the screen and the vertical and horizontal sizes are included. The drawing range is determined by the information of (height and width). Therefore, when this patch part is called up, the mask data is updated by calculating the part where the area where the menu is displayed and the mask area overlap in the processing of that part (that is, setting the overlapping part to be hidden) Call the service routine (default handler) of the original OS (before applying the patch). Even when the menu is displayed on the screen, the mask pattern is updated by subtracting the range in which the menu is displayed from the entire mask pattern. Therefore, as shown in the region “GB” in FIG. The moving image is displayed in an area excluding the menu display area, and screen arbitration is performed between the OS drawing area (menu portion) and the moving image image display area.

  Also, when the operator finishes selecting the menu, there is an OS service routine called at that timing. Therefore, by applying a patch to the service routine in advance, the mask area has a timing at which the menu is erased. Can be returned to the initial state (the state before the menu is displayed). When the menu is erased by this process, the mask data is again designated as a value indicating display (for example, logical value “1”) for all the drawing areas, and the masking process of the moving image area corresponding to the menu drawing area is canceled. Then, the whole area of the moving image is displayed as shown in FIG.

  The processing described above can be summarized as a bulleted list as follows.

(1) Whole display of moving image (2) User operation (menu operation, etc.)
(3) Drawing of GUI parts by OS (4) Detection of drawing area of GUI parts (acquisition of drawing range)
(5) Update of mask area according to drawing area of GUI part (6) Display of image data according to mask area (partial display of moving image)
(7) User operation (end of menu operation, etc.)
(8) Deletion of GUI part by OS (9) Update of mask area according to deletion of GUI part (10) Display of image data according to mask area (return to (1)).

  Note that the graphic engine 6 is in charge of the processes (6) and (10), and this is started by a program (application 9) executed by the CPU 2 to perform rectangular transfer according to mask data and image data, and to the video memory. The image is displayed by writing the data. In this example, the processes (5) and (9) are handled by the device driver 7 located between the graphic engine 6 and the OS 8 in FIG. That is, the driver sets the mask area for the moving image display area, or the mask area when the display area of the GUI part overlaps the moving image display area at the timing when the OS 8 displays the GUI part or the like. It has a function to update. As a result, the display area of the GUI component is not erased by drawing the moving image.

  In FIG. 4, for ease of explanation, one pull-down menu is illustrated as a GUI component. However, even when a plurality of such menus are displayed by a user operation while displaying a moving image, Processing is easy because the display area of each GUI component is detected and the sum is calculated (OR operation or the like) and the mask area is updated (the display area of each GUI part in the moving image display area). The image can be displayed in the area excluding. Further, in the data update of the mask area, the non-display area is not limited to the rectangular range, so that the moving image can be displayed while avoiding the display area of GUI parts having various shapes.

  When the graphic engine 6 is not required in the application of the present invention, for example, when the processing capability of the CPU is high and the decrease in the image processing speed is not a problem, the DMA transfer is performed as in the above-described form (II). There is no need to rely on. In other words, when image data is drawn in the image display area corresponding to the mask area, the image data is transferred to the video memory area while referring to the mask area setting data (mask data) by the program executed by the CPU. Just do it.

  According to the configuration described above, for example, the following advantages can be obtained.

  It is possible to prevent problems that may occur when a moving image is displayed on an OS that does not have a moving image display function as its own function, independently of GUI component drawing by the OS. That is, it is possible to prevent the GUI component on the screen from being overwritten by updating the image for displaying the moving image (drawing the next data). In addition, there is no need for processing such as stopping moving image display or shifting the display position by displaying GUI parts.

  It is possible to perform arbitration on the screen for a range where the GUI display of the OS and the moving image display overlap, and to guarantee a sense of unity in operation without impairing user operability.

It is explanatory drawing of the image processing example which concerns on this invention. It is a block diagram which shows the basic structural example of the information processing apparatus which concerns on this invention. It is a conceptual diagram which shows the hierarchical structure of a device and software. It is a figure for demonstrating the relationship between GUI component display and moving image display. It is a figure for demonstrating the conventional problem.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus, 2 ... Central processing unit, 3 ... Memory | storage device, 4 ... Display apparatus, 6 ... Transfer apparatus to a video memory area, 8 ... Operating system, MA, MB ... Mask area

Claims (5)

Information that includes a central processing unit that performs computations according to programs on the storage device and a display device that displays computation results, and is equipped with an operating system that has a function of drawing display components for user interfaces on the display screen of the display device In the processing device,
Based on the image data prepared in the storage device, the moving image is displayed in the image display area of the display screen independently of the drawing processing of the display component by the operating system,
If the display area of the display component and the image display area overlap each other when the display component is displayed, an image other than the portion corresponding to the display area of the display component in the image data constituting the moving image An information processing apparatus characterized by drawing data in the image display area. The information processing apparatus according to claim 1,
The information processing apparatus, wherein the operating system does not have a moving image display function as its own function. The information processing apparatus according to claim 1,
For the image display area, a mask area consisting of setting data for defining display and non-display of an image is set for each display position, and
When displaying the display component, the display area is detected, and when the display area and the image display area overlap, a portion corresponding to the display area of the display component is processed as a non-display area in the mask area. The information processing apparatus, wherein the mask area is updated and only a portion to be displayed according to the updated mask area is drawn on the display screen among the image data constituting the moving image. The information processing apparatus according to claim 3.
When performing drawing processing of image data on the image display area corresponding to the mask area, the memory address related to the setting data of the mask area, the address of the image data that is the copy source, and the address of the copy destination are designated. An information processing apparatus comprising: a device that transfers image data to a video memory area of the storage device; and image display is performed by starting the device by a program executed by the central processing unit. The information processing apparatus according to claim 3.
When image data is drawn in an image display area corresponding to the mask area, the image data is stored in the video of the storage device while referring to the setting data of the mask area by a program executed by the central processing unit. An image processing apparatus that displays an image by transferring it to a memory area.

Images