JP2006033607A - Program and system for processing digital image, and recording medium for the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily generate image information with sound by defining high definite still image information as a main body, by easily synchronizing the still image information with sound information with an arbitrary timing. <P>SOLUTION: A digital image processing program allows a computer to perform following procedures; i.e., a sound data string storing procedure to store a sound data string, constituted of sound data which are time-sequentially and continuously arranged and an image data string storing procedure to generate the image data string of a prescribed frame rate by time-sequentially and continuously or arbitrarily arranging free frames, without the still image data and real frames with the still image data, and storing the real frames of the image data string, in correlation with the sound data of the sound data string so as to be respectively synchronized by an arbitrary timing. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a digital video processing program, a digital video processing device, and a recording medium for the digital video processing device, and more particularly, digital video processing suitable for creating a high-definition digital video mainly composed of still images arbitrarily synchronized with audio. The present invention relates to a program, a digital video processing apparatus, and a digital video processing program recording medium.

In recent years, a DVD (Digital Versatile Disc) has rapidly spread as a high-quality digital video recording medium, and the DVD video standard is generally used as a video recording project for recording on this DVD. In the DVD video standard, a compression encoding method (MPEG-2 video encoding) called MPEG-2 is used for moving image data compression. According to this MPEG-2 video coding, for example, by using a coding rate of 4.5 Mbps on average, the video data (moving image data) for 135 minutes is converted into audio data, acoustic data, etc. while satisfying a certain image quality. Can be recorded together, and they can all be recorded on a DVD with a general capacity (about 4,7 Gbytes). The DVD video standard can be easily used in general households, and is considerably improved compared to the conventional VHS standard in terms of image quality, sound quality, and storage stability.
As an information processing apparatus that outputs video data and audio data, for example, there is a technique described in Patent Document 1.
JP-A-8-9357

  Patent Document 1 discloses an information processing apparatus that makes it easy to understand the correspondence between video and audio by simultaneously displaying two or more videos. This information processing apparatus makes it possible to set a plurality of video display windows on a bitmap display. Each video display window includes a video display area for displaying a video and a volume display area for displaying the volume of audio accompanying the video. The CPU reads the video file from the file storage device, decompresses it, and supplies it to the video display control device. The CPU also reads an audio file from the storage device, converts the data into a control signal, transfers it to the audio control device, calculates the volume and level, and displays a volume display that expresses them in brightness and hue as a video display. The control device is instructed to display in the volume display area of the video display window for displaying the corresponding video. When the video display is delayed in the volume output, the CPU thins out the video frame and synchronizes the video and the audio.

  Assume that the video is intended to realize what the user wants to see as clearly as possible (with as high a resolution or high image quality as possible). However, according to the MPEG-2 video coding as the conventional DVD video standard, high-quality broadcasting demanded by people such as the start of high-definition broadcasting for moving images has increased. In the case of display, although it is possible to ensure the same quality (720 × 480 pixels) as the television image quality compared to the moving image, the image quality that can realize a sufficient video effect particularly with an image quality of about 720 × 480 pixels, It can be said that it is insufficient. This is because the main purpose of MPEG-2 video encoding is to encode moving image data such as movie content, and it is sufficient that the image resolution is 720 × 480. However, in this image quality, still images are mainly used. When dealing with such a video, it is not possible to achieve a full video expression. That is, the conventional DVD video standard cannot satisfy even the image quality of a 16 mm movie in terms of handling still images, and is further insufficient to reproduce the amount of information possessed by 35 mm movies and 35 mm still image films.

  In addition, MPEG-2 video coding as a typical compression standard can process only images up to a maximum of 1,920 × 1,080 pixels (horizontal × vertical). Furthermore, such editing software and editing equipment for high-quality MPEG-2 video encoding are only put into practical use in the special business field, and are very expensive and have very high-level computer hardware. The highest processing capacity is required. This is because an image with a large resolution (size) increases in data amount dramatically, and when the image resolution (horizontal size and vertical size) is doubled, the size of the image data is 4 times the square of 2. It is to become. For example, for an uncompressed file, an image with a resolution of 3000 × 2000 has a size amount of the image data of about 17.4 times that of an image with a resolution of 720 × 480. The same thing can be expected when, for example, normal moving image format data created with an image of 3000 × 2000 is reproduced, as in the case of data creation. In other words, in a normal moving image, 24 frames or 30 frames per second must be continuously displayed in the same size according to the sound. Such high-resolution video is not in an environment that can be played at home. In other words, there is currently no environment in which video media can be played back at a resolution higher than the current DVD (720 × 480) on a popular TV, and a high-definition TV for high-definition TV has a low penetration rate.

  On the other hand, when utilizing many still images in the world as video videos, it is conceivable to use a personal computer display as one of the video playback environments, but currently it has an amount of information equivalent to 35 mm film. The image quality cannot be reproduced by a personal computer, and there is a limit in editing a still image having an information amount equivalent to 35 mm film while maintaining a high image quality. Furthermore, in conventional video creation software, still images are generally downsized and used for video, and are not used with their original high image quality. Taking advantage of the fact that a digital camera with high image quality, for example, a camera capable of shooting with 6.30 million pixels (3072 × 2048), can be generally obtained, the present inventor has made a high-quality digital camera (digital camera). ) Was eagerly studied whether images of a large size (resolution) as taken in (1) could be visualized with the same size (high quality). Furthermore, the present inventor has found that personal computer monitors (displays) that are widely used in the workplace and at home are very suitable for still image display, and due to recent technological advances, the images on personal computer monitors are rapidly becoming high-definition. Also, paying attention to the fact that the image quality is becoming high, we have intensively researched the technology that can use this to provide digital video or video video with the same amount of information (high image quality) as or similar to the conventional film image quality.

  Although the technique described in Patent Document 1 processes video information and image information, it cannot provide digital video or video video having an information amount (high image quality) equivalent to or close to the conventional film image quality.

  Therefore, the present invention can easily synchronize video information mainly composed of still images and audio information at an arbitrary timing and easily create video information with audio mainly composed of high-definition still image information. It is another object of the present invention to provide an easy-to-use digital video processing program, a digital video processing device, and a recording medium for the digital video processing program that can be played back more easily.

  According to a first aspect of the present invention, there is provided a digital video processing program comprising: A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series; and B) an empty frame having no still image data. In addition, real frames having still image data are continuously arranged in time series to create an image data sequence having a predetermined frame rate, and real frames of the image data sequence are arbitrarily set as audio data of the audio data sequence, respectively. An image data sequence storage procedure for storing the images in association with each other so as to synchronize with timing is executed by the computer.

  According to a second aspect of the present invention, there is provided a digital video processing program comprising: A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series; and B) generating an image data sequence at a predetermined frame rate. The image data sequence storing procedure is executed by a computer, and the image data sequence storing procedure stores the actual frames having still image data in association with the audio data of the audio data sequence so as to be synchronized at an arbitrary timing in order. A frame arrangement procedure; and C) when the adjacent real frames are arranged at a time interval exceeding the frame arrangement interval of the frame rate, the frame arrangement procedure is located at a position corresponding to the frame arrangement interval of the frame rate between the adjacent real frames. An empty frame having no still image data is additionally arranged so that the real frame and the empty frame And a null frame arrangement procedure to place to be arranged in order in the frame arrangement interval by the frame rate.

  According to a third aspect of the present invention, there is provided a digital video processing program comprising: A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series; and B) a plurality of frames in order at a predetermined frame rate. In the case of creating an image data sequence by arranging, a storage location securing procedure for securing the storage location of the frame in the storage means, and C) synchronizing with any of a plurality of reproduction timings in the audio data sequence of the frame Still image data is written to the storage location of the storage means for frames having a reproduction timing as actual frames, and frames other than the actual frames among the frames are held as empty frames to which still image data is not written. And causing the computer to execute an image data string storage procedure.

  A digital video processing program according to claim 4 is a digital video processing program for creating a digital video with audio in an AVI file format, and A) is continuously arranged in time series in an audio stream of an AVI file. An audio data sequence storing procedure for storing audio data; and B) an image data sequence storing procedure for storing image data continuously arranged in time series in the video stream of the AVI file, and the image data The column storage procedure is B1) a video stream preparation procedure for preparing the video stream at a predetermined frame rate in the AVI file, and B2) a reproduction timing synchronized with audio data at an arbitrary reproduction timing in the audio data sequence. , As a frame number and still image in the video stream The frame data consisting of the map data is written in order to make a real frame, and when there is a difference of two or more frame numbers between the adjacent real frames, an empty frame in which still image data is not written is added between the adjacent real frames. And a real frame data writing procedure.

  According to a fifth aspect of the present invention, there is provided a digital video processing program according to the fourth aspect, wherein the AVI file includes the video stream composed of compressed image data compressed using the AVI file compression program, and the AVI file compression program. The audio stream is composed of compressed audio data compressed by use.

  Here, the AVI file is largely composed of image data and audio data by ordinary non-compressed storage, and data volume compressed using a commonly distributed AVI file compression program, There are two types. Among them, the digital video processing program according to the fifth aspect includes an AVI file obtained by compressing video data and audio data by using a compression program.

  According to a sixth aspect of the present invention, there is provided the digital video processing program according to any one of the first to fourth aspects, wherein the still image is composed of bitmap data in an uncompressed state.

  A digital video processing program according to a seventh aspect of the present invention is the digital video processing program according to any one of the first to sixth aspects, wherein the audio data sequence and the image data sequence are stored in separate tracks and individually adjusted for reproduction timing. I am doing so.

  The digital video processing program according to an eighth aspect of the present invention is the digital video processing program according to any one of the first to seventh aspects, wherein when an empty frame exists between adjacent real frames of the image data sequence, The computer is caused to execute an auxiliary frame addition procedure for writing the still image data of the first real frame among the adjacent real frames to create an auxiliary frame at the position of the empty frame immediately before the last real frame.

  A digital video processing program according to a ninth aspect of the present invention is the digital video processing program according to any one of the first to seventh aspects, wherein if there is an empty frame between adjacent real frames of the image data sequence, The computer is caused to execute a pointer information addition procedure for writing pointer information of the first real frame among the adjacent real frames to all empty frames.

  According to a tenth aspect of the present invention, in the digital video processing program according to any one of the first to ninth aspects, the computer further executes a reproduction timing description procedure that describes the reproduction timing for each real frame of the image data sequence Let

  A digital video processing program according to an eleventh aspect of the present invention is the digital video processing program according to any one of the first to tenth aspects, wherein when an empty frame exists between adjacent real frames of the image data sequence, After the still image of the actual frame is reproduced, the still image of the first actual frame is held and reproduced until the next actual frame arrives.

  A digital video processing apparatus according to a twelfth aspect includes a computer storing the digital video processing program according to any one of the first to eleventh aspects.

  A recording medium for a digital video processing program according to a thirteenth aspect records the digital video processing program according to any one of the first to eleventh aspects.

  The digital video processing program according to claim 1 can easily synchronize video information mainly composed of a still image and audio information at an arbitrary timing, and has a video with audio mainly composed of high-definition still image information. Information can be created easily.

  The digital video processing program according to claim 2 can easily synchronize video information mainly composed of a still image and audio information at an arbitrary timing, and has a video with audio mainly composed of high-definition still image information. Information can be created easily.

  The digital video processing program according to claim 3 can easily synchronize video information mainly composed of a still image and audio information at an arbitrary timing, and has a video with audio mainly composed of high-definition still image information. Information can be created easily.

  The digital video processing program according to claim 4 can easily synchronize video information mainly composed of a still image and audio information at an arbitrary timing, and has a video with audio mainly composed of high-definition still image information. Information can be created easily. In addition, video data can be created as a popular AVI file, which is easy to use.

  In addition to the effect of the fourth aspect, the digital video processing program according to the fifth aspect can provide an AVI file in which the data capacity is further reduced by compression.

  The digital video processing program according to claim 6 can provide high-resolution video without compression in addition to the effects of any one of claims 1 to 4.

  The digital video processing program according to claim 7 can accurately synchronize image data with audio data at an arbitrary timing in addition to the effects of any one of claims 1 to 6.

  A digital video processing program according to an eighth aspect of the present invention, in addition to the effects of any one of the first to seventh aspects, includes a process for rewinding an image data sequence, and the like of an image data sequence to be subjected to the rewinding operation. When an empty frame exists between adjacent real frames, the still image data of the last real frame is prevented from being adopted as still image data to be reproduced after the rewinding operation, and the auxiliary frame (first It is possible to reproduce a still image of the same actual frame as that of the actual frame.

  A digital video processing program according to a ninth aspect of the present invention, in addition to the effect of any one of the first to seventh aspects, provides the image data that is a target of the rewinding operation or the like when performing a rewinding operation or the like of the image data sequence. When there is an empty frame between adjacent real frames in a column, the empty frame is read out via pointer information, and the first real frame of the adjacent real frames is reused and reproduced. Can do.

  The digital video processing program according to claim 10 can describe the reproduction timing in each real frame of the image data in addition to the effect of any of claims 1 to 9, and the image data can be combined with the audio data at an arbitrary timing. It can be accurately synchronized.

  The digital video processing program according to the eleventh aspect can reproduce the video accurately in the order in which the actual frames are arranged, in addition to the effect of any one of the first to tenth aspects.

  A digital video processing apparatus according to a twelfth aspect exhibits the effect according to any one of the first to eleventh aspects.

  A recording medium for a digital video processing program according to a thirteenth aspect exhibits the effect according to any one of the first to eleventh aspects.

  Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described. Throughout each embodiment, the same members, elements, or parts are denoted by the same reference numerals, and the description thereof is omitted.

Embodiment 1
1. Overview of Video Processing Program The digital video processing program according to Embodiment 1 causes a computer to execute an audio data string storage procedure and an image data string storage procedure. The digital video processing program is stored and used in an external storage unit of a computer device (not shown) having a known hardware configuration such as a CPU, ROM, RAM, bus, input / output device, external storage unit (HDD, etc.), etc. Sometimes called by the computer device to cause the computer device to execute each procedure (process) described later. Thereby, each function which a computer apparatus respond | corresponds is implement | achieved. Specifically, the digital video processing program stores, in the audio data sequence storage procedure, an audio data sequence (audio stream) composed of audio data continuously arranged in time series as audio storage in a storage means (not shown) such as a RAM. Store in the area. In addition, the digital video processing program, in the image data sequence storing procedure, has no still image data (still image data is not written) and a still frame having a predetermined spatial resolution and an empty frame having substantially no data amount. An actual frame having image data (in which still image data is written) is continuously arranged in time series to create an image data sequence having a predetermined frame rate. At this time, the digital video processing program associates the actual frame of the image data sequence with the audio data of the audio data sequence so as to synchronize at an arbitrary (optional) timing intended by the user, and stores the image in the storage means. Store in the area.

2. Data Structure FIG. 1 is an explanatory diagram showing the data structure of a video file with audio created by the digital video processing program according to Embodiment 1 of the present invention. FIG. 2 is an explanatory diagram showing a data structure of a general RIFF file format. FIG. 3 is an explanatory diagram showing the data structure of a general AVI file.

  The digital video processing program according to Embodiment 1 uses, for example, the AVI file format shown in FIG. 3 as the file format. The AVI file uses the RIFF file format shown in FIG. Therefore, the data structure of the RIFF file format and the AVI file, which are the premise of the data structure of the audio-added video file according to Embodiment 1, will be described first.

2-1. Data Structure of RIFF File First, an AVI (Audio Video Interleaved) file is a file format that can have both audio (voice) data and video (video) data. The AVI file is in a RIFF (Resource Interchange File Format) format. The RIFF format is a file format for making various resources (data) into one file, and has a structure that is compatible with resources of various formats. A typical WAVE file as audio data is also in the RIFF format. The AVI file stores data in units of streams. In the case of a video file with audio, one or more video streams (image data strings) and one or more corresponding audio streams (audio data strings) are separated into separate tracks (streams). Store. As a stream of the AVI file, there are a MIDI (Music Instrument Digital Interface) stream and a text stream in addition to the video stream and the audio stream. Here, the audio stream is composed of a WAVE file, and the video stream is composed of a collection of continuous bitmaps. The video is a collection of still images continuous in time series, and the AVI file uses bitmap data as a still image.

  First, as shown in FIG. 2, the data structure of the RIFF file is composed of a plurality of chunks (blocks) arranged hierarchically. Each chunk has an ID field (4 bytes) for identifying the type of the chunk, a field (4 bytes) indicating the size of the data part of the chunk, and a data part for storing actual data. The chunk may or may not have a sub chunk in the data part. When a chunk has sub-chunks, the highest-order chunk becomes a RIFF chunk, and a sub-chunk lower than that becomes a LIST chunk. Further, the LIST chunk may have a lower sub-chunk. The first 4 bytes of the data portion of the RIFF chunk and the LIST chunk are called a form type or list type, and are an identification code representing the contents of each chunk. The form type of the RIFF chunk contains four characters representing the data format stored in the file, such as “AVI” or “WAVE”. Similarly, the list type of the LIST chunk includes four characters indicating the contents of the list such as “hdrl” and “movi”.

2-3. AVI File Data Structure The AVI file has a multi-layered structure, and the example of the AVI file shown in FIG. 3 is one AVI file having one video stream and one audio stream. The data structure of this AVI file will be described. First, in the AVI file, the form type of the first RIFF chunk is “AVI”, and there are four sub-chunks: LIST (hdrl) chunk, JUNK sub-chunk, LIST (movi) chunk, and idX1 sub-chunk. Have The LIST (hdrl) chunk stores header information, and a number of LIST (strl) chunks corresponding to the number of avih sub-chunks and streams are consecutive. The avih sub-chunk stores information on the entire AVI file. The LIST (strl) chunk has three sub chunks: a header chunk (strh sub chunk), a format chunk (strf sub chunk), and an option data chunk (strn sub chunk) of each stream. Optional data chunks are not required. The format chunk has a WAVEFORMAT (or PCMWAVEFORMAT) structure in the case of an audio stream, and a BITMAPINFO structure in the case of a video stream. The JUNK sub-chunk is dummy data for setting the data boundary to 2048 bytes. The LIST (movi) chunk has audio data and video data. The first two characters of each sub chunk identifier indicate the stream number, and the latter two characters indicate the data type. Main data types include “db” (device-independent uncompressed bitmap (DIB)), “dc” (compressed DIB), “wb” (WAVE data), and the like. For example, “00db” represents “uncompressed DIB with stream number 00”. These data chunks may be grouped as LIST (rec) chunks. Further, in the idX1 sub chunk, an AVIINDEXENTRY structure is arranged in the reproduction order of each chunk. Chunks are played back according to this arrangement order. When the idX1 sub chunk does not exist, the chunk is played back according to the physical order (recorded order) of the chunks in the AVI file.

  Note that a video stream in a normal uncompressed AVI file is a collection of continuous bitmaps, but can be compressed by a generally available AVI compression program. In this case, various compression formats can be used, and by installing a codec (a driver that performs compression / decompression) corresponding to each compression format in the system, it is possible to perform expansion and reproduction with predetermined reproduction software. . Similarly, the audio stream of the AVI file can be compressed and expanded and played back.

2-4. 1. Data structure of audio-added video file As shown in FIG. 1, the audio-added video file 1 created by the digital video processing program according to the first embodiment is an AVI file, and its data structure includes a RIFF header 10 and a video. A stream 20 and an audio stream 30; The video stream 20 is configured by arranging the actual frames and empty frames at a predetermined frame rate (for example, 10 frames / second, 24 frames / second, 30 frames / second, etc.) in the order of frame numbers (frame numbers). . That is, the actual frame and the empty frame are sequentially arranged at a frame arrangement interval at a predetermined frame rate. Specifically, an actual frame is created by writing still image data having a predetermined spatial resolution composed of bitmap data in an arbitrary frame among frames (frames) arranged at the predetermined frame rate. Of the frames (frames) arranged at the predetermined frame rate, a frame in which the still image data is not written, that is, a frame other than an actual frame is held as the empty frame. In addition, the arrangement position of the actual frame is set as a reproduction timing position synchronized with audio data at an arbitrary position of the audio stream. That is, in an image data sequence created at a predetermined frame rate, real frames having still image data are sequentially stored in association with audio data in the audio data sequence so as to be synchronized at an arbitrary timing. When the adjacent real frames are arranged at a time interval exceeding the frame arrangement interval of the frame rate, still image data is placed at a position corresponding to the frame arrangement interval of the frame rate between the adjacent real frames. Empty frames that do not have are additionally arranged. Then, the actual frame and the empty frame are sequentially arranged at a frame arrangement interval according to the frame rate to form an image data string as a whole. Here, the storage location of each frame composed of the actual frame and the empty frame is secured in the storage means corresponding to the frame number (0 to n). In the actual frame, actual frame data composed of a set of the frame number and still image data is written in the corresponding storage location. In other words, the actual frame is stored in the storage means for the frame having a reproduction timing synchronized with the reproduction timing of any of a plurality of audio data in the audio data sequence in the frame, and the frame number and the still image data. Each set is written. In the empty frame, only the frame number is written in the corresponding storage location. That is, frames other than the actual frame among the frames are held as empty frames with no data amount to which still image data is not written.

  More specifically, the video stream 20 has the data structure in the order of frame numbers “0, 1, 2, 3... T, t + 1, t + 2, t + 3. N + 1 frames (frames) arranged in total. The frame includes the real frame and an empty frame. In FIG. 1, the actual frame is an actual frame FRt having a still image written in the frame of frame number t, an actual frame FRt + 3 having a still image (bitmap data such as DIB) written in the frame of frame number t + 3, and the like. Consists of. Also, the frame between the frame number t and the frame number t + 3, that is, the frame with the frame number t + 1 and the frame with the frame number t + 2 are an empty frame Vt + 1 and an empty frame Vt + 2 that do not have a still image. Thus, the video stream 20 arranges the real frame FR at an arbitrary position synchronized with the audio data at an arbitrary position (arbitrary reproduction timing) of the audio stream 30, and also arranges the empty frame VF in other frames. The frames composed of the actual frame FR and the empty frame VF are arranged in chronological order in the frame number order. On the other hand, the audio stream 30 has a data structure in which audio data such as WAVE is continuously arranged in time series, and has a time length corresponding to the video stream. In the audio stream 30, audio data ADt is arranged at a time position (reproduction timing position) corresponding to the actual frame FRt of the video stream 20, and at a time position (reproduction timing position) corresponding to the actual frame FRt + 3. Audio data ADt + 3 is arranged. Note that audio data is usually arranged between the audio data ADt and the audio data ADt + 3.

Here, the data structure of the video stream 20 includes, for example, still image arrangement data and image entity data. The arrangement data includes a frame number and a storage location (memory address) corresponding to the frame number. In addition, the entity data includes actual data composed of still images and a storage location (memory address) for storing the actual data. In the video stream, for example, a number of frames are stored in a predetermined storage location in the order of frame numbers as follows.
Frame number “0”: storage location “address 0”: actual data “image A” ⇒ actual frame frame number “1”: storage location “address 1”: actual data “none” ⇒ empty frame frame number “2”: storage Location “address 2”: actual data “none” ⇒ empty frame frame number “3”: storage location “address 3”: actual data “image B” ⇒ actual frame frame number “4”: storage location “address 4”: actual Data “image C” = actual frame frame number “5”: storage location “address 5”: actual data “none” = empty frame frame number “6”: storage location “address 6”: actual data “image D” = actual Frame frame number “7”: storage location “address 7”: actual data “none” ⇒ empty frame frame number “8”: storage location “address 8”: actual data “none” ⇒ empty frame frame number “9”: storage Location “Address 9”: Data "image E" ⇒ actual frame
...
...
...
Frame number “t”: storage location “address t”: actual data “image F” = actual frame RFt
Frame number “t + 1”: storage location “address t + 1”: actual data “none” ⇒ empty frame VFt + 1
Frame number “t + 2”: storage location “address t + 2”: actual data “none” ⇒ empty frame VFt + 2
Frame number “t + 3”: storage location “address t + 3”: actual data “image G” => actual frame RFt + 3
...
...
...
Frame number “n−1”: storage location “address n−1”: actual data “image Z” ⇒ actual frame frame number “n”: storage location “address n”: actual data “none” ⇒ empty frame

  Although the present invention has realized the creation of high-definition digital video mainly composed of still images using an AVI file, which is one of the moving image recording methods, an actual frame and an empty frame are arranged at arbitrary timings when the video file is created. In addition, it is possible to realize a video recording format that can be reproduced without any problem while referring to the actual frame even in the empty frame in synchronization with the audio data while arranging the empty frame. In this case, the same effect as the present invention is achieved. Is possible.

3. Processing of Digital Video Processing Program Next, the flow of processing of the digital video processing program of Embodiment 1 will be described. The digital video processing program is stored in an auxiliary storage device (HDD) of a computer device. Then, the computer apparatus appropriately reads the digital video processing program into a storage device such as a RAM, and executes the process under the control of the CPU.

4-1. Overall Processing FIG. 4 is a flowchart showing the overall processing of the digital video processing program according to Embodiment 1 of the present invention.
As shown in FIG. 4, first, when a user (user) turns on a computer device such as a personal computer and starts the digital video processing program according to the first embodiment, the digital video processing program according to the first embodiment. In STEP 100, a startup process (memory initialization, etc.) is executed, and in STEP 200, a main screen (operation screen) is displayed on a display (such as a personal computer monitor). Next, the user requests to create a new AVI file or call a saved AVI file by using a menu function on the main screen (for example, a new creation function of the “File” menu or a saved file opening function). Then, in STEP 300, the digital video processing program executes file new creation or storage file calling processing. Next, in response to the user's request, the digital video processing program executes a new creation or editing process of a video stream (image data sequence) at STEP 400, and a new creation of an audio stream (audio data sequence) at STEP 600. Or, edit processing is executed. Next, in STEP 700, the digital video processing program converts the created video stream and audio stream into an AVI file based on a user request or the like. Next, in STEP 810, the digital video processing program determines whether or not there is a request for compression processing of the AVI file created by the user. If there is a compression request, the digital video processing program selects the compression format, compression quality, etc. of the actual video stream according to the user's request in STEP 820, executes the compression process, and ends the entire process. On the other hand, if there is no request for compression processing of the AVI file from the user in STEP 810, the digital video processing program ends the entire processing. Note that the AVI file compression format conforms to the AVI file compression technology already in common use.

4-2. Video Stream Creation Processing FIG. 5 is a flowchart showing video stream creation processing of the digital video processing program according to Embodiment 1 of the present invention.
The video stream creation process of STEP 400 will be described in detail. First, for example, when a user requests to create a new file, the digital video processing program prepares an AVI file in STEP 410 as shown in FIG. 5, and a new AVI file is created in STEP 420 as shown in FIG. Prepare a video stream. Next, in STEP 430, the digital video processing program sequentially writes real frame data such as image data (BMP data) in predetermined frames in the frames of the video stream in response to a user's request. Create a video stream consisting of continuous real frames. That is, in step 430, real frame data is sequentially written in predetermined frames of a video stream (only referred to as “empty video stream” in this document) made up of frames having no real frame data prepared in STEP 420. A video stream in which empty frame data continues alternately in time series (referred to as “actual video stream” in this document) is created. Next, in step 440, the digital video processing program executes an actual frame editing process only when a user requests it. Next, in STEP 460, the digital video processing program executes the image setting change process only when the user requests it. Next, in STEP 470, the digital video processing program determines whether or not the user has requested termination of the video stream creation process. If there is a termination request, the process proceeds to STEP 480. The digital video processing program repeats the processing in STEP 430 to STEP 460 until the user issues a video stream creation processing end request in STEP 470. If there is a request to end the video stream creation process in STEP 470, the digital video processing program executes the actual video stream release process in STEP 480, and executes the release process of the AVI file in STEP 490. Exit.

4-3. Video Stream Preparation Processing FIG. 6 is a flowchart showing video stream preparation of the digital video processing program according to Embodiment 1 of the present invention.
The video stream preparation process in STEP 420 will be described in detail. As shown in FIG. 6, the digital video processing program first creates a new video stream (empty video stream) in the opened AVI file in STEP421. Next, in response to the user's request, in step 422, the digital video processing program designates a stream format including the size of a still image to be written in an actual frame, an image reproduction rate (frame rate), etc. Set the time resolution. The image size and frame rate of the stream format are used as default values. As will be described later, the image size and frame rate of the default values can be changed by a user request (editing operation). Note that, by default, the frame rate can be set to 10 frames / second or the like, and can be set to other frame rates such as 24 frames / second, 30 frames / second, etc. by an editing operation described later. The image size can be 320 × 240, 640 × 480 (VGA), 800 × 640 (SVGA), 1024 × 768 (XVGA), 1280 × 960 (UVGA), etc. In consideration of the resolution of the display that can be used, for example, it can be set to 3840 × 2880. Then, in STEP 423, the digital video processing program stores the stream format data in a memory (RAM or the like), and ends the video stream preparation process.

4-4. Real Frame Data Write Processing FIG. 7 is a flowchart showing real frame data write processing of the digital video processing program according to Embodiment 1 of the present invention.
The actual frame data writing process in STEP 430 will be described in detail. As shown in FIG. 7, in step 431, the digital video processing program first receives an actual frame addition request command from the user via the operation screen. In step 432, the reproduction timing (frame display time) of the actual frame is displayed. A screen for prompting the user to input or set is displayed on the monitor. For example, the digital video processing program displays a screen that prompts input of the actual frame playback time counted from the start of file playback, for example, “00 minutes 00 seconds 01”, “00 minutes 01 seconds 05”, and the like. When the user inputs the playback timing of the actual frame, the digital video processing program determines in STEP 433 whether or not the input playback timing is valid, proceeds to STEP 434 if it is valid, and STEP 435 if it is invalid. Display an error on the monitor. At this time, for example, the digital video processing program determines whether or not the input reproduction timing is valid based on the default frame rate set in STEP 422 of the video stream preparation process. That is, when the default frame rate is 10 frames / second, the comma second portion of the frame display time according to the frame rate (for example, the “01” portion of “00 minutes 00 seconds 01”) is “00”, “01”, “02”, “03”, “04”, “05”, “06”, “07”, “08”, “09”. Therefore, in the digital video processing program, the input value of the comma second part of the frame display time is “00”, “01”, “02”, “03”, “04”, “05”, “06”, “07”. , “08”, “09”, the input is accepted as valid, otherwise the input is rejected as invalid and an error is displayed.

  In STEP 434, the digital video processing program calculates a frame number (frame number) corresponding to the input frame display time, assigns the frame number to the actual frame (corresponding address), and writes it. For example, when the default frame rate is 10 frames / second, the frame numbers are arranged in ascending order so as to increase by “1” every 0.1 seconds. Therefore, when the input frame display time is “00:00”, the digital video processing program calculates that the frame number is “1” based on the frame rate and the frame display time, and the frame display time. Is “00:00:05”, the digital video processing program calculates that the frame number is “5” based on the frame rate and the frame display time. Next, in STEP 436, the digital video processing program determines whether or not the frame number written in the actual frame is greater than the frame number of the immediately preceding actual frame (actual frame added last time). If they are the same or smaller, an error is displayed in STEP 438. For example, when the frame number of the real frame added last time is “5”, that is, when the frame playback time is “00 minutes 00 seconds 05”, the frame number of the real frame added next is “5”. ”Or less (“ 05 ”,“ 04 ”, etc.), that is, the frame playback time is“ 00 minutes 00 seconds 05 ”or less (“ 00 minutes 00 seconds 05 ”,“ 00 minutes 00 seconds 04 ”, etc.). In this case, an error is displayed in STEP 438.

  Next, when the user designates any (desired) still image data (BMP data) to be written in the actual frame in STEP 437, the digital video processing program adds the designated image data to the new addition in STEP 439. The real frame having the frame number and the image data is created by writing in the real frame having the frame number. Next, in STEP 440, the digital video processing program determines whether or not the frame number of the actual frame is two or more larger than the frame number of the previous actual frame (actual frame added last time), that is, the frame number of the actual frame added this time. It is determined whether there is an empty frame number between the frame number of the actual frame added last time. If the actual frame number of this time is two or more larger than the frame number of the previous actual frame, the digital video processing program determines in STEP 441 that each frame number position between the two adjacent actual frames (according to the frame rate). An empty frame is added at each display time position). On the other hand, when the current frame number is larger by 1 than the frame number of the previous actual frame, that is, there is no empty frame number between the previous actual frame and the current actual frame, and they are in the frame number order. If they are lined up, the digital video processing program ends the actual frame data writing process. The empty video stream becomes an actual video stream by the actual frame data writing process.

4-5. Real Frame Editing Process FIG. 8 is a flowchart showing the real frame editing process of the digital video processing program according to Embodiment 1 of the present invention.
The actual frame editing process in STEP 450 will be described in detail. As shown in FIG. 8, the digital video processing program selects STEP 451 actual frame image data changing / deleting processing, STEP 452 actual frame image data editing processing, and STEP 453 actual frame reproduction timing changing processing, as shown in FIG. Run it. First, using the operation screen, the user selects an arbitrary (desired) actual frame from the actual video stream created in the actual frame data writing process STEP 430, and uses the image data as another image data. In step 451, the digital video processing program changes the current image data to the image data newly selected by the user for the selected actual frame. On the other hand, when the user requests to delete the image data for an arbitrary real frame, that is, to delete the real frame to be an empty frame, the digital video processing program is selected in STEP 451. The image data of the actual frame is deleted, and the frame is set as an empty frame. Further, when the user selects an arbitrary (desired) actual frame from the actual video stream and requests an editing operation of the image data using the operation screen, the digital video processing program reads STEP 451. Then, the selected real frame is edited according to the user's request. For example, the digital video processing program executes a process of deleting or correcting a part of the still image (BMP data) of the selected real frame. Further, when the user requests to change the reproduction timing of an arbitrary (desired) actual frame in the actual video stream using the operation screen, the digital video processing program is selected in STEP 453. For the actual frame, the current playback timing (frame display time) is changed to the playback timing (frame display time) newly designated by the user. At this time, the digital video processing program executes the same processing as the series of processing of STEP432 to STEP438, performs the validity judgment of the new reproduction timing, the validity judgment of the calculated frame number, and the series of STEP441 to STEP442. A process similar to the process is executed, and if necessary, an empty frame addition process is executed.

  On the other hand, the digital video processing program determines in STEP 454 whether or not the user has requested display of an actual frame image. When a user selects an arbitrary (desired) actual frame such as an actual frame changed in STEP 451 or an actual frame edited in STEP 452 and requests the image display using the operation screen, the digital video processing program In STEP 455, the image of the selected real frame is displayed on the monitor. If there is no image display request in STEP 454, the digital video processing program ends the actual frame editing process as it is.

4-6. Real Frame Setting Change Processing FIG. 9 is a flowchart showing real frame setting changing processing of the digital video processing program according to Embodiment 1 of the present invention.
The image setting change process in STEP 460 will be described in detail. When the user uses the operation screen to request a change process of the image setting (image format setting) of the actual video stream created in the actual frame data writing process STEP 430, the digital video processing program In response to the request, as shown in FIG. 9, first, a new image size setting process in STEP 461 and a new color depth setting process in STEP 462 are executed. That is, in STEP 461, the digital video processing program changes the image size portion (corresponding address) of the format information for the actual video stream from the default value (the current value if the default value has been changed) from the user. Change to the image data size specified in. Also, in STEP 462, the digital video processing program sets the color depth (number of colors per pixel) portion (corresponding address) of the format information about the actual video stream to the default value (if the default value has been changed, the current Value) to the color depth newly specified by the user. As the color depth, for example, a preset number of colors such as 8 bpp (256 colors), 16 bpp (about 30,000 colors), 24 bpp (about 17 million colors) can be used. Next, in STEP 463, the digital video processing program determines whether or not there has been a request for enlargement or reduction of the image from the user. If there is a request, in STEP 464, enlargement or compression of the image in accordance with the request of the user. Execute the process. That is, in STEP 464, the digital video processing program executes a process for enlarging or reducing the display size of the image of each real frame on the monitor or the screen when the actual video stream is reproduced. If there is no request for enlargement or reduction of the image from the user in STEP 463, the digital video processing program determines whether or not there is a request for AVI size compression processing (eg, size compression processing of the actual video stream) from the user in STEP 465. In step 466, the AVI size compression process is executed in response to the user's request, and the process proceeds to step 467. On the other hand, if the user does not request AVI size compression processing in STEP 465, the digital video processing program proceeds to STEP 467. In STEP 467, the digital video processing program stores the new image setting information in the memory, and ends the image setting changing process.

4-7. Audio Stream Creation Processing FIG. 10 is a flowchart showing audio stream creation processing of the digital video processing program according to Embodiment 1 of the present invention.
The audio stream creation process of STEP 600 will be described in detail. First, for example, when a request for creating a new file is received from the user, the digital video processing program prepares an AVI file at STEP 610 as shown in FIG. 10, and a new AVI file is added to the AVI file at STEP 620. Prepare an audio stream. Next, in STEP 630, the digital video processing program writes audio data such as a WAVE file into the audio stream in response to a user request, and creates a video stream including a continuous audio data sequence in time series. In other words, in the audio stream prepared in STEP 620 without audio data (referred to as “empty audio stream” in this document), audio data is written in time series in STEP 630, and the audio stream in which audio data is continuous in time series (this book) Create a “real audio stream” in the document. Next, in STEP 640, the digital video processing program determines whether or not the user has requested termination of the audio stream creation process. If there is an termination request, the process proceeds to STEP 650. The digital video processing program repeats the processing in STEP 630 until the user issues an audio stream creation processing end request in STEP 640. On the other hand, the digital video processing program executes the actual audio stream release process in STEP 640, and executes the AVI file release process in STEP 650, thereby ending the audio stream creation process.

4-8. Characteristics of Processing As described above, in the real frame data writing process as the image data string storage procedure, there is an empty frame that does not have still image data, that is, is not written (data amount is substantially zero), The image data sequence having a predetermined frame rate is created by continuously arranging the written real frames in time series, and the real frames of the image data sequence are converted into the audio data. The audio data in the data string are associated with each other at an arbitrary timing (at a specific timing) and stored in a predetermined storage location of the storage means. Here, the image data sequence storage procedure includes STEP 431 to STEP 439 as the actual frame arrangement procedure and STEP 440 to STEP 441 as the empty frame arrangement procedure. In the actual frame arrangement procedure, when an image data sequence is created so as to have a predetermined time length at a predetermined frame rate, an actual frame having still image data is associated with the audio data of the audio data sequence so as to be synchronized at an arbitrary timing. Store in order. Further, in the empty frame arrangement procedure, when the adjacent real frames are arranged at a time interval exceeding the frame arrangement interval of the frame rate, the position corresponding to the frame arrangement interval of the frame rate between the adjacent real frames. In addition, empty frames having no still image data are additionally arranged so that the actual frame and the empty frame are arranged in order at a frame arrangement interval according to the frame rate.

  Note that STEP 421 of the video stream preparation process in FIG. 6 is that when a plurality of frames are sequentially arranged at a predetermined frame rate to create an image data sequence, the storage location of the frame (predetermined address of the image storage area) is A storage location securing procedure secured in the storage means corresponding to the frame number is configured. Then, the image data sequence storage procedure includes the step of storing still image data in a storage location of the storage means for a frame having a reproduction timing synchronized with a reproduction timing of an arbitrary plurality of audio data in the audio data sequence. Can be defined as holding a frame other than the actual frame as an empty frame with no image data amount to which still image data is not written. Alternatively, in the image data sequence storage procedure, after a video stream creation procedure (video stream preparation process STEP420) for creating the video stream at a predetermined frame rate in the AVI file, audio at an arbitrary reproduction timing in the audio data sequence In order to make the playback timing synchronized with the data, frame data consisting of a frame number and bitmap data as a still image are sequentially written in the video stream to form a real frame, and the difference between the frame numbers of the adjacent real frames is If there are two or more, defined as having a real frame data writing procedure (actual frame data writing process STEP 430) for adding an empty frame of zero image data amount that does not write still image data between adjacent real frames You can also

  On the other hand, in the first embodiment, the audio data sequence and the image data sequence are stored in separate tracks, and the reproduction timing is adjusted individually. The digital video processing program according to Embodiment 1 causes a computer to execute a reproduction timing description procedure that describes the reproduction timing for each real frame of the image data sequence. Therefore, when the playback device plays back an AVI file created by this program, when playing back the image data sequence, if there is an empty frame between adjacent real frames of the image data sequence, the adjacent real frame After the first real frame still image is reproduced, at the reproduction timing of the empty frame, the first real frame still image arranged immediately before the empty frame is reproduced until the next real frame arrives. It is held and played.

4). GUI (user interface)
FIG. 11 is an explanatory diagram showing a state where an actual frame of a video stream is created in the GUI (Graphical User Interface) of the digital video processing program according to Embodiment 1 of the present invention.

4-1. GUI Display Areas An operation screen 100 of the digital video processing program according to the first embodiment includes, for example, a menu display area (menu bar) 110, a data content display area 120, and an image display area as shown in FIG. 130 and a GUI (graphical user interface). It should be noted that the operation screen is only an example, and it is of course possible to adopt other configurations such as adding a related editing function or devising usability. In FIG. 11, the menu display area 110 has “file menu”, “edit menu”, “setting menu”, “conversion menu”, and “help” menu as menus. The data content display area 120 lists the frame display time (reproduction timing) of the actual frame of the video stream created by the digital video processing program, the internal value corresponding to the frame number, the storage location of the BMP file as image data, and the like. indicate. Further, the image display area 130 displays an image of an actual frame selected by the user among the actual frames displayed in the data content display area 120. In FIG. 11, a plurality of real frames arranged in time series are displayed in the data content display area 120 together with their respective reproduction timings and frame numbers. In addition, the last actual frame is selected in the data content display area 120, and the image of the actual frame is displayed in the image display area 130.

4-2. File Menu FIG. 12 is an explanatory diagram showing a GUI file menu of the digital video processing program according to Embodiment 1 of the present invention.
As shown in FIG. 12, the file menu is for creating / opening / closing / saving files such as “New creation”, “Open”, “Overwrite”, “Save as”, “Exit application”, etc. These menu items can be displayed as drop-down menus, and by selecting and executing each menu item, the digital video processing program executes (implements a function) corresponding to the computer. That is, when “new creation” is selected and executed, the main processing (STEP 100 to STEP 700) in FIG. 4 is started. If “Open” is selected and executed, an existing AVI file saved in a predetermined storage location can be opened and the editing operation can be executed. If “overwrite” is selected and executed, the edited existing AVI file can be overwritten. Furthermore, when “Save As” is selected and executed, the newly created AVI file can be saved in a predetermined storage location. When “end application” is executed, the digital video processing program ends.

4-3. Edit Menu FIG. 13 is an explanatory diagram showing a GUI edit menu and frame display time setting screen of the digital video processing program according to Embodiment 1 of the present invention.
As shown in FIG. 13, the edit menu includes, for example, “Delete selected part”, “Erase all”, “Drag and drop”, “Display image”, “Change time”, “Change image”, “Add frame” Various menu items for file editing such as "" can be displayed as drop-down menus, and by selecting and executing each menu item, the digital video processing program executes (implements a function) processing corresponding to the computer. That is, when “Delete selected part” is selected and executed, the image data editing process (STEP 452) of the actual frame editing process of FIG. 8 is executed, and the part selected by the user from the images of the actual frame selected by the user. Can be deleted. Further, when “delete all” is selected and executed, the image data deletion process (STEP 451) of the actual frame editing process of FIG. 8 is executed, and the entire image of the actual frame selected by the user is deleted and the actual frame is deleted. be able to. Furthermore, “add frame” corresponds to the actual frame addition request (STEP 452) of the actual frame data writing process of FIG. 7, and when the user selects and executes “add frame”, the processing of STEP 432 is executed and the operation screen is displayed. A frame display time setting screen 111 is displayed at 100 to prompt the user to input. When the user inputs a desired value in the input fields (minute, second, comma second) of the setting screen 111 and presses the setting button, the processing of STEP 433 to STEP 441 is executed, and an actual frame is added to the video stream. . When “drag and drop” is selected and executed, an existing BMP file saved in a predetermined storage location can be dragged and dropped into the data content display area 120 (from the saving destination window) on the operation screen 100. The frame display time setting screen 111 is displayed, and the processing of STEP 433 to STEP 441 is executed in the same manner as in the case of “frame addition”, and an actual frame is added to the video stream. Furthermore, when an actual frame at a specific frame display time is selected from the actual frames displayed in the data content display area of the operation screen 100 and “time change” is selected and executed, the actual frame of the actual frame editing process of FIG. A reproduction timing change process (STEP 453) is executed, and a frame display time setting screen 111 is displayed on the operation screen 100. Therefore, the user can use the frame display time setting screen 111 to change the frame display time for the actual frame. Further, when an actual frame at a specific frame display time is selected from the actual frames displayed in the data content display area of the operation screen 100 and “change image” is selected and executed, the actual frame of the actual frame editing process in FIG. An image data change process (STEP 451) is executed. Therefore, the user can change the image data for the actual frame using the predetermined processing screen of the operation screen 110. Furthermore, when an actual frame at a specific frame display time is selected from the actual frames displayed in the data content display area of the operation screen 100 and “image display” is selected and executed, the image display of the actual frame editing process in FIG. 8 is performed. The process (STEP 455) is executed, and the image of the actual frame is displayed in the image display area 130 of the operation screen 110.

4-4. Setting Menu FIG. 14 is an explanatory diagram showing a GUI setting menu, an image setting screen, and an audio setting screen of the digital video processing program according to Embodiment 1 of the present invention.
As shown in FIG. 14, the setting menu can display various menu items such as “screen size” and “audio track” as drop-down menus, and digital video can be displayed by selecting and executing each menu item. The processing program executes (implements a function) corresponding to the computer. That is, when “screen size” is selected and executed, the setting screen 112 for image setting is displayed on the operation screen 110, and the setting screen 112 can be used to execute the image setting processing STEP461 to STEP467 of FIG. At this time, a default value (320 × 240 in FIG. 14) is displayed in the “image size” input field of the setting screen 112. Here, the user directly inputs a desired image size in the “image size” input field of the setting screen 112 or presses an input button such as “VGA” to set the corresponding image size to “image size”. In this state, when the user presses the “SET” button, the new image size setting process in STEP 461 is executed, and the old set value is changed to the new set value. . In addition, when the user selects a radio button for “image resolution” on the setting screen 112 (8 bpp, 16 bpp, and 24 bpp are illustrated in FIG. 14) and presses the “set” button, the new color depth setting process of STEP 462 is performed. Is executed, and the old setting value is changed to the new setting value. Further, when the user turns on the check box of “perform image enlargement / reduction” on the setting screen 112 and presses the “setting” button, the image enlargement / reduction process of STEP 464 is executed. Further, when the user turns on the check box of “Delete AVI size” on the setting screen 112 and presses the “Setting” button, the AVI size compression process of STEP 466 is executed. On the other hand, when “audio track” is selected and executed, a setting screen 113 for inserting an audio track is displayed on the operation screen 110, and the audio stream creation processing of FIG. 10 can be executed using this setting screen 113. That is, when the user turns on the check box of “Insert Audio Track” on the setting screen 113 and presses the “Select” button, an audio file saved in a predetermined storage location can be selected. In this state, when the user presses the “set” button, the audio stream creation process is executed in STEP 610 to STEP 660, and an audio track is added.

4-5. Conversion Menu FIG. 15 is an explanatory diagram showing a GUI conversion menu and compression setting screen of the digital video processing program according to Embodiment 1 of the present invention.
As shown in FIG. 15, the conversion menu can display various menu items such as “convert to AVI” as a drop-down menu, and by selecting and executing each menu item, the digital video processing program Execute processing (realizes function) corresponding to the computer. That is, when “Convert to AVI” is selected and executed, a setting screen 114 for compression is displayed on the operation screen 100. Here, when the user selects a predetermined compression format that is generally distributed and already installed on the setting screen 114 and presses the “OK” button, the conversion processing to AVI in FIG. 4 (STEP 700) and A compression process (STEP 820) is executed. In this state, the AVI file can be saved in a predetermined storage location using the save function of the file menu.

5. Reproduction Operation FIG. 16 is an explanatory diagram showing a reproduction operation of a video file with audio (AVI file) created by the digital video processing program according to Embodiment 1 of the present invention.
The video file with audio (AVI file) created by the digital video processing program according to the first embodiment can be played back using various playback devices corresponding to the AVI file. The reproduction operation at this time will be described. First, as shown in FIG. 16, a video stream of an audio-added video file (AVI file) created by a digital video processing program is, for example, a predetermined time so that an actual frame and an empty frame are in a predetermined order at a predetermined frame rate. They are arranged in time series at intervals of reproduction timings t0 to t9. In FIG. 15, in the video stream, real frames RFt, RFt + 3, RFt + 4, RFt + 6, and RFt + 9 are arranged at reproduction timings t0, t3, t4, t6, and t9. That is, the frame numbers corresponding to the reproduction timings t0, t3, t4, t6, and t9 are written in the actual frames RFt, RFt + 3, RFt + 4, RFt + 6, and RFt + 9. The playback timing (frame display time) is a timing set by the user so as to synchronize with the audio data at an arbitrary time position of the audio track (audio stream) as described above. Also, empty frames VFt + 1, VFt + 2, VFt + 5, VFt + 7, and VFt + 8 are arranged at reproduction timings t1, t2, t5, t7, and t8 between the real frames RFt, RFt + 3, RFt + 4, RFt + 6, and RFt + 9. That is, the frame numbers corresponding to the reproduction timings t1, t2, t5, t7, and t8 are written in the empty frames VFt + 1, VFt + 2, VFt + 5, VFt + 7, and VFt + 8. Then, the playback device plays back the actual frames and the empty frames at the predetermined frame rate (playback timing) in the order of the frame numbers. At this time, after displaying the image (BMP data) of the actual frame RFt at the reproduction timing of t0, the reproduction apparatus holds and displays the image of the actual frame RFt until the reproduction timing t3 comes. Similarly, after displaying the image of the actual frame RFt + 4 at the playback timing of t4, the playback device holds and displays the image of the actual frame RFt + 4 until the playback timing t6 arrives, and displays the image of the actual frame RFt + 6 at the playback timing of t6. After the image is displayed, the image of the actual frame RFt + 6 is held and displayed until the reproduction timing t9 comes, and the image of the actual frame RFt + 9 is displayed at the reproduction timing of t9. That is, according to the video file with audio created by the digital video processing program, if the playback device is compatible with the AVI file, the previous real frame image is held and displayed until the next new real frame image comes. To do.

  In this way, the video stream images are continuously displayed in chronological order to display moving images or slide images, and the audio stream is reproduced in synchronization with a specific reproduction timing of the images. . At this time, playback timing control is performed for each track (stream), and the playback timing can be individually adjusted for the moving image track (video stream) and the audio track (audio stream). And any audio data can be accurately synchronized.

Embodiment 2
The digital video processing program according to the second embodiment is the last configuration of the first embodiment, in the case where an empty frame exists between adjacent real frames of the image data sequence in the configuration of the first embodiment. An auxiliary frame addition procedure for creating an auxiliary frame by writing still image data of the first real frame among the adjacent real frames at the position of an empty frame immediately before the actual frame is executed by the computer. . Other configurations are the same as those in the first embodiment. As a result, when a rewinding operation, a pause operation, or the like of the image data sequence is performed in the image reproduction procedure, there is an empty frame between the adjacent real frames of the image data sequence that is the target of the rewinding operation. In this case, the still image data of the last actual frame is prevented from being adopted as still image data to be reproduced after the rewinding operation, and the still frame of the auxiliary frame (same as the still image of the first actual frame) is prevented. The image is played back. As a result, the original image (original image) can be displayed without erroneously displaying the next image after the rewinding operation or the like.

Embodiment 3
The digital video processing program according to the third embodiment has the same configuration as that of the first embodiment, and further includes all empty spaces between the adjacent real frames when there is an empty frame between adjacent real frames of the image data sequence. The computer is caused to execute a pointer information addition procedure for writing the pointer information of the first real frame among the adjacent real frames to the frame. Other configurations are the same as those in the first embodiment. Therefore, in the image reproduction procedure, when performing a rewind operation of the image data sequence, if there is an empty frame between adjacent real frames of the image data sequence that is the target of the rewind operation, Reading through the pointer information, the first real frame of the adjacent real frames is reused and reproduced. As a result, the original image (original image) can be displayed without erroneously displaying the next image after the rewinding operation or the like.

Embodiment 4
FIG. 17 is an explanatory diagram showing the data structure of a video file with audio created by the digital video processing program according to Embodiment 4 of the present invention.
As shown in FIG. 17, the audio-added video file 101 created by the digital video processing program according to the fourth embodiment is an AVI file, and its data structure includes RIFF header 110 and a plurality (five in FIG. 17). Video streams 121, 122, 123, 124, 125 and corresponding (five in FIG. 17) multiple audio streams 131, 132, 133, 134, 135. Each video stream 121, 122, 123, 124, 125 has the same structure as the video stream 20 of the first embodiment. Also, each audio stream 131, 132, 133, 134, 135 has the same structure as the audio stream 30 of the first embodiment. More specifically, the video streams 121, 122, 123, 124, 125 have frame numbers “0, 1, 2, 3... T, t + 1, t + 2, t + 3. It has a total of n + 1 frames (frames) arranged in the order of (n-1, n). The frame includes the real frame and an empty frame. In FIG. 17, the actual frame is an actual frame FRt having a still image written in the frame of frame number t, an actual frame FRt + 3 having a still image (bitmap data such as DIB) written in the frame of frame number t + 3, and the like. Consists of. Also, the frame between the frame number t and the frame number t + 3, that is, the frame with the frame number t + 1 and the frame with the frame number t + 2 are an empty frame Vt + 1 and an empty frame Vt + 2 that do not have a still image. In this way, the video streams 121, 122, 123, 124, and 125 are at arbitrary positions synchronized with audio data at arbitrary positions (arbitrary reproduction timings) of the corresponding audio streams 131, 132, 133, 134, and 135. The real frame FR is arranged, and the empty frame VF is arranged in other frames, and the frames including the real frame FR and the empty frame VF are arranged in time series in the order of the frame numbers. On the other hand, the audio streams 131, 132, 133, 134, and 135 have audio data such as WAVE continuously arranged in time series as their data structures, and the corresponding video streams 121, 122, 123, 124, It has a time length corresponding to 125. In the audio streams 131, 132, 133, 134, and 135, audio data ADt is arranged at a time position (reproduction timing position) corresponding to the actual frame FRt of the video streams 121, 122, 123, 124, and 125. The audio data ADt + 3 is arranged at the time position (reproduction timing position) corresponding to the actual frame FRt + 3. Note that audio data is usually arranged between the audio data ADt and the audio data ADt + 3.

Effects According to each of the above embodiments, first, it is possible to effectively use the photographing data of a widespread high-quality digital camera. In other words, there are currently 2 million pixels (1600 pixels × 1200 lines), 3.2 million pixels (2048 × 1536), 4 million pixels (2272 × 1704) digital cameras as popular price ranges. A digital camera with 63,000,000 pixels (3072 × 2048) has been sold at a price range of 100,000 yen. This resolution of about 3072 × 2048 is close to the resolution of 35 mm film. According to the digital video reproduction program of the above-described embodiment, these high-resolution still images can be written in an actual frame, and further, data reproduction is considerably prepared as compared with a normal recording format. Even in this case, by arranging an appropriate number of empty frames between the actual frames, the entire data amount can be greatly reduced while the image is accurately synchronized with the sound. That is, according to each of the above-described embodiments, the information amount of a still image film such as a 35 mm film can be utilized as it is, and it can be visualized and digitalized with audio explanation without downsizing much of the information that the film has. As a result, it is possible to improve the video effect in the screening and the like, and to improve the explanation, the narrative, the recordability, etc. by the audio explanation / music to the still image. Further, according to each of the above embodiments, an image can be maintained at an original high quality without compressing video data, and deterioration due to compression can be reduced. As a result, it is possible to positively utilize negative films and positive films that are held, photographed and stored by research and government institutions, companies, households, individuals, etc., which are human intellectual assets.

  Here, an image mainly composed of a still image has an advantage that an impression or emotion received from a photograph can be more effectively evoked and the persuasiveness of the photograph can be promoted by associating the sound with the photograph. Therefore, according to the digital video processing program of the above-described embodiment, when creating a “slide show” using still images, it is possible to synchronize audio and image frames at arbitrary timings effectively. You can give a presentation. In addition, as described above, a digital camera image or the like that has already achieved a considerably high image quality can be used as a video video. Furthermore, according to the digital video processing program of the above-described embodiment, high-quality video can be realized on a personal computer monitor that is already widely used in current homes and workplaces. In other words, while the personal computer monitor and its accessory devices are optimal for displaying still images, it will be a matter of time before commercialization of liquid crystal panels with higher resolution (about 3000 × 2000) is realized by future technological advances. It is a problem, and it seems that an environment in which a high-quality video created by the digital video processing program of the above-described embodiment can be reproduced without any problems will be established at an early stage. Note that such high-quality monitors have already been realized for some business use, and for medical use, a 22.2 type monitor (3880 × 2400) having 9.2 million pixels is sold at an actual selling price of 900,000. In addition, personal computer monitors with a resolution of 1024 × 768 class are already in widespread use, and monitors with higher resolutions (such as 1600 × 1200) are also being commercialized one after another.

  In a general video recording method (processing by a recording program), frames are arranged continuously without gaps from the beginning to the end of the video (without interposing empty frames as in the above embodiment), so a large number of frames are arranged. Need to. Therefore, the amount of data becomes large and handling becomes heavy, and at present, various problems arise in creating and creating a moving picture in excess of the DVD standard. Conventional slide show creation software that is distributed is a method that assumes existing moving image editing. Finally, frames are continuously arranged without gaps, and the amount of files cannot be reduced. That is, even if the appearance is still, the same frame is continuously arranged at the frame rate arrangement interval, and data is wasted. On the other hand, according to the digital video processing program according to the above-described embodiment, it is possible to place the actual frame apart and synchronize with the desired sound at an arbitrary reproduction timing set by the user. Therefore, a video (for example, a slide show) created by the digital video processing program according to the above embodiment has a small data amount and is lightly handled. Also, since the AVI file system that is generally widely used as the video recording system is used, it is easy to use. Unlike the general moving image format in which about 30 frames or 24 frames are displayed one by one in one second, and the motion of the image is smoothed by the afterimage phenomenon of the frames, the digital video processing program according to the above embodiment Since the arrangement of the actual frame is a more sparse reproduction timing, the image is displayed as a still image, and the transition of the actual frame (image) is simply displayed so as to change to the next frame in a short time. However, a part of the video created by the digital video processing program according to the above embodiment is 10 frames per second, 30 frames, 20 frames, etc. at a certain limit, generally at the same frame rate as a moving image. It is also possible to insert a moving image portion where an actual frame is arranged. This part is exactly the same as a general moving image.

FIG. 1 is an explanatory diagram showing the data structure of a video file with audio created by the digital video processing program according to Embodiment 1 of the present invention. FIG. 2 is an explanatory diagram showing a data structure of a general RIFF file format. FIG. 3 is an explanatory diagram showing the data structure of a general AVI file. FIG. 4 is a flowchart showing the overall processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 5 is a flowchart showing video stream creation processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 6 is a flowchart showing video stream preparation of the digital video processing program according to Embodiment 1 of the present invention. FIG. 7 is a flowchart showing real frame data writing processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 8 is a flowchart showing real frame editing processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 9 is a flowchart showing actual frame setting change processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 10 is a flowchart showing audio stream creation processing of the digital video processing program according to Embodiment 1 of the present invention. FIG. 11 is an explanatory diagram showing a state where an actual frame of a video stream is created in the GUI (Graphical User Interface) of the digital video processing program according to Embodiment 1 of the present invention. FIG. 12 is an explanatory diagram showing a GUI file menu of the digital video processing program according to the first embodiment of the present invention. FIG. 13 is an explanatory diagram showing a GUI edit menu and frame display time setting screen of the digital video processing program according to Embodiment 1 of the present invention. FIG. 14 is an explanatory diagram showing a GUI setting menu, an image setting screen, and an audio setting screen of the digital video processing program according to Embodiment 1 of the present invention. FIG. 15 is an explanatory diagram showing a GUI conversion menu and compression setting screen of the digital video processing program according to Embodiment 1 of the present invention. FIG. 16 is an explanatory diagram showing a reproduction operation of an audio-added video file (AVI file) created by the digital video processing program according to Embodiment 1 of the present invention. FIG. 17 is an explanatory diagram showing the data structure of a video file with audio created by the digital video processing program according to Embodiment 4 of the present invention.

Explanation of symbols

RFt, RFt + 3: Real frame, VFt + 1, VFt + 2: Empty frame ADt, ADt + 3: Audio data

Claims (13)

A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series;
B) An empty frame having no still image data and an actual frame having still image data are continuously arranged in time series to create an image data sequence of a predetermined frame rate, and the actual frames of the image data sequence are A digital video processing program that causes a computer to execute an image data string storage procedure that is stored in association with the audio data of the audio data string so as to be synchronized with each other at an arbitrary timing. A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series;
B) causing a computer to execute an image data sequence storage procedure for creating an image data sequence at a predetermined frame rate;
The image data string storage procedure includes:
A real frame arrangement procedure for sequentially storing real frames having still image data in association with audio data in the audio data sequence so as to be synchronized at an arbitrary timing;
C) When the adjacent real frames are arranged at a time interval exceeding the frame arrangement interval of the frame rate, still image data is placed at a position corresponding to the frame arrangement interval of the frame rate between the adjacent real frames. A digital video processing program comprising: an empty frame arrangement procedure in which empty frames that are not included are additionally arranged so that the real frame and the empty frame are arranged in order at a frame arrangement interval according to the frame rate. A) an audio data sequence storing procedure for storing an audio data sequence composed of audio data continuously arranged in time series;
B) A storage location securing procedure for securing the storage location of the frame in the storage means in the case of creating an image data sequence by sequentially arranging a plurality of frames at a predetermined frame rate;
C) Among the frames, still image data is respectively written into storage locations of the storage means for frames having reproduction timings synchronized with any of a plurality of reproduction timings in the audio data sequence to form real frames, and A digital video processing program for causing a computer to execute an image data sequence storage procedure for holding a frame other than the actual frame among frames as an empty frame in which still image data is not written. A digital video processing program for creating digital video with audio in an AVI file format,
A) an audio data string storage procedure for storing audio data continuously arranged in time series in the audio stream of the AVI file;
B) causing a computer to execute an image data string storage procedure for storing image data continuously arranged in time series in a video stream of an AVI file,
The image data string storage procedure includes:
B1) A video stream preparation procedure for preparing the video stream at a predetermined frame rate in the AVI file;
B2) Frame data composed of a frame number and bitmap data as a still image are sequentially written in the video stream so as to be a reproduction timing synchronized with the audio data of an arbitrary reproduction timing in the audio data string to obtain an actual frame. And an actual frame data writing procedure for adding an empty frame in which still image data is not written between the adjacent actual frames when there is a difference of two or more frame numbers between the adjacent actual frames. Digital video processing program.   The AVI file is composed of the video stream composed of compressed image data compressed using an AVI file compression program and the audio stream composed of compressed audio data compressed using an AVI file compression program. 5. The digital video processing program according to claim 4, wherein   5. The digital video processing program according to claim 1, wherein the still image is composed of uncompressed bitmap data.   7. The digital video processing according to claim 1, wherein the audio data sequence and the image data sequence are stored in separate tracks and the reproduction timing is individually adjusted. program.   Further, when an empty frame exists between adjacent real frames of the image data sequence, the first of the adjacent real frames is positioned at the position of the empty frame immediately before the last real frame of the adjacent real frames. 8. The digital video processing program according to claim 1, further comprising: causing a computer to execute an auxiliary frame addition procedure for writing the still image data of the real frame to create an auxiliary frame.   Further, when there is an empty frame between adjacent real frames of the image data sequence, pointer information of the first real frame of the adjacent real frames is written in all empty frames between the adjacent real frames. 8. The digital video processing program according to claim 1, wherein the computer executes a pointer information adding procedure.   10. The digital video processing program according to claim 1, further causing a computer to execute a reproduction timing description procedure for describing a reproduction timing for each real frame of the image data sequence.   When there is an empty frame between adjacent real frames of the image data sequence, after the still image of the first real frame of the adjacent real frames is reproduced, the first frame until the next real frame arrives. The digital video processing program according to any one of claims 1 to 10, wherein a still image of a real frame is held and reproduced.   A digital video processing apparatus comprising a computer storing the digital video processing program according to any one of claims 1 to 11.   12. A computer-readable recording medium for a digital video processing program, wherein the digital video processing program according to any one of claims 1 to 11 is recorded.

Images