CN1383543A - Karaoka system - Google Patents

Abstract

The concept of singing karaoke exclusively relies on audio based techniques. The invention consists of developing the concept of video immersion: the user will see his (her) image inserted into the video clip or movie at the place of his (her) favorite dancer, singer or player, and will therefore be able to play clip/song on video tape, and replace any famous star. More precisely, the invention relates to a karaoke system in which are provided successive means for picking up an image of the user and his/her voice, analyzing and processing the obtained signals, mixing the audio/video signals thus analyzed and processed and a pre-recorded material and displaying the combination signals thus obtained.

Description

A karaoke system

The present invention relates to a karaoke system for singing in a sequence such as a video clip or film.

In a karaoke system as described eg in European patent application EP0782338, music, lyrics or any kind of audio data is transmitted from a transmission station to a distribution station. The music control unit of the main module of the system broadcasts music through the built-in speaker of the monitor, and broadcasts sound from a microphone not shown through said speaker. The image control means displays a background image (such as a video image or a still image extracted from the background image storage means) on the monitor, and the lyrics control means displays the lyrics by superimposing the lyrics on the background image. Image pick-up equipment, such as a CCD camera, picks up the image of the singer, and it is superimposed on the monitor screen as the superimposed image by the video image control device. Such a system can be defined as the so-called "video mixing" in the karaoke concept.

The purpose of the present invention is to propose another type of karaoke system with additional functions.

For this purpose, the system relates to a karaoke system for singing during a sequence such as a video clip or film, and includes a series of pickup devices for picking up the user's image and sound, a an analysis and processing device with at least a portion of the user separated from the background, a mixing and rendering device for combining the output signal of said analysis and processing device with pre-recorded material, and a device for displaying the combined Signal display device.

As of today, the concept of singing karaoke relies only on audio-based technology, which provides limited functionality and makes it impossible to actually insert the user into the video virtual world. The proposed solution, which introduces the concept of video mixing, allows the extension of this karaoke concept to video and, in general, the development of a full audio-video insertion concept: according to said concept, in the song The voices and faces in the video clip can be replaced by those of a fortuitous singer (hereafter also referred to as a user because he or she can in fact be a singer, performer, dancer, etc. . . . ). The same proposed technology may find similar applications in other contexts, such as the field of e-commerce or video editing for pre-recorded content.

The present invention will be described by way of example below with reference to the accompanying drawings, in which:

Figure 1: Block diagram of a karaoke system according to the present invention.

Figure 2: Another embodiment of the karaoke system according to the invention.

As shown in FIG. 1 , the different subsystems necessary to realize the karaoke system according to the invention are mainly an analysis and processing device 11 and a mixing and reproduction device 12 .

The analysis and processing device 11 for receiving images and sounds of the user (person shown in black) picked up by the pickup device 10 includes a segmentation circuit for separating, for example, the user's face from the background, thereby defining An alpha plane (if the user is placed on stage, such a circuit could be based eg on a chroma key technique in front of a blue screen). Mixing and rendering device 12 is a circuit that uses the shape information analyzed in device 11 to synthesize the user with a pre-recorded video or audio-video background delivered by medium 13 (the said pre-recorded material is shown on the left side of the medium). out). This synthesis completes the audio synthesis for mixing the user's voice with the pre-recorded musical background from the song. Then, using the alpha plane defined by the device 11, the two sources are easily combined according to a relationship of the following type: [(Video 1 x Alpha)+(Video 2 x (255-Alpha))]/255= final video. Finally, a display device 14, such as a monitor, is used to finally display the final result (ie the combination of pre-recorded material and specific user-specific).

Obviously, for improved quality, the analysis done in the device 11 can produce 8-bit alpha planes that allow for better blending at the fronteer of veneered objects. Additionally, it is noted that the system may replace only the user's head or his or her entire body.

Different situations can be considered with respect to the type of audio-visual source:

(a) Both audio/video sources are not compressed: this option can be used e.g. in karaoke restaurants, where the singer's full body is framed in the clip/film (pre-recorded data can be stored on tape and occasionally Singer videos can be analyzed and transferred directly to the video mixer);

(b) One or both sources are compressed: an adapted framework for this situation is the newly developed MPEG-4 standard, which enables object shapes and alpha planes—here the face of the casual user Encoding - (The MPEG-4 standard has defined an overall system architecture that enables the synthesis of audio and video objects).

Different cases of application of the invention can also be considered:

(a) the user may wish to record the results of the mixing operation, which is illustrated in Figure 2, which shows a system similar to the embodiment of Figure 1, but including an additional recording device 25;

(b) In some cases, karaoke systems may work online: then pre-recorded clips may be stored on a database (e.g., the Internet), and the user records his or her performance at home and intends to generate a combination of karaoke clips and Put it on his or her personal home page (the use of compression techniques is especially useful in this case, and more generally in all applications that operate in bandwidth-constrained environments);

(c) In addition, in some cases, the user may intend to only place his or her head in the position of the original singer's head, which includes further processing in the mixing and reproduction device 12, because the user's head The position of the song needs to match the orientation and posture of the original singer's body.

Claims (1)

1. A kind of karaoke system, it is used for singing in a sequence process as video clip or film, and comprises a series of pick-up equipment that is used to pick up user's image and sound, one is used for at least An analyzing and processing device separated from the background, a mixing and reproducing device for combining the output signal of said analyzing and processing device with pre-recorded material, and a display device for displaying the combined signal.

Images