KR102757579B1 - Appratus and method for digital media watermarking using frame data - Google Patents

Abstract

The present invention relates to a device and method for inserting a watermark to hide information in digital media, and more particularly, to a device and method for hiding information using a sequence of watermarks inserted in units of frames. According to one aspect of the present invention for solving the above-described problem, a watermark insertion device includes a frame data generation unit for generating frame data, an insertion symbol generation unit for generating an insertion symbol, a watermark information generation unit for generating watermark information based on the insertion symbol, and a watermark insertion unit for inserting a watermark into the frame data based on the watermark information to generate watermarked frame data, wherein the insertion symbol belongs to a set including at least a first symbol and a second symbol, and the watermark information can be generated differently depending on the symbol of the insertion symbol.

Description

{APPRATUS AND METHOD FOR DIGITAL MEDIA WATERMARKING USING FRAME DATA}

The present invention relates to a device and method for inserting a watermark to hide information in digital media, and more specifically, to a device and method for hiding information using a sequence of watermarks inserted frame by frame.

Recently, almost all tasks and services in daily life, such as office work, payment services, certificate issuance, banking, and e-commerce, are being conducted through online and computerized environments. In such online and computerized environments, digital media becomes the object of information creation, viewing, storage, editing, and transmission. Here, digital media can mean various forms of digital information including images, texts, and multimedia, such as digital documents, images, voices, videos, and 3D data.

However, since these digital media can be easily copied or transmitted without losing the information due to the nature of digital information, they are highly vulnerable to leakage through illegal acts. Therefore, in order to provide a sound digital content usage environment, a means of protection that can prevent digital content from illegal acts, that is, a security system, is essential. For example, digital media can be protected through encryption and decryption systems, watermark or fingerprinting systems, and screen capture prevention systems.

One such method is digital watermarking. Many commonly used digital watermarking methods involve embedding some additional information into digital media by performing a machine readable manipulation at a specific location in time or space. The embedded additional information can be detected by analyzing the watermarked digital media by a device or software configured to interpret the manipulation.

The above additional information may be information indicating ownership of the digital media, or may be information indicating that copying or viewing is restricted. Such additional information may be inserted into the digital media in an invisible form, particularly by performing manipulations that are not recognizable to the viewer of the digital media but can be read by a machine.

In addition, forensic watermarking methods that insert traceability information as additional information into leaked images using digital watermarking are being used recently. If the recipient information of the digital media is inserted into the digital media and distributed, it can be helpful in identifying individuals who use the digital media in an unauthorized manner or participate in illegal leaks of the digital media.

In order to insert forensic watermarking, a unique watermarked digital media with information of the recipient inserted is required each time an original digital media is supplied to a specific recipient. Since digital media is supplied to the public, separate watermarked digital media cannot be prepared for all recipients. Therefore, it is desirable to implement real-time forensic watermarking by inserting a digital watermark in real time during the process of the recipient viewing the digital media. However, since inserting a digital watermark requires considerable computational resources, it can place a considerable burden on the supplier or recipient of the digital media.

Accordingly, conventionally, for real-time forensic watermarking of digital media, a method of inserting traceability information in real time by combining different copies by transmission unit (e.g., GOP (Group of Frame), packet, or chunk) using copies of digital media that have been watermarked in advance in several types has been used. The above-described method enables only the supplier of digital media to easily implement real-time forensic watermarking by consuming minimal additional computational resources.

Each of the above-mentioned transmission units consists of a considerable number of frames, and thus has a certain length in terms of time (quantity). Therefore, when using the conventional real-time forensic watermarking method described above, there is a problem that a considerably long amount of digital media must be obtained in order to detect the information inserted through the watermark from the combination method. In addition, there is a problem that it is vulnerable to a collusion attack by merging copies of different digital media based on the above-mentioned transmission unit.

According to one aspect of the present invention for solving the above-described problem, a watermark insertion device includes a frame data generation unit for generating frame data, an insertion symbol generation unit for generating an insertion symbol, a watermark information generation unit for generating watermark information based on the insertion symbol, and a watermark insertion unit for generating watermarked frame data by inserting a watermark into the frame data based on the watermark information, wherein the insertion symbol belongs to a set including at least a first symbol and a second symbol, and the watermark information can be generated differently depending on a symbol of the insertion symbol.

The above watermark insertion device further includes a media acquisition unit that reads digital media including at least one frame data, and an insertion sequence acquisition unit that reads an insertion sequence including at least one insertion symbol, wherein the frame data generation unit is configured to sequentially read one frame data from the digital media to generate frame data, and the insertion symbol generation unit can be configured to sequentially read one insertion symbol from the insertion sequence to generate an insertion symbol.

The above insertion symbol sequence may include at least one of information related to contents represented by the frame data, identification information related to a viewing device on which the watermarking frame data is displayed or received, identification information related to a user of the viewing device, and information related to a connection for communicating with the viewing device.

The above watermark information may include at least one of information related to the embedding power of the watermark or information related to the embedding method of the watermark.

The frame data generating unit further includes a frame separating unit that divides the frame data into two or more regions, and the watermark information generating unit further includes an area watermark information generating unit that generates area watermark information for at least one of the two or more regions, and the watermark information can be generated based on the at least one area watermark information.

The above-mentioned area watermark information may include at least one of information related to the embedding power of the watermark or information related to the embedding method of the watermark.

The above-described insertion symbol generation unit is configured to generate a plurality of insertion symbols including at least a first symbol and a second symbol, and the area watermark information generation unit can generate the area watermark information by using the first symbol as an insertion symbol in a first area among the two or more areas and using the second symbol as an insertion symbol in a second area.

The watermark insertion device further includes a memory that stores at least one frame data or at least one watermark frame data as previous frame data, and the watermark information generation unit further includes a suitability evaluation unit that analyzes the frame data and at least one of the previous frame data to evaluate watermark insertion suitability for the frame data, and the watermark information can be generated by including information for not inserting a watermark if the suitability falls below a threshold value.

The above suitability evaluation unit may be configured to determine the suitability based on at least one of the continuity of the frame data, brightness change, saturation change of the frame data, motion vector size of the frame data, scene transition included in the frame data, and type of frame included in the frame data whether it is a key frame (e.g., an I frame) or a reference frame (e.g., a P or B frame).

According to one aspect of the present invention for solving the above-described problem, a watermark detection device includes a frame data generation unit that generates watermarked frame data, a watermark detection unit that detects a watermark from the watermarked frame data and generates watermark information based on the detected watermark, and a symbol detection unit that generates a detection symbol based on the watermark information, wherein the detection symbol may belong to a set including at least a first symbol and a second symbol based on the detected watermark information.

The above watermark information may include at least one of information related to the power of the detected watermark or information related to the method of detecting the watermark.

The above frame data generating unit further includes a frame separating unit that divides the watermarked frame data into two or more regions, and the watermark detection unit further includes a region watermark detection unit that detects a region watermark for at least one of the regions and generates region watermark information based on the detected region watermark, and the watermark information can be generated based on the at least one region watermark information.

The above-detected area watermark information may include at least one of information related to the power of the detected area watermark or information related to the method of the detected area watermark.

The above watermark detection device may further include a sequence restoration unit that generates a detection sequence based on at least one of the detection symbols.

The above detection sequence may include at least one of information related to contents represented by the frame data, identification information related to a viewing device on which the watermarking frame data is displayed or received, identification information related to a user of the viewing device, and information related to a connection for communicating with the viewing device.

According to one aspect of the present invention for solving the above-described problem, a watermark insertion method includes a step of generating frame data, a step of generating an insertion symbol, a step of generating watermark information based on the insertion symbol, and a step of inserting a watermark into the frame data based on the watermark information to generate watermarked frame data, wherein the insertion symbol may belong to a set including at least a first symbol and a second symbol, and the watermark information may be generated differently depending on a symbol of the insertion symbol.

The above watermark information may include at least one of information related to the embedding power of the watermark or information related to the embedding method of the watermark.

According to one aspect of the present invention for solving the above-described problem, a watermark detection method includes a step of generating watermarked frame data, a step of detecting a watermark from the watermark frame data and generating watermark information based on the detected watermark, and a step of generating a detection symbol based on the watermark information, wherein the detection symbol may be a method belonging to a set including at least a first symbol and a second symbol based on the detected watermark information.

The above watermark information may include at least one of information related to the power of the detected watermark or information related to the method of detecting the watermark.

According to one aspect of the present invention for solving the above-described problem, a watermark insertion system may include a media acquisition module for reading digital media including at least one frame data, an insertion sequence acquisition module for reading an insertion sequence including at least one insertion symbol, a watermark insertion module for sequentially reading one frame data from the digital image, sequentially reading one insertion symbol from the insertion sequence, generating watermark information based on the insertion symbol, and inserting a watermark into the frame data based on the watermark information to generate watermark frame data, and a media reconstruction module for sequentially combining the watermark frame data to output a watermarked digital media, wherein the insertion symbol belongs to a set including at least a first symbol and a second symbol, and the watermark information may be a system generated differently depending on the insertion symbol.

According to the present invention, watermarking can be performed on a frame-by-frame basis, making it possible to extract a watermark from a shorter section of digital media, and since it can be easily combined with other watermarking methods, it has the advantageous effect of forming part of a robust watermarking system by composite watermarking.

Figure 1 is a basic conceptual diagram of a digital media distribution system configured by the present invention.
Figure 2 is a basic conceptual diagram of a method for generating a watermark information sequence based on a frame sequence of digital media and a sequence of information to be inserted in the present invention.
Figure 3 is a basic conceptual diagram of a method for watermarking digital media using a watermark information sequence in the present invention.
Figure 4 is a basic conceptual diagram of a process for detecting information inserted from watermarked digital media in the present invention.
Figure 5 is a block diagram of one embodiment of a watermark insertion device according to the present invention.
Figure 6 is a block diagram of one embodiment of a watermark detection device according to the present invention.
Figure 7 is a conceptual diagram of frame data in which region division has been performed.
FIG. 8 is a block diagram of an embodiment of a watermark insertion device according to the present invention that further includes a region division function of frame data.
FIG. 9 is an example diagram of a method for inserting a watermark based on region division in a modified embodiment of a watermark insertion device according to the present invention.
Figure 10 is a block diagram of an embodiment of a watermark insertion device according to the present invention that utilizes pre-processed digital media, and
FIG. 11 is an exemplary diagram showing a method of inserting a watermark using a frame selection unit in a modified embodiment of a watermark insertion device according to the present invention.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Although the terms first, second, etc. may be used to describe various components, the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, the first component could be referred to as the second component, and similarly, the second component could also be referred to as the first component, without departing from the scope of the present invention. The term "and/or" includes any combination of a plurality of related listed items or any of a plurality of related listed items, and is non-exclusive unless otherwise indicated. The listing of items in this application is merely an exemplary description to easily explain the spirit and possible implementation methods of the present invention, and therefore, is not intended to limit the scope of embodiments of the present invention.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries, such as those defined in common dictionaries, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

In describing the invention in this application, the embodiments may be described or illustrated in terms of unit blocks that perform the described function or functions. The blocks may be expressed as one or more devices, units, modules, parts, etc. in this application. The blocks may be implemented in hardware by one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods that are not limited thereto. Alternatively, the blocks may be implemented in software by application software, operating system software, firmware, or information processing software implementation methods that are not limited thereto. One block may be implemented by being separated into multiple blocks that perform the same function, or conversely, one block may be implemented to perform the functions of multiple blocks simultaneously. The blocks may also be implemented by being physically separated or combined by any criterion. The blocks may be implemented to operate in an environment where their physical locations are not specified and are separated from each other by a communication network, the Internet, a cloud service, or a communication method that is not limited thereto. All of the above implementation methods are within the scope of various embodiments that can be taken by a person skilled in the field of information and communication technology to implement the same technical idea, and therefore, any detailed implementation method should be interpreted as being included within the scope of the technical idea of the invention of the present application.

Hereinafter, with reference to the attached drawings, a preferred embodiment of the present invention will be described in more detail. In order to facilitate an overall understanding in describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted. In addition, it is assumed that the multiple embodiments are not mutually exclusive, and that some embodiments can be combined with one or more other embodiments to form new embodiments.

Basic concept of the present invention

Figure 1 is a basic conceptual diagram of a digital media distribution system constructed by the present invention.

The digital media source (110) is an initial supply source that supplies digital media (115), and depending on the embodiment, may be a device or system that reads and provides digital media from a recording medium on which digital media is recorded, or a device or system that reads and provides a recording medium on which non-digital media is recorded in the form of digital media, or a device or system that provides information acquired in real time in the form of digital media. In another embodiment of the present invention, the digital media may be a device or system that provides digital media to which at least one information protection means, such as a watermark or encryption, has already been applied.

In the present invention, the digital media (115) is considered as information requiring protection. In one embodiment of the present invention, the digital media (115) may be information that may be exposed to illegal use or distribution, such as unauthorized duplication, if distributed to the public. The present invention allows the digital media (115) to pass through a watermark insertion module (120) before being distributed, thereby inserting (i.e., hiding) predetermined information in the form of a watermark into the digital media (115), and using this, creating a first watermarked digital media (125) and distributing it to a first viewing device (130).

The above-mentioned predetermined information may include information related to the title, length, type, language, subtitle, copyright holder, distributor, supply channel, and viewing age restriction of the content included in the digital media (115). Alternatively, the above-mentioned predetermined information may include information for maintaining the security of the digital media (115). Specifically, the information may include copy permission information, copy protection information, and traceability information. Alternatively, the above-mentioned predetermined information may include information for controlling a device for receiving and viewing the first watermarked digital media (125), such as the first viewing device (130). Specifically, the information may include metadata for additional services or terminal control commands that the first viewing device (130) can recognize as a signal.

The above-mentioned predetermined information may be information inserted by real-time watermark insertion by the watermark insertion module (120), and may be traceability information that allows the first watermarked digital media (125) distributed to the first viewing device (130) as a copy of the digital media (115) to be identified and distinguished from other copies, such as the second watermarked digital media (125a) distributed to the second viewing device (130a).

The above traceability information may include identification information related to the first viewing device (130). As a specific example, it may include the type of device hardware of the first viewing device (130), an identifier of the device hardware (e.g., device UUID), an identifier of device software, a version of device software, or other information necessary for identifying the first viewing device (130).

In addition, the traceability information may include at least one of information related to a means for distributing the watermarked digital media (125) from the watermark insertion module (120) to the first viewing device (130). The distribution means may use a conventional wired or wireless digital communication network, or may use a hard-wired wiring between the watermark insertion module (120) and the first viewing device (130), or may use a physical recording medium including a hard disk, an optical medium, or a flash memory. Accordingly, the traceability information may include a communication network address, a network interface identifier, or network path information of the first viewing device (130) when the distribution means is a communication network, may include identification information of the wiring path when the distribution means is a fixed wiring, and may include a unique identifier for identifying a copy of a specific physical recording medium when the distribution means is a physical recording medium.

In addition, the first device information (135) may include identification information related to the user (150) of the viewing device. As a specific example, the first device information (135) may include user identification information (e.g., service ID) of the user (150), personal identification information (e.g., name, identification information, etc.), details of the usage fee paid by the user (150) to receive the watermarked digital media (125), or financial transaction information performed by the user (150) to pay the usage fee, or other information necessary for identifying the user (150).

In order to generate the above-mentioned traceability information, the watermark insertion module (120) or the distribution side system including it may be configured to receive information related to identification from the first viewing device (130) in the form of first device information (135). For example, the first viewing device (130) may provide its own communication network address and a unique identifier of the user (150) as the first device information (135) to the watermark insertion module (120) or the distribution side system including it, and the watermark insertion module (120) may be configured to distribute the first watermarked digital media (125) having the traceability information including the first device information (135) inserted therein to the viewing device (130). Thereafter, the first watermarked digital media may be provided (137) in a form that can be viewed by the user (150).

The operation of the watermark insertion module (120) described above may be commonly applied to each viewing device that wishes to view a copy of the digital media (115). For example, the traceability information for the first viewing device (130) and the traceability information for other viewing devices (130a) such as the second viewing device may each be uniquely distinguished information, and thus, the first watermarked digital media (125) distributed to the first viewing device (130) and the second watermarked digital media (125a) distributed to the second viewing device, etc. (130a) may both include different traceability information inserted by the same watermark method. Similarly, the second viewing device, etc. (130a) may be configured to provide the second device information, etc. (135a) to the watermark insertion module (120) or a distribution system including it.

Meanwhile, the digital media distributed by the watermark insertion module (120) may be subject to information detection by the watermark detection module (140) if necessary. The purpose or reason for the detection may be anything, but for example, if the first watermarked digital media (125) with the traceability information inserted therein is copied and distributed without authorization by an illegal act of the user (150), the detection may be performed for the purpose of determining whether the user (150) is responsible for it by making the copied copy the target of the detection.

The above watermark detection module (140) may detect the first watermarked digital media (115) distributed from the watermark insertion module (120) to the first viewing device (130), or may detect the result provided (137) by the first viewing device (130) to the user in a viewable form.

For example, the watermark detection module (140) may operate by obtaining the first watermarked digital media (125) in the form of a media file. Or, it may operate by obtaining the first watermarked digital media (125) over a communication network. Or, for example, if the first watermarked digital media (125) includes digital image content, it may operate by recording the result displayed (137) in the form of an image to the user (150). In addition, any method may be used as long as it can directly or indirectly obtain the digital media (125) including the first watermarked digital media (115) and collect information that satisfies the quality and quantity for detecting the watermark.

The watermark detection module (140) can detect (145) information inserted (i.e., hidden) into the first watermarked digital media (125) by the watermark insertion module (120) and generate restoration information (155). The restoration information (155) can be considered to be the same as the predetermined information that the watermark insertion module (120) can insert. If the restoration information (155) is traceability information, it can help identify a specific user (150) from the restoration information (155).

Basic concept of watermark insertion and detection according to the present invention

Next, the basic concept of the watermark insertion procedure based on the present invention is explained.

FIG. 2 is a basic conceptual diagram of a method for generating a watermark information sequence based on a frame sequence of digital media and a sequence of information to be inserted in the present invention.

The digital media (210) to which the watermark is inserted may be digital media configured based on frames (215), and may be, for example, digital images or audio encoded in a frame-based coding format.

In the description related to the above drawing 2, the frame is used as a general term for a standard unit designated to encode the contents of a specific digital media, such as a frame or picture, in any encoding or storage standard for the digital media. The frame may be, for example, a frame or picture representing an individual image of digital video media, a frame representing a unit encoding section of digital audio media, a paragraph representing a unit storage section of digital document media, or an individual object or polygon constituting digital 3D polygon media, and may correspond to any other encoding or storage unit.

In addition, the frame may be limited to some components, not all information components, of a specific digital media. For example, in the case of digital image media using the YCbCr color expression method, the frame may be limited to only the brightness (Y) channel among any frame selected from the image media, and there is no problem in implementing the technical idea of the present invention even if any other components are selected and combined. In addition, the frame does not necessarily have to be interpreted as expressing one frame, and may target a section composed of two or more frames, or may be an information unit such as a slice, macroblock, or block that further divides one frame.

While not denying the various implementation possibilities as mentioned above, in order to facilitate the explanation of the concept of the present invention, in Fig. 2, digital media (210) is considered as digital image media according to an encoding standard such as MPEG, and the frame (215) is also considered as a frame according to the encoding standard.

The content of the watermark to be inserted, which is the insertion information (220), may be any content, and it is sufficient that it corresponds to additional information for achieving a certain technical purpose by insertion. As an example of the additional information, an example related to the predetermined information inserted by the watermark insertion module (120) in relation to the above-mentioned Fig. 1 may be applied.

The above insertion information (220) may be an insertion sequence composed of a plurality of insertion symbols (225). The insertion symbol may be selected from a set composed of at least two types of distinct identification symbols. In addition, the insertion symbol may be a number having a base of 2 or higher. The insertion sequence (220) may be a result of encoding insertion information (227) in a form that can be interpreted by a human by a symbol composition method of the insertion symbol. For example, if the insertion symbol is a symbol that can have a value of 0 or 1 as digital information in binary, the English letter "A" (227), which corresponds to one example of the insertion information, may be converted into the decimal number 65 by ASCII encoding and then converted again into the binary number 1000001, thereby being composed of an insertion sequence expressed by at least 7 insertion symbols. It is easy to see that the encoding method of information and the type and base of the inserted symbol symbol can all utilize various technologies that can be applied by a person skilled in the art, and do not constitute elements limiting the technical idea of the present invention.

While not denying the various implementation possibilities as mentioned above, in order to facilitate the explanation of the concept of the present invention, in Fig. 2, the content of the insertion information (220) is considered as an insertion sequence in which traceability information is digitally encoded, and the insertion symbol (225) is considered as a single digit of a digital binary notation having a symbol of 0 or 1.

As shown in FIG. 2, if the length of the insertion information (220) is N and is composed of a total of N insertion symbols (225), one frame (215) can correspond (230) to one insertion symbol (225). That is, the insertion information (220) of length N can be inserted into a frame sequence (217) of length N composed of N frames. FIG. 2 exemplifies a situation in which digital information 01001000 composed of binary numbers is inserted by encoding the English letter "A" in an 8-bit space using ASCII code, and in this example, the length N is 8.

Next, the process of inserting the above insertion information (220) into each frame of the frame sequence (217) is described.

For each symbol included in the above-mentioned insertion information (220), watermarking information having different properties may be generated according to the type of the symbol. The properties of the above-mentioned watermark information may include whether or not a watermark is inserted, the insertion strength (power), or the method of the watermarking algorithm used for insertion. In addition, the watermark information may be configured in the form of a pattern, overlay image, property value, or command indicating a specific watermarking method by any one of the above-mentioned properties. The details thereof will be described later in the description related to the configuration of the watermark insertion device.

By combining the watermarking information generated for all symbols of the above insertion information (220), a watermarking information sequence (260), which is a set of attribute information for inserting the information of the above insertion information (220) into a frame sequence (217) of digital media (210) through watermarking, can be generated.

Regarding the above-described procedure, some frames illustrated in Fig. 2 are described as examples. A first symbol (251) corresponds to a first frame (241), and since the value of the first symbol is "0", first watermark information (261) having an attribute corresponding to symbol "0" can be generated, and a second symbol (252) corresponds to a second frame (242), and since the value of the second symbol is "1", second watermark information (262) having an attribute corresponding to symbol "1" can be generated.

By repeating the same procedure as above N times (8 times in the example of FIG. 2) which is the length of the inserted information (220), a watermark information sequence (260) consisting of a total of N pieces of watermark information to be applied to each frame belonging to the frame sequence (217) can be generated.

Figure 3 is a basic conceptual diagram of a method for watermarking digital media using a watermark information sequence in the present invention. The watermark information sequence (260) generated by the above-described method can be utilized in the process of inserting a watermark corresponding to each frame included in the original digital media (210).

Referring to the example illustrated in FIG. 3 by applying the example illustrated in FIG. 2, the first watermark information (260) included in the watermark information sequence (260) may indicate information related to the properties of a watermark to be inserted into the first frame (241), which is one of the frames (215) included in the original digital media (210), as described above.

When information of a first frame (241) is acquired from the original digital media (210) and digital watermarking (350) is performed on the first frame (241) using the first watermark information (261), a first watermarked frame (341) in which information of the first symbol (251) is inserted into the first frame (241) can be generated. In the same procedure, digital watermarking is repeated on frames (215) according to the watermark information sequence (260) to acquire watermarked frames (315), and when each watermarked frame (315) is connected in the original order, a watermarked digital media (310) can be finally acquired.

In conclusion, by the procedure described with reference to the above FIGS. 2 and 3, insertion information of length N can be inserted across N consecutive frames on digital media. This procedure can be performed continuously by repeating every N frames, or it can be implemented so that a padding section unrelated to watermarking is formed for an arbitrary length of the repetition every N frames and then inserted again, or it can be implemented so that N frames are cut out from a specific part of the digital media and inserted into a frame sequence where at least one position is fixed. In addition, it can be implemented so that the length N of the insertion information is not fixed, and the last symbol of the insertion information or the padding section in which the watermark is not inserted indicates the end of the insertion information. More detailed examples of modifications will be described again together with the embodiments.

Next, a watermark detection procedure based on the present invention is described.

Figure 4 is a basic conceptual diagram of a process for detecting information inserted from watermarked digital media in the present invention.

The watermarked digital media (310) can be the target of a detection attempt to detect the inserted information. For detection, the watermarked frames (315) belonging to the watermarked digital media (310) can be sequentially read and detection can be attempted on a frame-by-frame basis.

When the watermarked frame (341) acquired for analysis is input into a digital watermarking detection (450) algorithm, the type of watermark information (461) inserted into the watermarked frame (341) can be identified. In addition, based on the property of the watermark information (461), the detection symbol (410) indicated by the corresponding watermark information (461) can be restored. The detection symbols are sequentially acquired (425) in a predetermined number, for example, in the same sequence length N as mentioned in FIGS. 2 and 2, and then combined to acquire the detection sequence (420).

The above detection sequence can be restored to detection information (427) by applying the same method in reverse as when it was inserted. For example, if the detection symbol is a symbol that can have a value of 0 or 1 as binary digital information, the detection sequence "0100001" (420) can be obtained as a result of combining eight detection symbols (425), and when this is interpreted by ASCII encoding, it corresponds to the decimal number 65 and can be configured to be restored as the English letter "A".

Watermark insertion device and its operation method

FIG. 5 is a block diagram of one embodiment of a watermark insertion device according to the present invention.

A watermark insertion device (520) according to one embodiment of the present invention may be combined with a media acquisition unit (510) providing input, an insertion information acquisition unit (520), and a media recombinant unit (590) generating output to form a watermark insertion module (500). The configuration of the above module shows a structure separated according to the division of functions in one embodiment, and it is obvious that it can be implemented in various ways, such as various implementation methods of unit blocks as described above.

The media acquisition unit (510) may be configured to read digital media (501) as input to the watermark insertion module (500). The digital media (501) is the same as the digital media (115, 210) in the description referring to FIGS. 1 and 2, and the embodiment thereof may also be applied to the same example.

The media acquisition unit (510) may be configured to receive digital media (501) via a digital communication network. Alternatively, the media acquisition unit (510) may be configured to read digital media (501) stored in a fixed or variable digital recording medium. Alternatively, the media acquisition unit (510) may be configured to process digital media (501) generated in real time from an input device such as a camera or a recorder. The media acquisition unit (510) may also be modified and implemented in various embodiments to implement information input of digital media (501).

The above media acquisition unit (510) may be configured to sequentially separate frames, which are the standard units for composing information of digital media (501), preferably from the starting point of digital media (501) information, to generate frame data (515) and provide the frame data to a watermark insertion device (530).

As described above with respect to digital media (210) in the description of FIG. 2, the separated frame data (515) may be, for example, a frame or picture representing an individual image of digital video media, a frame representing a unit encoding section of digital audio media, a paragraph representing a unit storage section of digital document media, or an individual object or polygon constituting digital 3D polygon media, or may be information of any other encoding or storage unit.

In the present invention, assuming that the frame data (515) is a continuous frame sequence for a certain length, a representative embodiment is to sequentially insert a watermark into each frame data (515) belonging to the frame sequence.

However, the media acquisition unit (510) may be configured to sequentially and continuously separate the digital media (501) to generate one frame data (515), but may also be configured to separate in a set order including reverse order, or to separate only one frame data (515) at a set interval. Alternatively, it may be configured to separate two or more frame data (515) by one separation. Any method other than the above example may be used as long as the method of the media acquisition unit (510) generating the frame data (515) from the digital media (501) follows a rule set in advance so that it can be recognized by a watermark detection device later.

The insertion information acquisition unit (520) may be configured to read insertion information (502) as an input given to the watermark insertion module (500). The insertion information (502) includes predetermined information inserted into the digital media (115) in the description referring to FIGS. 1 and 2, and the embodiment thereof may also apply the same example. In particular, the insertion information (502) may be traceability information that can uniquely identify at least one of a viewing device or a recipient of the digital media.

The above-mentioned insertion information (502) may be the predetermined information expressed in a form that can be interpreted by a human, but depending on the embodiment, it may be provided as an insertion sequence in an encoded form by expressing the predetermined information using at least one symbol. In encoding using the symbol, the symbol may be configured to be selected from a set of two or more types of distinct identification symbols. In addition, the symbol may be a number having a base of 2 or higher.

For example, if the above-described insertion information (502) is encoded as digital information in binary, the set of the identification symbols may be composed of {0, 1}. In this case, the English letter "A" corresponding to one example of the above-described insertion information is converted into the decimal number 65 by ASCII encoding, and then converted again into the binary number 1000001, so that it may be composed of an insertion sequence expressed by at least seven symbols (in this example, bits) belonging to {0, 1}. It can be easily seen that the encoding method of the above-described insertion sequence and the type and base of the symbol can all utilize various technologies that can be applied by a person skilled in the art, and do not correspond to elements limiting the technical idea of the present invention.

The above insertion information acquisition unit (520) may be configured to separate symbols that serve as a standard unit of information expression from the insertion information (502) or the insertion sequence constituting the insertion information, preferably sequentially from the starting point, to generate an insertion symbol (525), and provide the same to the watermark insertion device (530).

In the present invention, a representative embodiment is set in which the insertion symbol (525) assumes a sequence of continuous insertion information for a certain length, and sequentially inserts watermarks based on each insertion symbol (525) belonging to the insertion sequence.

However, the insertion information acquisition unit (520) may be configured to sequentially and continuously separate the insertion information (502) to generate one insertion symbol (525), but may also be configured to separate in a set order including a reverse order, or to separate only one insertion symbol (525) at a set interval. Alternatively, it may be configured to separate two or more insertion symbols (525) by one separation. Any method other than the above example may be used as long as the method of the insertion information acquisition unit (520) generating the insertion symbol (525) from the insertion information (502) follows a rule set in advance so that the information can be restored in a watermark detection device later.

The watermark insertion device (530) may be a device that inserts a watermark into frame data (515) using an insertion symbol (525), and may be composed of a frame data generation unit (540), an insertion symbol generation unit (550), a watermark information generation unit (560), a watermark insertion unit (570), and a memory (580).

The frame data generation unit (540) may be configured to read frame data (515) provided from the media acquisition unit (510) for the watermark insertion device (530) and provide (545) the frame data to the watermark insertion unit (570). The frame data generation unit (540) may be configured to provide (545) the input frame data (515) without processing, depending on the embodiment, or may be configured to generate frame data directly from digital media (501) and provide (545) by being directly connected to the media acquisition unit (510) or configured with the same module. Alternatively, the frame data generation unit (540) may be configured to provide (545) the input frame data (515) by applying at least one procedure of modification, correction, processing, scaling, format change, re-encoding, or insertion of another watermark to the input frame data (515) in another embodiment. If the frame data that is provided as input to the watermark insertion device (530) is configured to generate frame data that is the target of watermarking and provide it (545) to the watermark insertion unit (570), any additional function may be added to the frame data generation unit (540).

The insertion symbol generation unit (550) may be configured to read the insertion symbol (525) provided from the insertion information acquisition unit (520) for the watermark insertion device (530) and provide (555) the watermark information generation unit. The insertion symbol generation unit (550) may, depending on the embodiment, be configured to provide (555) the input insertion symbol (525) without processing, or may be configured to generate (555) the insertion symbol directly from the insertion information (502) by being directly connected to the insertion information acquisition unit (520) or configured as the same module. Alternatively, the insertion symbol generation unit (550) may, in another embodiment, be configured to provide (555) the input insertion symbol (525) by applying at least one procedure of modification, correction, calculation, conversion, processing, or re-encoding. If a symbol is provided as input to a watermark insertion device (530) and inserted (i.e., hidden) through watermarking and configured to be provided (555) to a watermark information generation unit (560), any additional function may be added to the insertion symbol generation unit (550).

The watermark information generation unit (560) can generate and provide watermark information (565) for instructing the operation method of the watermark insertion unit (570). The watermark information (565) can be generated differently depending on the value or type indicated by the insertion symbol (555) input to the watermark information generation unit (560). That is, the watermark information generation unit (560) can be configured to have a role of instructing the watermark insertion unit (570) to indicate a watermark that is distinguished by the insertion symbol (555).

The above watermark information (565) may include at least one of information corresponding to whether a watermark is inserted, the inserted content, the inserted method, the inserted power, the inserted transparency, the inserted location, or other insertion parameters of the watermark.

The configuration of the above watermark information (565) may vary depending on the type of watermark algorithm implemented in the watermark insertion unit (570). For example, in the case of a watermark algorithm that can insert watermarks that are each distinct by adjusting the insertion strength, the insertion strength may be included in the watermark information (555). Or, in the case of a watermark algorithm that can insert watermarks that are each distinct by adjusting the insertion position, the insertion position may be included in the watermark information (555). Or, when a plurality of watermark algorithm methods are implemented in the watermark insertion unit (570), information designating one of the methods may be included in the watermark information (555).

In addition, the watermark information (565) may include information for simply executing or not executing a watermark, depending on the embodiment. For example, if the insertion symbol (555) is a binary number, it may include information for instructing to execute a watermark if the value is 1, and not to execute a watermark if the value is 0.

In the process of generating the above watermark information (565), frame data generated by the frame data generation unit (540) may be referenced (547). The above-mentioned frame data (547) may also be stored in a memory (580) that records and stores previously processed frame data (545) or watermarked frame data (575), and more than one may be referenced from the memory (580) at the same time.

The watermark information generation unit (560) may be configured to analyze the contents of a frame, including at least one of the currently processed frame data (545), the currently processed watermarked frame data (575), or the previously processed frame data or watermarking data stored in the memory (580), through reference (547) to the frame data.

Meanwhile, when inserting a watermark into digital media, especially when maintaining the invisibility of the watermark, there may be a potential threat that the watermark insertion efficiency may be reduced or the invisibility may be violated and the watermark may be exposed in a section where a specific change occurs in the original digital media content (501). Accordingly, or if it is analyzed that watermark insertion is not suitable for the current frame data (545) for reasons other than those mentioned above, the configuration of the watermark information to be provided to the watermark insertion unit (570) may be changed, or the watermark information may be configured so as not to insert a watermark.

Accordingly, the watermark information generation unit (560) may be configured to include or be linked to a suitability evaluation unit (not shown). The suitability evaluation unit may be configured to evaluate the level of confidence in the insertion of a watermark by an insertion symbol (555) into the currently processed frame data (545) by analyzing the content of the frame (e.g., detecting the specific change). The suitability evaluation unit may evaluate the confidence level as a quantitative value. In addition, if the quantitative value of the confidence level is below a predetermined threshold value, the watermark insertion may be configured to make a pass/fail evaluation as inappropriate.

The present invention does not limit the type of digital media, and it should be understood that the specific change indicates a change that interferes with the useful operation of the watermark insertion unit (570) among the changing aspects of different frame data according to the content type of the digital media, as is well known to those skilled in the art with watermark technology for each type of digital media.

In one embodiment of the present invention, when the digital media corresponds to video content, the specific change may include a change in at least one of continuity, brightness, saturation, color, motion vector size, scene transition, and type of key frame (e.g., I frame) or reference frame (e.g., P or B frame) of the frame data.

In one embodiment of the present invention, if the digital media corresponds to audio content, the specific change may include a change in at least one of continuity, amplitude change, frequency change, and voice form (e.g., human speech, music, noise, etc.) of the frame data.

The techniques for detecting each of the changes exemplified above may follow what is conventionally known to those skilled in the art in digital media. For example, in order to detect a scene change in video content in the above-described embodiment, the method for detecting a scene change frame disclosed in Patent Publication No. 2017-0090868 may be used.

The watermark insertion unit (570) may be configured to receive the frame data (545) and the watermark information (555) as inputs, insert a watermark into the frame data (545) based on the watermark information (555), and generate watermarked frame data (575).

Any watermark algorithm method known to provide the same function may be used by the watermark insertion unit (570). In addition, the watermark algorithm method may be adopted in a way that makes it difficult for a person to discern whether a watermark has been inserted from watermarked frame data, such as invisible or inaudible insertion. However, depending on the embodiment, a watermark algorithm method that applies a change that a person can discern, such as visible or audible insertion, may be used, and a method in which an invisible/inaudible method and a visible/audible method are mixed may be used.

The watermark algorithm method used by the above watermark insertion unit (570) may be implemented as either a real-time or non-real-time watermark insertion algorithm. In an environment where real-time is important, such as a digital media distribution environment via a communication network, an algorithm that guarantees real-time can be used, and in an environment where it is sufficient to complete the watermark ahead of the watermark reproduction time, an algorithm that does not guarantee real-time can be used.

The above watermark insertion unit (570) may be configured to use two or more multiple watermark algorithm methods. The multiple watermark algorithms may be selected differently according to the insertion symbol (555) by the watermark information (565), may be selectively applied by other rules, or may be applied serially and repeatedly.

The watermark information (565) generated by the watermark information generating unit (560) or the watermark insertion algorithm of the watermark inserting unit (570) may be configured so that the watermarked digital media (595) can include information encrypted by a watermark key. The watermark key encrypts information inserted through the watermark using an arbitrary key value, thereby preventing the watermark from being read by an unauthorized third party, and can also help improve detection performance when different watermarks are inserted into the same digital media.

The media recombinant unit (590) may be configured to receive the watermarked frame data (575), which is the output of the watermark insertion unit (570), as the output of the watermark insertion device (530), and restore it into watermarked digital media (595).

Since the media acquisition unit (510) previously separated the frame data (515) from the digital media (501) and provided it to the watermark insertion device (530), it must be restored back to the form of digital media in order to be distributed. The media recombining unit (590) may be configured to reconstruct the watermarked frame data (575) in accordance with the separation method of the media acquisition unit (510). For example, if the media acquisition unit (510) sequentially separates the frame data (515) over time, the media recombining unit (590) may be configured to combine the watermarked frame data (575) over time.

The media recombining unit (590) may be configured to provide the watermarked digital media (595) to another device, module, function, or system. In one embodiment of the present invention, the media recombining unit (590) may be configured to be connected to a transmitter for a communication network to transmit the watermarked digital media (595) to a specific viewing device. In another embodiment of the present invention, the media recombining unit (590) may be connected to a computing device including a recording function for an electronic recording medium to record the watermarked digital media (595) on the recording medium. In yet another embodiment of the present invention, the media recombining unit (590) may be configured to be directly connected to a viewing device to display the watermarked digital media (595) to a user.

Watermark detection device and its operation method

Figure 6 is a block diagram of one embodiment of a watermark detection device according to the present invention.

A watermark detection device (620) according to one embodiment of the present invention may be combined with a media acquisition unit (610) that provides input and an information restoration unit (690) that generates output to form a watermark detection module (600). The configuration of the above module shows a structure separated according to the division of functions in one embodiment, and it is obvious that it can be implemented in various ways, such as various implementation methods of unit blocks as described above.

The media acquisition unit (610) may be configured to read watermarked digital media (601) as input to the watermark detection module (600). The watermarked digital media (601) is the same as the watermarked digital media (125, 125a, 137, 310) in the description referring to FIGS. 1 and 2, and the embodiment thereof may also be applied to the same example.

The media acquisition unit (610) may be configured to receive watermarked digital media (601) via a digital communication network. Alternatively, the media acquisition unit (610) may be configured to read watermarked digital media (601) stored in a fixed or variable digital recording medium. Alternatively, the media acquisition unit (610) may be configured to process watermarked digital media (601) collected via an input device such as a camera or a recorder. The media acquisition unit (610) may also be modified and implemented in various embodiments for implementing information input of watermarked digital media (601).

The above media acquisition unit (610) may be configured to sequentially separate frames, which are the standard units for composing information of watermarked digital media (601), preferably from the starting point of the watermarked digital media (601) information, to generate watermarked frame data (615), and provide the same to a watermark detection device (620).

As described above with respect to the watermarked digital media (215, 215a) in the description of FIG. 2, and as described above with respect to the frame data (515) in the description of FIG. 5, the watermarked frame data (615) may be information of any encoding or storage unit.

In the present invention, the frame data (615) is assumed to be a continuous watermarked frame sequence for a certain length, and a representative embodiment is to restore information originally inserted into digital media by sequentially detecting watermarks on each watermarked frame data (615) belonging to the watermarked frame sequence.

However, the media acquisition unit (610) may be configured to sequentially and continuously separate digital media (601) to generate one watermarked frame data (615), but any other method may be used as long as the watermark detection module (600) follows a rule set in advance for the operation of the media acquisition unit (520) or media recombinant unit (590) in the corresponding watermark insertion module (500).

The watermark detection device (620) may be a device that detects and outputs a detection symbol (655) from watermarked frame data (615), and may be composed of a frame data generation unit (630), a watermark detection unit (640), a symbol detection unit (650), and a memory (680).

The frame data generation unit (630) may be configured to read watermarked frame data (615) provided from the media acquisition unit (610) for the watermark detection device (620) and provide (635) the watermark detection unit (640). The frame data generation unit (520) may be configured to provide (635) the input watermarked frame data (615) without processing, depending on the embodiment, or may be configured to generate watermarked frame data directly from watermarked digital media (601) and provide (635) the same by being directly connected to the media acquisition unit (610) or configured with the same module. Alternatively, the frame data generation unit (630) may be configured to apply at least one procedure of modification, correction, processing, scaling, format change, re-encoding, or detection of another watermark to the input watermarked frame data (615) and then provide (635) the same. If the watermark detection device (630) is configured to generate watermarked frame data that is provided as input to the watermark detection device (630) and is subject to detection and then provided (635) to the watermark detection unit (640), any additional function may be added to the frame data generation unit (630).

In addition, according to one embodiment of the present invention, the watermarked frame data (635) may be provided to the watermark detection unit (640) and stored in the memory (680). The memory (680) may be configured to store and provide (637) at least one previously watermarked frame data for the operation of the watermark detection unit (640).

The watermark detection unit (640) may be configured to receive the watermarked frame data (635) as input and generate watermark information (645) using a watermark detection algorithm.

The watermark detection algorithm method used by the watermark detection unit (640) may be any method that can read the watermark inserted by the watermark insertion module (500). In addition, the watermark algorithm method used by the watermark detection unit (640) may be implemented as either a real-time or non-real-time watermark detection algorithm. According to one embodiment of the present invention, the watermark detection module (600) is configured to operate on watermarked digital media (601) that has already been collected and stored, and may be an algorithm that does not guarantee real-time performance but has improved detection accuracy.

The above watermark detection unit (640) may be configured to use two or more multiple watermark detection algorithms. The multiple watermark detection algorithms may also be used under any conditions, methods, or orders as long as they can read the watermark by the multiple watermark algorithms used in the watermark insertion module (500).

The watermark information (645) generated by the watermark detection unit (640) may include at least one of information corresponding to whether a watermark is inserted, the inserted content, the inserted method, the inserted power, the inserted transparency, the inserted location, or other insertion parameters of the watermark. In one embodiment of the present invention, the watermark information (645) may be configured with the same information as the watermark information (565) generated by the watermark insertion module (500), or may be configured with different information from the insertion module (500) for ease of detection. In addition, the watermark detection unit (640) may be configured to receive a watermark key and detect a watermark encrypted by the watermark key when inserted.

In the process of generating the above watermark information (645), watermarked frame data generated by the frame data generation unit (620) and stored in the memory (680) may be referenced (637). The watermark detection unit (640) may utilize the reference (637) of the watermarked frame data to supplement or strengthen the watermark detection performance.

By way of example, as an embodiment of the present invention, if the watermark insertion module (500) inserts an invisible watermark for adjusting the image brightness for each frame of digital media composed of video content, in order to separate the artificial image brightness change generated by the watermark from the natural change in the image brightness that originally existed in the original digital media (501), at least two or more watermarked frame data (635) may be stored in the memory (580) and then statistically analyzed by referencing (637), thereby detecting the change generated by the watermark based on the trend curve of the image brightness obtained from each frame data. In addition, it is obvious that an analysis and detection method spanning multiple frames, which is used in various digital watermark insertion and detection algorithms known to those skilled in the art, may be used.

In addition, the watermark detection unit (640) may be configured to include or be linked to a suitability evaluation unit (not shown). The suitability evaluation unit may be configured to evaluate a confidence level of whether a watermark has been inserted into the currently processed watermarked frame data (635) by analyzing the content of the watermarked frame data (for example, detecting a specific change as exemplified in the description of the watermark insertion module (500). The suitability evaluation unit may evaluate the confidence level as a quantitative value. In addition, if the quantitative value of the confidence level is lower than a predetermined threshold value, it may be considered that a watermark has not been inserted, and the watermark detection unit (640) may be configured not to generate watermark information (645).

The present invention is not limited to the type of digital media, and the watermark detection algorithm method, the method of enhancing detection performance, and specific changes should be understood as representing all technical configurations configured to enable detection of valid watermark information (640) from the changing aspects of different frame data according to the content type of the digital media, as is well known to those skilled in the art having watermark technology for each type of digital media.

The symbol detection unit (650) may be configured to receive the watermark information (645), determine which detection symbol the watermark information (645) should be interpreted as, and output the detection symbol (645) as an output of the watermark detection device (620).

The above detection symbol (645) may be any information unit that can reconstruct the insertion information (502) given to the watermark insertion module (500), but preferably, it may be the same symbol as the insertion symbol (555), or at least a symbol selected from a set of the same size. As an example, by way of explanation through one embodiment of the present invention, if the insertion symbol (555) is a symbol selected from the set of {0, 1} as binary digital information, the detection symbol (655) may also be selected from the set of {0, 1}, or may be selected from the set of {TRUE, FALSE} having the same size.

The above watermark information (645), in one embodiment of the present invention, may be identical to the watermark information (565) at the time of insertion in the watermark insertion module (500), or may be information that can infer the contents of the watermark information (565) at the time of insertion. In this embodiment, the symbol detection unit (650) may be configured to generate a detection symbol (655) by categorizing and distinguishing the contents of the watermark information (645) and thereby confirming whether it corresponds to a certain detection symbol value.

The above watermark information (645), in another embodiment of the present invention, may be configured as information that can directly derive the value of the detection symbol (655) or the insertion symbol (555). In this case, the watermark information (645) may be configured to generate the detection symbol (655) simply by re-outputting it without any special processing.

The information restoration unit (690) may be configured to receive the detection symbol (655), which is the output of the symbol detection unit (650), as the output of the watermark detection device (620), and restore it to restoration information (695). The restoration information may be considered to be the same as the restoration information (155) described above in the description of FIG. 1. In addition, it may be considered to be information that is the same as or symmetrical to the insertion information (502) described above in the description of FIG. 5 or predetermined information included in the insertion information (502). The symmetrical information means a relationship in which one piece of information can be specifically corresponded to another piece of information, such as a public key versus a private key or the input and output of a hash function, but is not limited to the relationship in the example above.

In the watermark insertion module (500), the insertion symbol (525) is separated from the insertion information (502) to watermark the frame data (515). Therefore, in order to obtain restoration information (695) corresponding to the insertion information (502), the detection symbol (655) obtained from the watermarked frame data (615) must be recombined.

The information restoration unit (690) may be configured to reconstruct the detection symbol (655) in accordance with the separation method in the insertion information acquisition unit (520). For example, if the insertion information acquisition unit (520) sequentially separates the insertion symbol (525) over time, the information restoration unit (690) may be configured to sequentially combine the detection symbols (655) over time to obtain the restoration information (695).

The above restoration information (695) can be utilized by the user of the watermark detection module (600). In one embodiment of the present invention, with additional reference to the description of FIG. 1 and FIG. 5 above, the insertion information (502) included in the watermarked digital media (125, 125a, 502, 601) can include traceability information that can uniquely identify at least one of a viewing device or a recipient of the digital media, and the traceability information can be detected and restored with the restoration information (155, 695). Accordingly, it can help to identify for which viewing device (130, 130a) or user (150) the watermarked digital media (601) was originally provided.

Modified embodiment

Hereinafter, some modified embodiments that may be assumed in the watermark insertion device and its operating method according to the present invention will be described.

According to one embodiment of the present invention, frame data to be watermarked may be divided into a plurality of areas, and watermark information with different contents may be generated in the plurality of areas so that different watermarks may be inserted.

FIG. 7 is a conceptual diagram of frame data in which region division has been performed. When there is one frame data (730), the frame data (730) can be divided into region A (710) and region B (720). When the frame data corresponds to, for example, one frame of a moving image digital media, region A (710) and region B (720) can be set by spatially dividing the frame image. At this time, the method of dividing the region may be in any form. For example, the example shown in FIG. 7 shows an example of dividing the frame image vertically into region A (710) and region B (720) of the same size, but the sizes of region A (710) and region B (720) may be defined differently.

The above method of dividing the region can be applied in various ways, such as horizontal division, division by a dividing line defined by a polygon, or division by a dividing line defined by a curve including a circle and an ellipse.

The above-described method of segmenting the region may also be defined by different concepts depending on the type of digital media. For example, if the frame data is derived from audio digital media, the segmentation may be implemented by segmentation by frequency band, for example, if the frame data is derived from text digital media, the segmentation may be implemented by cross-segmentation by line, for example, if the frame data is derived from digital media including three-dimensional object data, the segmentation may be implemented by segmentation by object. The method of re-segmenting the information unit of the digital media may be any method that can be used by a person skilled in the art in other types of digital media or other methods.

The above region division method may further be configured to divide the frame data into two or more regions. In addition, the above region division method may be a method of dividing such that at least one region overlaps at least one other region, as long as the watermark insertion method allows it.

Fig. 8 is a block diagram of an embodiment of a watermark insertion device according to the present invention that further includes a region division function of frame data. Fig. 8 is an embodiment modified from Fig. 5, and parts that are not changed from Fig. 5 are indicated by the same symbols.

A watermark insertion module (800) according to a modified embodiment of the present invention may include a plurality of insertion information acquisition units. For example, referring to FIG. 8, it may include a first insertion information acquisition unit (821) that receives first insertion information (801) and a second insertion information acquisition unit (822) that receives second insertion information (802). The function and operation of each of the plurality of insertion information acquisition units may be identical or similar to the insertion information acquisition unit (520) described with FIG. 5, and may be implemented with reference to the embodiment of the insertion information acquisition unit (520) described with FIG. 5.

The watermark insertion device (830) included in the watermark insertion module (800) may include a plurality of insertion symbol generation units. For example, referring to FIG. 8, it may include a first insertion symbol generation unit (851) that receives and transmits (855) a first insertion symbol (825), and a second insertion symbol generation unit (852) that receives and transmits (856) a second insertion symbol (826). The function and operation of each of the plurality of insertion symbol generation units may be identical or similar to the insertion symbol generation unit (550) described with reference to FIG. 5, and may be implemented with reference to the embodiment of the insertion symbol generation unit (550) described with reference to FIG. 5.

The watermark information generation unit (860) of the watermark insertion device (830) included in the watermark insertion module (800) may include a plurality of area watermark information generation units. For example, referring to FIG. 8, the watermark information generation unit (861) may include a first area watermark information generation unit (861) that receives a first insertion symbol (855) and generates first area watermark information, and a second area watermark information generation unit (862) that receives a second insertion symbol (856) and generates second area watermark information. The function and operation of each of the plurality of area watermark information generation units may be identical or similar to the watermark information generation unit (560) described with reference to FIG. 5, and may be implemented with reference to the embodiment of the watermark information generation unit (560) described with reference to FIG. 5.

The first and second region watermark information generating units (861, 862) may be configured to generate first region watermark information and second region watermark information for the first region and the second region, respectively, formed by separating the frame data (845). In addition, the watermark information generating unit (860) may be configured to generate watermark information (865) to be finally executed on the frame data (845) based on the first region watermark information and the second region watermark information.

The frame data generation unit (840) of the watermark insertion device (830) included in the watermark insertion module (800) may further include a frame separation unit (841). The frame separation unit (841) may be configured to generate region frame data (845) by dividing the frame data (515) into two or more regions by a specified method. However, the region frame data (845) may be identical to the undivided frame data (545) of FIG. 5, depending on the configuration of the watermark information generation unit (860).

In the embodiment shown in Fig. 8, insertion information, insertion symbol, and area watermark information are configured to be independently generated for each separated area of frame data. It is obvious that the basic intent of the present embodiment does not deviate even if the same information is used in some of the above independent generation processes or processes processed by non-separated functional units are mixed. In addition, in the above Fig. 8, the description is based on the case where the frame is divided into two areas, but as the number of divisions increases, a third, fourth, ... insertion information acquisition unit, an insertion symbol generation unit, and an area watermark information generation unit may be further included.

FIG. 9 is an exemplary diagram showing a method for inserting a watermark based on region segmentation in a modified embodiment of a watermark insertion device according to the present invention. According to the watermark insertion module (800) illustrated in FIG. 8, the segmented regions of frame data can be independently watermarked by different methods. FIG. 9 (a) to (d) illustrate various implementation methods of the present invention that can be implemented through the independent watermarking.

Fig. 9 (a) is for a first embodiment of a method for inserting a watermark by utilizing a divided area. When there is insertion information of length N (8 in the example of Fig. 8), the same insertion information can be inserted into the divided area by a different watermarking method. One insertion symbol is generated from one insertion information, and the one insertion symbol is transformed into a plurality of different area watermark information by an independent area watermark information generating unit, and watermarked frame data can be obtained by the watermark information generated by the combination thereof. That is, watermarked area A frame data (911) and watermarked area B frame data (912) can be configured to coexist in one frame data. For example, area A (931) and area B (932) in one frame data (930) can include different watermarks representing the same insertion symbol.

Fig. 9 (b) is about a second embodiment of a method for inserting a watermark by utilizing a divided area. When there is insertion information of length N (8 in the example of Fig. 8), it can be configured to insert multiple insertion symbols into one frame data by utilizing the divided area. The frame data in which the watermarked A region frame data (913) and the watermarked B region frame data (914) are combined can be configured to include two pieces of insertion symbol information at the same time. Therefore, in this case, the length of the frame sequence required to insert insertion information of length N becomes N/2.

Fig. 9 (c) is about a third embodiment of a method for inserting a watermark by utilizing a divided area. When there is first insertion information of length N (8 in the example of Fig. 8) and second insertion information of length M (6 in the example of Fig. 8), the divided area can be configured to insert insertion symbols derived from different insertion information into one frame data. The frame data in which watermarked A region frame data (915) and watermarked B region frame data (916) are combined can be configured to include insertion symbols for two different pieces of insertion information at the same time.

The insertion cycle for the first insertion information of the length N and the insertion cycle for the second insertion information of the length M may be configured to be inserted asynchronously, as in the third embodiment. Alternatively, they may be configured to be inserted synchronously, as in the fourth embodiment shown in (d) of Fig. 9. In this case, a padding section P may be defined to prevent a watermark from being inserted in at least one area in order to synchronize the insertion cycles.

Despite the description of the above various embodiments, it is obvious that a person skilled in the art can easily construct a method of applying the present invention in a manner other than that exemplified above in order to achieve any technical goal that can be achieved by dividing frame data by region and performing different watermarking, and therefore the above embodiments do not limit the scope of the rights of the present invention.

In addition, when the watermark insertion module (800) of the method of dividing the area as described above is used, the watermark detection module may also be implemented as a modified embodiment of the corresponding method. The watermark detection device (620) included in the watermark detection module (600) of FIG. 6 further includes a frame separation unit in which the frame data generation unit (630) divides watermarked frame data into two or more areas, the watermark detection unit (640) further includes an area watermark detection unit in which the watermark for at least one of the areas is detected and area watermark information is generated based on the detected area watermark, and the watermark information (645) may be modified to be generated based on the at least one area watermark information.

According to another embodiment of the present invention, the watermark insertion module may be configured to achieve the effect of actual watermarking by reading frame data from a plurality of preprocessed digital media that have undergone preprocessing such as inserting a watermark in advance without actually running a watermarking algorithm, and selectively reading the frame data based on the insertion information.

Fig. 10 is a block diagram of an embodiment of a watermark insertion device according to the present invention that utilizes preprocessed digital media. Fig. 10 is an embodiment modified from Fig. 5, and parts that are not changed from Fig. 5 are indicated by the same symbols.

A watermark insertion module (1000) according to a modified embodiment of the present invention may include a plurality of media acquisition units, and the plurality of media acquisition units may be configured to acquire digital media that has been preprocessed differently, respectively. For example, referring to FIG. 10, the module may include a first preprocessing media acquisition unit (1011) that acquires a first preprocessed digital media (1001), and a second preprocessing media acquisition unit (1012) that acquires a second preprocessed digital media (1002). The function and operation of each of the plurality of preprocessing media acquisition units may be identical or similar to the media acquisition unit (510) described with reference to FIG. 5, and may be implemented with reference to the embodiment of the media acquisition unit (510) described with reference to FIG. 5.

However, the plurality of preprocessed media acquisition units (1011, 1012) may be configured to read digital media that has been watermarked through preprocessing in advance instead of the original digital media (501). At this time, the first preprocessed digital media (1001) may be watermarked in advance by inserting a first symbol into all frame data, and the second preprocessed digital media (1002) may be watermarked in advance by inserting a second symbol into all frame data. The plurality of preprocessed media acquisition units (1011, 1012) may be configured to separate frames from the plurality of preprocessed digital media (1001, 1002), respectively, to generate first frame data (1005) and second frame data (1006), and to provide (1045, 1046) the same to the watermark insertion device (1030).

The watermark insertion device (1030) included in the watermark insertion module (1000) may include a frame selection unit (1070) instead of the watermark insertion unit. The frame selection unit may be configured to receive watermark information (565) from the watermark information generation unit (560) and select frame data in which a watermark indicated by the watermark information is previously inserted. For example, if the insertion symbol (555) generated from the insertion information (520) is the first symbol, the first frame data (1005) derived from preprocessed digital media (1001) that is previously watermarked only with the first symbol may be provided (1045) and configured to output as watermarked frame data (575).

FIG. 11 is an exemplary diagram showing a method of inserting a watermark using a frame selection unit in a modified embodiment of a watermark insertion device according to the present invention.

The first preprocessed digital media (1111) and the second preprocessed digital media (1112) are each the same original digital media in which a watermark is inserted. The symbol “0” is inserted continuously into all frames belonging to the first preprocessed digital media (1111), and the symbol “1” is inserted continuously into all frames belonging to the second preprocessed digital media (1112). When the insertion symbol is determined as “01000001” through the insertion information, the frame selection unit (1070) can be configured to select and output frame data derived from the preprocessed digital media on the side matching the value of the insertion symbol as watermarked frame data.

The modified embodiment method shown through the above FIGS. 10 and 11 can prevent excessive computational load due to watermark insertion in the watermark insertion module (1000) by preparing the first preprocessed digital media (1001, 1111) and the second preprocessed digital media (1002, 1112) by watermarking them in advance, and thus, when distributing one original digital media to multiple users in real time, it can be helpful in inserting and providing tracking information corresponding to each user in real time.

In addition, in the above drawing 10, the description was made based on the set size of 2 to which the value of the inserted symbol belongs (for example, {0,1}), but if the size of the set increases, the third, fourth, ... preprocessed digital media, preprocessed media acquisition unit, and frame data generation unit may be further included or used.

Finally, the embodiments described through the above-described Figures 7 to 9 and the embodiments described through the above-described Figures 10 to 11 may be used in a combined or mixed form. For example, a frame may be divided into two and watermarked, with a watermark inserted into the first area by a watermark insertion unit, and the content corresponding to the second area selected from one of the preprocessed digital media, and the combined result may be used as watermarked frame data.

The present invention has been described above with reference to drawings and embodiments, and has also dealt with some foreseeable modified embodiments. However, as already mentioned above, it does not mean that the protection scope of the present invention is limited by the drawings or embodiments presented above, and it will be understood that a person skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the following claims.

Claims (20)

A media acquisition unit for reading digital media containing at least one frame;
A frame data generation unit that sequentially reads frames from the above digital media and generates frame data;
A frame separation unit that divides the above frame data into N areas;
An insertion sequence acquisition unit that reads at least one insertion sequence;
An insertion symbol generation unit that generates N insertion symbols based on the above insertion sequence;
A watermark information generation unit that generates N area watermark information based on the above-mentioned inserted symbol; and
A watermark insertion unit is included, which inserts a watermark into the frame data based on the above-mentioned area watermark information to generate watermarked frame data.
The above N is a natural number greater than or equal to 2,
The above insertion symbol belongs to a set including at least the first symbol and the second symbol,
A watermark insertion device in which the above area watermark information is generated differently depending on the symbol of the inserted symbol.
In paragraph 1,
The watermark information for the above area is,
A watermark insertion device, characterized in that it includes N area watermark information generated by different watermark methods based on the same insertion symbol.
In paragraph 1,
The above insertion symbol generation unit,
It is configured to sequentially read the above insertion sequence and generate N insertion symbols,
The watermark information for the above area is,
A watermark insertion device, characterized in that it includes N area watermark information generated based on each of the N inserted symbols.
In paragraph 1,
The above insertion sequence acquisition unit is,
It is configured to read N different insertion sequences,
The above insertion symbol generation unit,
It is configured to generate N insertion symbols from the N insertion sequences,
The watermark information for the above area is,
A watermark insertion device, characterized in that it includes N area watermark information generated based on each of the N inserted symbols.
In paragraph 4,
At least one of the above N different insertion sequences has a different length from the rest,
The above insertion symbol generation unit,
It is configured to generate a padding symbol for synchronizing the length of the above N insertion sequences,
The watermark information for the above area is,
A watermark insertion device characterized by including padding area watermark information configured not to insert a watermark based on the padding symbol.
In paragraph 1,
A watermark insertion device, wherein the insertion sequence includes at least one of information related to contents represented by the frame data, identification information related to a viewing device on which the watermarked frame data is displayed or received, identification information related to a user of the viewing device, and information related to a connection for communicating with the viewing device.
In paragraph 1,
A watermark insertion device, wherein the above-mentioned area watermark information includes at least one of information related to the insertion strength (power) of the watermark or information related to the insertion method of the watermark.
In paragraph 1,
Further comprising a memory storing at least one frame data or at least one watermarked frame data as previous frame data,
The above watermark information generation unit further includes a suitability evaluation unit that analyzes the frame data and at least one of the previous frame data to evaluate the suitability of watermark insertion for each of the N areas,
A watermark insertion device, wherein the above-mentioned area watermark information is generated by including information for not inserting a watermark if the suitability falls below a threshold value.
In Article 8,
A watermark insertion device, wherein the suitability evaluation unit is configured to determine the suitability based on at least one of continuity of the frame data, brightness change, saturation change of the frame data, motion vector size of the frame data, scene transition included in the frame data, and I/P/B type of the frame included in the frame data.
A media acquisition unit for reading digital media containing at least one frame;
A frame data generation unit that sequentially reads frames from the above digital media and generates watermarked frame data;
A frame separation unit that divides the above watermarked frame data into N regions;
A watermark detection unit detecting at least one watermark from the N regions and generating N region watermark information based on the detected watermark; and
A symbol detection unit that generates N detection symbols based on the above-mentioned area watermark information; and
A sequence restoration unit for generating at least one detection sequence based on a detection symbol generated from the symbol detection unit;
The above N is a natural number greater than or equal to 2,
The above-detected symbol belongs to a set including at least a first symbol and a second symbol, based on the above-detected watermark information,
A watermark detection device, characterized in that the above-mentioned area watermark information is different depending on the above-mentioned detection symbol.
In Article 10,
The above detection symbol is,
A watermark detection device, characterized in that the same detection symbol is generated from at least two different area watermark information.
In Article 10,
The above sequence restoration unit,
A watermark detection device configured to generate the detection sequence by sequentially arranging the N detection symbols.
In Article 10,
The above sequence restoration unit,
A watermark detection device configured to sequentially arrange each of the N detection symbols to generate N different detection sequences.
In Article 10,
A watermark detection device, wherein the above-detected area watermark information includes at least one of information related to the power of the detected area watermark or information related to the method of the detected area watermark.
In Article 10,
A watermark detection device, wherein the detection sequence includes at least one of information related to contents represented by the watermarked frame data, identification information related to a viewing device on which the watermarked frame data is displayed or received, identification information related to a user of the viewing device, and information related to a connection for communicating with the viewing device.
In a method for inserting a watermark using an information processing device,
A step of reading digital media containing at least one frame;
A step of sequentially reading frames from the above digital media to generate frame data;
A step of dividing the above frame data into N regions;
A step of reading at least one insertion sequence;
A step of generating N insertion symbols based on the above insertion sequence;
A step of generating N area watermark information based on the above insertion symbol; and
A step of generating watermarked frame data by inserting a watermark into the frame data based on the above-mentioned area watermark information;
The above N is a natural number greater than or equal to 2,
The above insertion symbol belongs to a set including at least the first symbol and the second symbol,
A watermark insertion method in which the above area watermark information is generated differently depending on the symbol of the inserted symbol.
In Article 16,
A method wherein the above-mentioned area watermark information includes at least one of information related to the embedding power of the watermark and information related to the embedding method of the area watermark.
In a method for detecting a watermark using an information processing device,
A step of reading digital media containing at least one frame;
A step of sequentially reading frames from the above digital media to generate watermarked frame data;
A step of dividing the above watermarked frame data into N regions;
A step of detecting at least one watermark from the N regions and generating N region watermark information based on the detected watermark; and
A step of generating N detection symbols based on the above area watermark information; and
A step of generating at least one detection sequence based on the detection symbol;
The above N is a natural number greater than or equal to 2,
The above-detected symbol belongs to a set including at least a first symbol and a second symbol, based on the above-detected watermark information,
A watermark detection method, characterized in that the above-mentioned area watermark information is different depending on the above-mentioned detection symbol.
In Article 18,
A method wherein the above-detected area watermark information includes at least one of information related to the power of the detected watermark and information related to the method of the detected area watermark.
A media acquisition module for reading digital media containing at least one frame;
An insertion sequence acquisition module for reading an insertion sequence containing at least one insertion symbol;
A watermark insertion module that sequentially reads one frame data from the digital media, divides the frame data into N regions, generates N insertion symbols based on the insertion sequence, generates N region watermark information based on the insertion symbols, and inserts a watermark into the frame data based on the region watermark information to generate watermarked frame data; and
A media reconstruction module is included that sequentially combines the above watermarked frame data to output a watermarked digital media.
The above N is a natural number greater than or equal to 2,
The above insertion symbol belongs to a set including at least the first symbol and the second symbol,
A watermark insertion system in which the above area watermark information is generated differently depending on the above insertion symbol.