HK1118410A - Method and apparatus for creating a digital picture with a watermark as well as removing the watermark from the digital picture and utilizing a jpeg compatible image - Google Patents

Description

Method and apparatus for generating digital pictures with removable watermarks

Cross reference to the application

This application was filed on day 10/2 of 2002 and is entitled "Method and apparatus for Creating A Digital Picture with Textual Material", filed on part with application number 10/261409 (currently patent US6,888,569); and is entitled "Method and Apparatus for creating a Digital Picture with a Watermark as well as Method and system for Removing the Watermark From the Digital Picture", filed with application No. 11/028,212.

Technical Field

The present invention relates to methods and apparatus for appending non-photographic material, including text and/or audio material, to digital photographs contained in image files, such as those using the JPEG format, which include displayable icons, and for transferring these files to a remote source.

Background

Current technology on the market allows cameras to take digital or non-digital pictures and attach date and time stamps to the pictures. The date and time stamp becomes permanently non-modifiable information that is attached to the washed out picture or as a pixel pasted into the digital picture. This makes it impossible for a particular viewer of the picture to see the additional data attached to the picture if they do not want to do so. Furthermore, it is not possible if someone wants to modify or remove the data.

Other prior art techniques allow users to add titles to their digital pictures. However, these titles are stored in a database on the local machine or on an internet site. If the user wants to move a picture from one environment to another or send a digital picture to another user, the title cannot move with the picture. Furthermore, if the sound file is attached to a JPEG file, the same problem exists when transmitting both the photograph and the sound.

Some issued U.S. patents generally relate to the subject matter of transferring JPEG files. For example, U.S. patent 6,014,170 issued to Pont et al is directed to an information processing apparatus that stores information in a recording unit. Each recording unit includes at least one of main image data, sub image data, and audio data. As shown in fig. 5, the main image data, the sub-image data, and the audio data are stored in a mixed form in the random access memory. The main image data includes header (header) information and image data. The header information includes index information, information indicating a type of data, information indicating a date of image capturing, information indicating a start address of an area in which the data is stored, and information indicating a data length of the image data. The data structure of the sub-image data may be similar to the image structure of the main image data. However, the patent does not describe the situation where such information-e.g., text data or other types of topical material-can be altered by an authorized user when data is transferred from one location to a second location.

U.S. patent 5,805,215 to Mizoguchi shows an information processing method and apparatus using a notebook type electronic camera. Various information related to a specific image, such as the date of the image, is automatically recorded along with the image data.

U.S. patent 5,170,253 to Horiuchi et al describes an apparatus for adding a title to an image produced by a video camera. The title to be displayed is selected by a control signal from a control signal input terminal. The patents to Mizoguchi and Horiuchi et al, when discussing the addition of material to a digital photograph, do not suggest a way to ensure that only authorized individuals can change non-photographic material after it is transmitted from a first location to a second location.

U.S. patent 6,833,865 to Fuller et al describes a digital acquisition system such as a digital picture camera using an embedded metadata engine. Although FIG. 2A includes a single memory storage device 70 used to store both image and metadata, the metadata and image data are stored in a non-JPEG file format and are not transferred in the manner described herein.

Sesek et al patent application publication 2005/0046706 shows an apparatus for acquiring an image and additional metadata about the image. In the case of this particular application, this additional data is typically directed to the geographic location of the acquired image. The image data acquisition module aggregates the photo object information in block domain 2 and stores the acquired geographic data in storage cartridge 304. Thus, it can be seen that the metadata associated with the image is stored in a different memory location.

Patent application publication 2004/0201740 to Nakamura et al describes a system for acquiring and storing images and metadata associated with each image. The image and metadata are stored in a table format as shown in fig. 6. However, the figure merely indicates that the metadata is associated with a particular image file, but is not stored within the particular image file.

A number of us patents describe the use of icons along with image data. One such patent is U.S. patent 6,774,939 to Peng. This patent describes a recording and playback (playback) apparatus provided with image data and a plurality of icons associated with the image data. The image data 50, 50A, and 50B are displayed on the monitor 42. The monitor will also have a plurality of icons 46 associated with the image data. These icons are used to control various features of the playback device. It may be noted that although memory 120 stores image file 210, the metadata associated with icon 46 is not stored along with image file 210.

Another U.S. patent provided with icons is U.S. patent 6,522,354 to kawamura et al. This patent describes a system that allows a user to delete a sound file associated with an image without knowing where the sound file is stored. The icon is displayed on the camera rather than on the image itself and is used to identify the file.

Although the use of JPEG files to create digital pictures constitutes a significant breakthrough in the field of creating real images and the ease with which digital pictures can be transmitted to different remote locations, it also poses the problem of protecting the rights of individuals who create and own the rights to these images. Several techniques have been used, primarily using watermarks to attempt to protect these protected digital pictures from unauthorized use.

For example, a conventional watermark can be provided for this purpose. A graphical watermark (i.e., not for resale) is permanently added to a copy of a digital image that can be obtained from an internet download. The watermark is translucent or transparent to allow the end user to see the appearance of the image, but prevents the image from being used due to the presence of the watermark. Once the end user has paid for the use of the digital image through a Digital Rights Management (DRM) arrangement, such as purchasing the rights to the image or obtaining a license to use the digital image, this sends the end user a link to download the original image without the watermark. However, the problem that results from this solution is that the vendor must keep two copies of the picture online and once the second picture is downloaded, the protection against unauthorized copying of the digital image is no longer present. This is particularly important because the end user may only purchase limited use of the digital image in a particular environment. Furthermore, even if the end user has full rights to use the digital image, since the digital image is transmitted over the internet, the unauthorized user(s) will be able to use the image without watermarking for their own unauthorized use. In this case it is important to note that there is no way to prevent unauthorized use of the transmitted digital image without the watermark.

A second method of protecting digital pictures from unauthorized use may be through the use of invisible watermarks. Typically, an invisible watermark consists of embedded information between pixels of a digital image in such a way that this additional information cannot be removed. Virtually all new driver licenses in the united states use this technique. A disadvantage of this method is that although unauthorized use of the electronic image can be monitored by a person electronically examining the image with the invisible watermark, unauthorized use of the image cannot be prevented because there is no visible deterrent (deterent).

From the foregoing, it will be appreciated that there remains a need for an improved system and method for attaching non-photo material to a photo that transmits the photo and non-photo material and that only allows an authorized person to modify the non-photo material at a second location remote from the first location.

It will also be appreciated from the above description that there is a need to provide a method and system for watermarking a digital picture or image to prevent unauthorized use of the digital picture, while transmitting the watermarked digital picture to an authorized user who has the ability to remove the watermark from the digital picture or has removed the watermark from the digital picture to allow authorized use of the digital picture, but while preventing unauthorized use of the digital picture.

It will also be appreciated from the above description that there is a need to provide a system and method for informing a user of information located within a JPEG file. A system must be developed that will transmit along with the JPEG file that will not be eliminated or destroyed upon the transmission of the additional JPEG file.

It will also be appreciated from the above description that there is a need to provide a system and method for adding audio material to a JPEG file that does not require the replacement of a portion of the original photograph with additional text material.

Disclosure of Invention

The deficiencies of the prior art are addressed in the present invention, which allows an individual to take a digital picture and attach textual data to the picture file itself so that the data accompanies the picture wherever it is. If the digital picture is sent over the internet or any other medium, the accompanying text (or another audio file) is always in the picture itself and attached to the digital picture as ASCII text, thereby enabling the digital picture to be edited. The information provided in the JPEG markup will include a password that only allows a particular individual(s) to edit the digital non-picture material. Furthermore, the present invention may allow an individual to modify certain information in the digital picture, but not other information in the picture. In addition, the present invention may allow a particular individual to modify one type of information while other individuals modify a different type of information. Although the present invention will be explained with respect to the JPEG file format, it is important to note that it can be applied to any image file format capable of storing metadata in an image file.

The text/audio material attached to the picture will be viewable to anyone who receives the picture. The user will determine exactly where and how the text will be displayed in the photograph. The text material can be generated as a header displayed around any border of the picture or can be directly included in the picture itself by overlaying certain pixels of the picture. However, provided the picture owner sends permission to allow, only those with the appropriate photo tag application will be allowed to modify the text when they wish. Furthermore, based on the password(s) included in the application, a particular individual may have the ability to modify all text/audio material, while other individuals can only edit portions of that material.

It is clear that the text/audio material attached to the digital picture and added to the JPEG file need not be a substitute for any part of the original digital photograph. This is particularly true with respect to audio material that would merely be appended to a JPEG file that includes the original digital photograph. In this example, the added audio portion of the file would be transmitted along with the raw digital photograph as it is inserted into the raw JPEG file, which would then include both the raw digital photograph and the added audio material, making up the augmented JPEG file. The JPEG file could include the provided text material in addition to or as a replacement for the original digital photograph.

Other individuals receiving the photograph may not need any special techniques to view the title or other type of data attached to the photograph because the creator of the additional data and/or sound will append the additional information as part of the digital photograph. Non-photo material will be provided in its natural form in the digital photo file, so this will be possible if the creator needs to modify the text.

Furthermore, since the actual text/audio data will be stored within a JPEG file, it is now possible to retrieve the text data in all of its pictures on their personal computer or other device. For example, if textual material such as photo locations is added to digital photos, the owner of those photos can retrieve only those photos taken at a particular location or of a particular person. Furthermore, if multiple photographers send the headlined material to a news agency (e.g., a newspaper), the agency can retrieve all photographs submitted by a particular photographer, or all photographs on a particular subject, or all photographs by date of capture. The combinations that can be retrieved will vary with the type of material stored in the JPEG file.

Now, it is possible, based on the invention, to place each picture in a separate database key (support) of data describing the relevant attributes of the picture, and to build up a database of pictures, where each picture has information describing the content of the picture, when combined with many other pictures tagged with photos. This way, when combined with many other pictures, this will form a database of pictures, which can be retrieved and processed as groups of pictures or as individual pictures.

The application software used to produce the desired results of attaching text titles and audio material directly to a digital photograph, viewing the photograph, and editing the photograph can reside in many different devices that will know how to edit and display any form of the digital data, such as a personal computer, handheld computer, or even a camera. Photo tagging technology may be released as a Software Development Kit (SDK) to prevent third parties from incorporating the concept of adding and editing text and audio titles to digital photos in their own products. The present invention extends the current market file format by recovering a digital picture in JPEG format by using a special reserved tag in the JPEG file for adding data needed to store metadata into the JPEG file for editing.

Once the text and audio data are provided as metadata in a JPEG file, it is important to make the presence of this metadata apparent to the recipient (receiver) of such data. Cellular phones, personal computers, blackberries (blackberries), etc. are capable of receiving, storing, and displaying JPEG images. However, since the displays of some of these devices (especially cell phones and blackberries) are so small that it is impractical to install an application with a standard Graphical User Interface (GUI) to identify and access the embedded metadata.

Each metadata type will be represented by a graphical image or icon that can be placed anywhere in the image itself, including its borders. If there are multiple metadata types or different files of the same type of metadata, each metadata type may be represented by its own icon. For example, if an image includes an audio message, an mp3 music file, and a text message, then there are three icons associated with the image.

In order for the device to be able to access and display the embedded metadata, the receiving device must include appropriate enabling software that can be readily made available to the display device, for example by downloading over the internet. Instead, the enabling software may also be installed directly at the time of manufacture of the device. This can be installed on a chip, firmware, or thin client (client) layer. Thus, once the JPEG file including the metadata as well as the icons is transmitted to the remote user, the user can "click" on the desired icon and will access its content appropriately. For example, if an icon of a voice message is "clicked," the voice message will be played through a speaker. If the text icon is "clicked on," the message will replace the image and be displayed with or overlay a portion of the image.

By the mere fact that the icons are visually associated with the image itself, the image in fact becomes a self-contained application that does not require a separate GUI because the icons on the GUI replace the GUI. If the user receives an image that includes these icons, but does not have the code needed to access the embedded content, then these messages will not be displayed. However, it is important to note that the metadata associated with each icon, as well as the metadata used for the icon display, is not corrupted, regardless of the number of times the JPEG file is to be transferred. This is true despite the fact that some devices to which JPEG is transferred cannot access the metadata, or that subsequent devices to which the data is transferred can access the metadata. This is also true since the icons are directly stored in the JPEG image together with the image data, text data, and sound data.

The present invention also addresses the problem of properly managing the digital rights of owned digital pictures (e.g., copyrighted). Although the teachings of the present application describe systems and methods for adding text material to a digital picture, the text material, although not limited thereto, is generally directed to situations in which only a small portion of the digital picture is removed and replaced by text material. Therefore, it does not address the situation: wherein the watermark is applied to a large portion of the digital picture to prevent unauthorized use of the digital picture.

The watermark is applied to the digital picture as a virtual overlay of the picture in such a way as to allow an authorized user to see the digital picture with the watermark applied thereto, but not to allow the user to make appropriate use of the digital picture from which the watermark has not been removed. The watermark is applied to the digital picture and stored in a JPEG file in a manner such that transmission of the digital picture with the watermark to an authorized end user will allow the end user to have the watermark removed, either by the end user or by the party that initiated the transmission of the watermarked digital picture. In one embodiment of the invention, if the end user is only given limited rights to use digital pictures that do not include a watermark thereon, the watermark will reappear once the limited use is completed.

It is noted that the icons associated with the metadata can also include watermark icons, allowing subsequent users to add one or more watermarks to or remove one or more watermarks from the displayed image.

Drawings

The present invention will be more fully understood and appreciated from the following detailed description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram of an environment used in the present invention;

FIG. 2 is a block diagram showing the major components of a camera used to take digital photographs;

FIG. 3 is a diagram showing a JPEG file of text material contained therein;

FIG. 4 is a diagram showing a JPEG file loaded into a memory;

FIG. 5 is a flow chart illustrating a process of editing text data tags;

FIG. 6 is a block diagram illustrating editing operations of photo tags;

FIG. 7 is a diagram showing a photograph; while

FIG. 8 is a diagram showing the photograph of FIG. 7 including text material provided therewith;

FIG. 9 is a flow chart showing the manner in which a watermark is added to a digital picture;

FIG. 10 is a flow chart illustrating the manner in which watermarks are removed from a digital picture;

FIG. 11 is a diagram showing an original digital picture;

FIG. 12 is the diagram of FIG. 11 with a watermark applied to the original digital picture;

FIG. 13 is a depiction of a JPEG file in accordance with the teachings of the present invention;

FIG. 14 is a diagram showing how icons of the present invention are stored;

FIG. 15 is a diagram illustrating exemplary icons associated with visual images; and

FIG. 16 is a diagram illustrating the use of the enablement system to display icons on a receiver.

Detailed Description

FIG. 1 illustrates an environment in which the present invention operates. Any type of standard digital camera 10 is used to take pictures. The digital picture is transferred to a first computer or similar input device 12 and finally to a second computer or similar memory device 14. The digital picture is transferred to the hard drive 18 of the first digital computer 12 by any conventional means, such as physically loading a disk into the hard drive 18, or by wired or wireless communication between the camera 10 and the hard drive 18. The computer is provided with a monitor 20 and a keyboard 22. The keypad 22 is provided with standard alphanumeric keys 24 and a plurality of standard function keys 26. The keyboard 22 is connected to standard memory contained in the hard disk drive 18 by a connection 40. The hard disk drive 18 is connected to the monitor 20 by a connection line 42.

Similar to the first computer 12, the second computer 14 includes a hard disk drive 28 having memory provided therein, a monitor 30, and a standard keyboard 32 including standard alphanumeric keys 34 and standard function keys 36. The keyboard 32 is connected to the hard drive 28 by a communication line 44, and the display 30 is connected to the hard drive 28 by a communication line 46. Information is transferred between the hard disk drive 18 of the first computer 12 and the hard disk drive 28 of the first computer 14 via any known link 38, such as a wired link, a wireless link, and the internet.

Fig. 2 illustrates generally the operating components 50 of the standard digital camera 10. These components include an image capture device 52 for capturing images, and a storage device 54 for capturing one or more images in digital format. The CPU 56 cooperates with the memory 58 to control the operation of the digital camera. The information contained in the storage device 54 is then introduced to a network interface 60 for transmission of the digital image(s) to a communication network 62, such as the internet, or any other type of wired or wireless communication.

It is an object of the present invention to allow titles to be added to photos in such a way that both the title and the photo are stored as one JPEG file, thereby enabling the entire file to be transferred from one memory device (e.g., a computer) to a second memory device (e.g., a computer). Further, the present invention will allow a person to modify textual information, the person having rights from the person who generated the photograph and the attached title material. It is also noted that the title need not be defined as textual material, but may also include sound to enrich the photograph, such as music or any other type of verbal annotation that supports sound. The sound file will be in any standard sound file format, such as WAV. Other types of text material would include a title describing the photo content, a photographer name, a copyright notice, and a timestamp. The photographer name and copyright notice may include an ASCII string with the name of the photographer and the copyright notice. It is also noted that the term "text material" as used herein includes one or more of the above metadata attributes as well as any other attributes that would fall within the scope of the present invention. Further, it is to be noted that the computer or memory device may also be a camera or a cellular phone.

The text attribute can be appended to the bottom of the photograph or embedded directly into the photograph in any form. Information relating to the location of the text material may be included in a JPEG file. Furthermore, the present invention will allow the text material to be displayed in digital photographs in a variety of fonts, colors and sizes, and display angles. In addition, the X, Y location and rotation angle of any of the above text attributes in the picture, the background color (if the text material is not embedded in the photograph), and the text color, font name, font size and font style (any combination of bold, underline, italics, etc.) are also stored in the JPEG file format.

The standard JPEG format uses multiple markers to divide a stream of JPEG files into its constituent structures. Each tag is two bytes long, the first byte always has the value FF, and the second byte contains a code that specifies the tag type. The JPEG standard is quite flexible when it involves ordering of multiple markers within a file. The only rule used is that each file must start with the start of image (SOI) marker and end with the end of image (EOI) marker. In most cases, all other indicia can be presented in any order.

For example, FIG. 3 illustrates a typical JPEG file that would appear on the displays 30 of, for example, the first computer 12 and the second computer 14. The JPEG file drawn on the display will include a main portion 70, and the digital photograph will appear on the main portion 70. Appended to the bottom of the JPEG file photo would be the photo tag portion 74 of the JPEG. The portion illustrated in fig. 3 will include title indicia 76, copyright indicia 78, and header indicia 80. Other textual material such as the name of the photographer and a description of the photograph may also appear in section 74. As indicated previously, the start of the JPEG file will include the SOI marker 82 and end with the EOI marker 84.

The JPEG format supports generic markup for third party applications for a multitude of uses. The JPEG format contains a set of 16 markers designated APP0-APP15, which are used to store application specific data. These markers will allow the storage of information beyond what is specified by the JPEG standard. While it is not important in the present invention which of these tags are used, the photo tag of the present invention will store all relevant text data in the APP10 tag. The JPEG file format does not require the use of any and all of these markers. They are simply definitions of a number of markers that can be ignored by the standard application that opens and reads the JPEG file. Since they are used for application specific data, it is quite likely that most JPEG files will not have any APP markers.

The concept of assigning numerical values to these APP tags gives 15 possible APP tags that can be used. The present invention chooses to use the APP10 marker on a completely arbitrary basis. However, if another application decides to use the APP10 tag for its own reasons, the APP10 of the present invention will be distinguished from these other APP10 tags by including an 8 byte length signature in the tag structure.

The APP10 marker used by the present invention in JPEG files will contain substantially the same data structures as shown in table I.

TABLE I

Data item	Data type	Description of the invention
			Marking	2 bytes	JPEG mark (APP10)
Data length	Character (Chinese character)	The length of data contained in the mark
			Signature	8 bytes	Photo tag standard signature, "PhoTags (photo signatures)"
Type (B)	Byte(s)	Type 1 of photo tag-header 2-no display data (text or other) 3-displayed text data
			Data of	Byte(s)	A data byte. The data structure will be extended below to the header tag and text data tag types. For the type of sound, the data is a sound bit in natural form.

The photo tag application requires three types of tags, namely a header tag, a not-displayed tag, and a displayed text data tag. The particular type of flag will be indicated in the type field of the header structure as shown in table I.

The header marker 80 (see fig. 3) simply resides before the EOI marker 84. This tag is the first tag loaded by the photo tagging Software Development Kit (SDK) and is of a fixed size so that it can be accessed correctly. The structure of the header tag is shown in table II. The header tag is used to point to the first photo tag in the file so that it can be quickly accessed, rather than sequentially scanning all tags in the file. If the user decides that the header will not be embedded in a picture but is displayed in an attachment area 76 (see fig. 3) above or below the picture, the header tag also contains the byte height of the external header appended to the JPEG file. The header tag will also contain where the title is located, password(s) to allow protection of all data in the picture and to allow an individual to change the data, background color of the title area, current photo tag version number and structure length so that it can support backward compatibility. This is important because the length of the header is read as the first piece of data when the structure is loaded and is the last bit of the header structure that is just before the JPEG EOI tag. The header tag also contains the particular photo tag version number used.

TABLE II

Data item	Data type	Description of the invention
			PtrToFirstMarker (pointer to first marker)	Double character	Pointer to location in file containing first photo tag
HeightOfCaption (height of title)	Character (Chinese character)	Height byte of the outer header area, if not used, is 0
			Caption position (title position)	Byte(s)	1-Top 2-bottom 3-Right 4-left
Cipher code	Character (8)	Password allowing protection of 8 characters of all data in a picture
			Width of frame	Character (Chinese character)	This number is greater than 0 if the user wants to attach the frame to the photo. This number will determine the pixel width of the frame.
BackgroundClr (background color)	Double character	Background color of header area, if used
			VersionNo (version number)	Character (Chinese character)	Version of current photo tag
Length OfHdr (header length)	Character (Chinese character)	The length of the structure. Therefore, we can support backward compatibility.

The undisplayed data tag can hold any dataform of a single element and would include the concept of an open field name. This means that the application can create its own field type. As shown in table III, these internal fields can include, but are not limited to, title type, photographer name, copyright notice, photo date, text description, and sound. Other types of fields may also be included, such as photo prices. Table III shows the structure of the data not to be displayed flag. The data markers allow the user to store any form of data in a JPEG file that is not used for display. The data can be used to retrieve various items in the database. It is important to note that the data need not be textual, but may also include voice data. This is important since the retrieval capabilities of the invention will no longer be limited to text data only, but will also include the possibility to retrieve audio data. Various input devices associated with the present invention, such as voice recognition devices, can be used in the retrieval process.

TABLE III

Data item	Data type	Description of the invention
			Data type	1 byte	Type of data to be maintained: 1-text 2-integer 3-floating point 4-date 5-byte string (for sound)
Name of field	Text	Field names (e.g., title, price, etc.)
			Data length	Character (Chinese character)	Length of data
Data of	Dependent on the type described in the data type	Stored actual data

The header markers 76 and copyright markers 78 are stored in the JPEG file before the header markers 80. The types of text data stored in the file may include title, time stamp, photographer name, copyright notice, and voice note. Table IV shows the structure of the test data flag. If the text is to be embedded in a JPEG file picture, it stores the bits of the picture before overwriting so that it can support editing and reconstruction of old pictures if needed. All text fields are designed to be of variable length, so there should be a length field before each text field data attribute.

Table IV shows the displayed text labels. The displayed text label stores the display attributes of a piece of text and is essentially a superset of the non-displayed labels in that it contains all the information contained in table III and all the information contained in table IV. The main difference between table III and table IV is that table IV also contains display attributes for the text. In addition, if text is to be embedded in the photograph, the displayed text label stores the bits of the photograph prior to creating any overlays. This is important so that if this is necessary, the photos and text material can be edited after the JPEG file is edited and the photos created before the overlay are reconstructed. The RGBBit stores the bits of the picture region that are overwritten. These are picture pixels that have been deleted from the photograph when the text material was embedded therein. This is illustrated in fig. 7 and 8, where typical photographs are shown. The photograph 150 shown in FIG. 7 contains the phrase "park walk" 152 provided in a rectangle 154. The photograph 160 in FIG. 8 has the phrase "park walk" deleted, which is not contained in the rectangle 162. Thus, if the photograph 150 is edited to delete the phrase "park walk," the picture pixels covered by that particular phrase will now be included in their appropriate locations in the rectangle 162. Thus, whenever text needs to be edited, by storing information in the RGBBit, the pixels are simply reloaded without leaving the text in a process called decomposition (composing) that will be explained further on. Finally, if text material is to be embedded in the photograph using a process called composition (composing) which will be explained further, the pixels with the newly edited text are then inserted into the picture again.

TABLE IV

Data item	Data type	Description of the invention
			TextColor (text color)	Colour(s)	Color of text
FontLen (font length)	Character (Chinese character)	Length of font type
			FontName (font name)Scale)	Character(s)	Character font
FontSize (font size)	Character (Chinese character)	Size of font
			FontStyle (font)	Character (Chinese character)	Bold and underline format of font
Position of	Rect	X, y, w, h of the position where the text is to be displayed in the picture
			XyRotation (xy rotation)	Character (Chinese character)	Rotating x, y degrees of text
Zrotion (z rotation)	Character (Chinese character)	Rotation of text about z-axis
			RGBBITLen (RGB bit length)	Character (Chinese character)	Length of RGB bit string
RGBBits (RGB bit)	Byte(s)	Bits of covered picture area

Referring to fig. 4, a photo tag application is loaded into the memory of one of the computers 12, 14 or the camera 10, cell phone or any other storage device so that appropriate text material can be created and appended to a JPEG file. This data can also be modified by an authorized user, as will be described. Each data type contained in the memory will be linked as a linked list whereby photo tag header 90 points to photo tag header tag 92 and photo tag header tag 92 points to photo tag copyright tag 94. As can be appreciated, other text material types will operate in the same manner, such as photographer name, time stamp, and voice note. Thus, referring to Table II, to retrieve the data in copyright indicia 94, one can start with a pointer to the first indicia and then sequentially go to the next indicia. Once a particular tag is loaded, it is possible to access and edit all of its attributes. For example, to read a copyright notice, one may traverse all nodes to reach the copyright tag and then load a text string from the copyright tag.

The present invention uses a software development kit loaded in a device in which a JPEG file including text material is created or edited and in a device in which a complete JPEG file including text material is to be transmitted. Initially, as shown in FIG. 5, a JPEG file may be loaded into a device that may be a camera, computer, cell phone, or similar tool. The photo tag software will then check whether the JPEG file has any photo tag markers inside the file. If this is the case, the position of the header tag 80 (see FIG. 3) pointing to the first tag, which in turn points to the next tag, as previously described, is determined. In step 100 or 102, if there is no photo tag marker in the JPEG file, an appropriate marker is created as shown in steps 112, 114. This is accomplished by invoking a create mark-up that has the ability to generate a new title that includes all types of text material as previously described. Once the mark is created, all of the specific attributes, such as title location, font type, font color, etc., may be set.

If the loaded JPEG file has the appropriate markers, all data markers are loaded into memory as shown in step 104. This is true whether the JPEG file has all of the markers in it when loaded or whether the markers were recently created. At this point, access to each of the data markers is now available. It is now also possible to retrieve all picture bits of the bitmap located in the JPEG file so that the picture can be displayed on the screen, as shown in step 106. At this point, any data attributes provided in the data markers can be edited, as in step 108. As will be explained later, this editing step can only be completed if the appropriate password has been entered by a possible editor. After all data attributes are reviewed and edited, the JPEG file will now be saved, if necessary, as in step 110.

The process of saving the photo tag file is accomplished by initially decomposing the image in step 116, wherein the actual photo is decomposed and then reconstructed, and removing the photo tag within the JPEG file. This means that at the end of the decomposition step 116, the photograph will appear to have returned to its original, uncapped state. This is important because all text data must be reconstructed within the photograph after it is edited.

The method of decomposition will initially include the step of traversing the data-labeled nodes. For each node, the RGBBits attributes are obtained. These bits are the original picture pixels of the area currently covered by the text. If there is no overlay, this step will not be needed. For each RGB bit, it must be re-overlaid on top of the rectangular position (second attribute in table IV). This step will effectively erase the pixels with the tagged text and will return the original rectangular pixels that were included before the text overlay. This step must be performed for all text labels that have been created in the photo tag JPEG file.

At this point, the picture must be reassembled after all old data has been removed and the image returned to its original pre-camera tag state, as indicated at step 118. This means that all text labels have to be reconstructed into the picture. This is done by traversing all the data-labeled nodes, and for each node the appropriate rectangular position must be extracted in the picture where the text is to be overlaid. This is produced by extracting the original pixel bits from the photo, which is decomposed and saved in the RGB bit attributes of the text labels. Then, text strings, fonts, colors, etc. from all attributes are written into the picture. Thus, the original picture can always be reconstructed since the picture portions following the text material are preserved. This process must be done for all text labels created in the photo tag JPEG file. Finally, the reassembled image is saved by saving the JPEG's picture pixels, saving the photo tag data tag, saving the photo tag header, and then rewriting the EOI tag, as in step 120. At this point, the JPEG file generated and including the text material can be transmitted to various devices shown in fig. 1.

As indicated previously, the present invention allows for the transfer and editing of JPEG files that include text material. However, to ensure that only authorized individuals can edit the material, the photo tagging software includes a security mechanism that allows the creator of the JPEG file to protect the data within the file from being modified by the creator's license. The security system will also decide which photo tags will be seen on the picture and how they will be viewed.

Because the header tag shown in Table II includes a password, the security aspect of the present invention will work. Thus, as shown in FIG. 6, when JPEG with a photo tag is loaded into one of the devices shown in FIG. 1 at step 130, a check is made at step 132 to determine if the user has entered a password at step 136. All editing operations will be stopped unless a valid password is entered. At step 138, the software will check to ensure that the password entered by the user contained in the header tag is an authenticated password. If this is not the case, the user will have another opportunity to enter a valid password. If no valid password is entered after a predetermined attempt, editing will not be allowed. However, if a valid password is entered at step 140, normal editing processing will be enabled at step 134.

Furthermore, as indicated previously, the software of the present invention can be structured in such a way as to allow a first person to edit all non-photographic material included in a JPEG file, while preventing a second person from modifying some of the data, but allowing the second person to change other portions of the data. This is achieved by including more than one password as shown in table II and fig. 6. To edit non-photographic material on a JPEG file, full or limited access would be given, based on assigning different passwords to different individuals. Finally, the application may be designed to prevent any editing of some data, such as a timestamp.

An embodiment of the present invention will now be described in which a watermark is created to cover all or a significant portion of a digital picture stored in a JPEG file. Typically, 75% to 90% of the digital picture will be covered by the watermark(s), although technically the entire digital picture or a relatively small portion of the digital picture may be covered by the watermark(s). For purposes of explanation, it is noted that a digital picture is created and a watermark is applied to the digital picture and stored in a JPEG file in a manner similar to the method of creating a digital picture and text material replacing a portion of the digital picture. The original portion of the digital picture that has been replaced is stored in a marked portion of a JPEG file. Similar to that embodiment, the present invention will store the appropriate material replaced by the watermark in the marker portion 74 of the JPEG file 70 as shown in FIG. 3. The material will be indicative of the material of the original digital picture that was replaced or virtually overlaid by the appropriate watermark(s). Further, a password as will be explained next is embedded in the header mark 80 of the JPEG file 70, thereby preventing unauthorized access to the original pixels. Digital pictures are created in a similar manner to the system shown in FIG. 1 using a digital camera 10 in communication with a first computer system 12, the first computer system 12 having a hard drive 18 with memory contained thereon into which the digital pictures are loaded. Note that the memory of the hard disk drive 18 will include memory capable of loading multiple software programs. It is also noted that although FIG. 1 illustrates the use of the digital camera 10 to create digital pictures to be loaded into the hard disk drive 18, any method of transferring digital pictures to the computer system 12 can be used.

Using software contained in the hard disk drive 18 or any memory device in communication with the computer system 12, a JPEG file is created that includes the raw digital picture therein. In this regard, reference is made to fig. 9, which shows a flow chart of a method in which a digital photograph is provided with a watermark. As previously mentioned, FIG. 3 and step 170 previously describe and illustrate the creation of a JPEG file including therein a raw digital picture along with JPEG marks. A software program included in the memory of the computer system 12 will allow the watermark to be virtually overlaid with the original digital photograph, as shown in step 172. Fig. 11 illustrates this original digital picture 180 as it appears on a computer monitor. This screen with the original digital picture 180 should be displayed on the monitor 20 of the first computer system 12.

The software program takes the form of a watermark generator used by an operator of the system 12. The operator can create the watermark and its font type and size. The software program may generate translucent glyphs and algorithmically reproduce the watermark multiple times across a horizontal line of the digital picture to ensure that the entire image is covered by the watermark to prevent unauthorized use of any portion of the protected digital picture. A graphic including an operator identification may then be added to the digital picture at the appropriate location. Copyright notices may also be added to the data picture in a similar manner.

A JPEG file as shown in fig. 3 would be created and would include the raw digital picture 180. Appropriate watermark metatags, along with graphical metatags, and copyright notices would also be included in the JPEG file marker. The JPEG file marker will also include the locations in the pixels in the original digital picture that were replaced by the watermark, graphics, and copyright notice to create an "altered" digital picture. The watermark will be constructed using software included in a generator that is used to virtually cover a substantial portion of the original digital picture 180. The software will include a number of types of watermarks that can be used. Once a particular watermark is selected, it will be added to the JPEG marker and cover the screen considerably, as shown by the "altered" digital pictures that include watermarks 196, 198, 200, 202, 204 and 206. These watermarks will be added by using an "add watermark" button associated with the computer monitor 20.

It must be noted that the user will only need to select the type of watermark to be applied to the original digital picture, in this case the watermark "Active Watermarks". The software will then calculate the number and arrangement of watermarks to create an "altered" digital picture 194. The location of each of these watermarks will be included in the metadata included in the JPEG marker at step 172. At this point, a digital picture 194 with the appropriate watermark will be created, as shown in FIG. 12.

Further features of the present invention will enable an owner or other individual associated with the original digital picture to include additional graphics, such as a company logo 192 to be revealed in both the original digital picture 180 and the "altered" digital picture 194. The add graphics button 188 in fig. 11 can be used to add the graphics 192 to the original picture 180 and the remove graphics button 210 in fig. 12 will be used to remove the graphics 192 from the "changed" digital picture 194. In addition, the appropriate copyright notice 182 is added to the original digital picture 180 and will remain in the "changed" digital picture 194. Depending on the software used, the inclusion or removal of this copyright notice may be handled separately by the operator of computer 12 or may be given way to the ultimate recipient of the "changed" digital picture 194. The removal of the copyright notice may be accomplished using a remove copyright key 190.

Due to the fact that it is important to maintain control over the use of the original digital picture 180, authorization to change the picture content, including removing the watermark(s), must be strictly maintained. Thus, at step 176, the encryption password will be added to the JPEG mark, such as the header mark 80 in FIG. 3. Once the JPEG file, including the original digital photograph, the "changed" digital picture, the appropriate metadata associated with the watermark, graphics, copyright notice, and encryption password, is added to the JPEG file to create a complete JPEG file, the complete JPEG file is saved at step 178.

The resulting JPEG file will contain an original photograph that is viewable to a user using any operating system capable of viewing the JPEG file. It will also include watermarked text virtually superimposed over a substantial portion of the original digital picture. The JPEG file will also contain graphical material such as logos and copyright notices that will or will not be hidden from view.

The original pixels of the digital picture without the watermark will be stored in the JPEG file in a compressed and encrypted format. Thus, a user modifying the pixels in an unauthorized manner using any standard editing software will destroy the original digital picture so that it will never be restored. Thus, only the "changed" digital picture as shown in fig. 12 will be left for an unauthorized user.

Once the JPEG file or files shown in FIG. 3 are created for the digital picture or pictures and appropriately tagged, the completed JPEG file(s) are transferred to end user computer system 14 shown in FIG. 1. This step is shown in fig. 10, and fig. 10 illustrates the removal of the watermark from the original digital photograph. Once the JPEG with the appropriate photo tag is sent to the end user at step 212, the end user will be able to view the "changed" digital picture 194 on his monitor 30. This "altered" digital picture 194 has a substantial portion of the original digital picture obscured by the watermark provided thereon. At this point, if the end user or any unauthorized user attempts to print a digital picture, the digital picture will be provided with the watermark thereon.

Two alternatives will be used to provide the end user with access to the original digital picture 180. In a first alternative, the password is entered by the original user at step 214 when the end user contacts the creator of the original digital image, and the creator recreates the original digital image without the watermark using the embedded and encrypted password provided in the JPEG header. The original digital picture will then be seen by and printed by the end user without the watermark appearing thereon. It is noted that the logo 192 and copyright notice 182 will typically remain in the original digital picture printed by the end user. It is important to note that the resulting original digital picture will only be stored in the end user's computer memory. The end-user is then able to use the original digital picture as he or she desires, as long as the licensing agreement between them and the operator or copyright owner allows it.

A second alternative to allowing the end user to view and print the original digital picture 180 without the inclusion of the watermark is to transmit an active ticket application to the end user that includes a password therein for unlocking the digital picture. Thus, when an end user enters a password, it will allow the end user to remove the watermark using button 208 of FIG. 12 and remove the graphic using button 210. Once unlocked, the watermark and graphics disappear, but are actually hidden from the viewer. Copyright notices will still typically appear. It is noted that in certain situations, the end user can remove the copyright notice using button 190. However, it is important to note that this removal of copyright notices may be beyond the end-user's control based on the software application provided to the end-user. It is also important to note that a licensed end user will only wish to use a portion of the original digital photograph 180. If so, the original user typically desires to have copyright notice 182 rendered even in a small portion of the original digital picture 180. Thus, when any portion of the original digital picture 180 is downloaded and printed, software included on the original user's computer and transmitted to the end user will automatically insert the copyright notice 182. The operator will have the option of routinely resetting the lock without notifying the end user (in which case the watermark and graphic will appear on the image) and changing the password originally sent to the end user, preventing the end user from deleting the watermark.

Returning to fig. 10, based on any of the above choices, properly entering a password at step 214 hides the watermark on image 216. Thereafter, based on the particular software program, a determination is made at step 218 as to whether a copyright notice needs to be added. If the copyright notice is to be added, step 220 creates copyright metadata and displays it on the pixels of the image. If the copyright notice is not to be printed, the original digital picture will be reassembled at 222 and appear on the end-user screen shown in FIG. 11.

The present invention prevents unauthorized removal of watermarks from an original digital picture because the original pixels of the original digital picture are currently stored in a marked area of a JPEG file. If someone tries to delete the label in the marked area, an unauthorized user will unknowingly destroy the original digital picture and the watermark will become a permanent part of the image that deletes the original pixels. Further, since the hidden text data is provided in the markup part of JPEG and an unauthorized user attempts to delete the metadata, the original owner can detect that the hidden information has been removed and can conclude that there is an attack.

Fig. 13 conceptually depicts a JPEG image included in the storage device 54. The type of information contained in the JPEG image has been described and illustrated previously with respect to tables I, II, III and IV. Although generally described as an image, the JPEG information 300 depicted in FIG. 13 will include textual information 302, visual information 307, and audio information 306. Along with the visual information stored in portion 314, textual information 302 and audio information 306 will be stored indestructibly in the respective portions, as shown in tables III and IV, among others. The metadata will be stored in the various invisible parts of the JPEG's, as shown by tags 1(308), tags 2(310), and tags 3 (312). Tag 1 will typically include metadata wrapped as an XML form. The tag 2(310) will use an XSL style sheet that allows the data to be displayed as a document on the receiving device. Up to 64K of other special data will be displayed in tags 3 (312). The entire package is transmitted as a viewable JPEG and using standard protocols such as MMS and SMTP. Once received and displayed by the receiving device, some or all of the visual metadata title will be immediately visible. As explained earlier, since the metadata and the visual information that the metadata is to replace will be stored as a single memory, the metadata in the form of text material can be removed at will without destroying the text material or the visual material. Thus, no information is lost and thus can be replaced at will, regardless of the number of JPEG 300 transmissions made to each receiver.

Fig. 14 embodies a manner in which metadata including icons is directly stored in JPEG format. This format is very similar to the format described in tables I, II, III and IV as described previously. In this example, a 2MB sound item 318 is associated with a sound icon 320. As explained previously, the sound item 318 can be added to a JPEG file without replacing any part of the raw data photograph. Similarly, text item 322 is associated with text icon 324. These icons, as well as the image, sound and text items, are all stored in a JPEG file and can thus be transmitted along with all file elements. While this example illustrates the use of a sound icon 320 and a text icon 324, it can be appreciated that many different kinds of icons will be able to be used based on the type of information stored in the JPEG file.

As previously described, each image marker is stored in a standard JPEG marker APP10 and is identified by a signature such as "photo markers" (PhoTags). It is also noted that standard JPEG markers can be 64 kbytes at the most.

Since the sound file in the example drawn in fig. 14 is very large, the sound file is split into two sound files capable of accommodating many individual APP tags when saving images. When the image is reloaded, all the unassociated sound files are reassembled into the original sound stream. The sound icon 320 and the text icon 322 are represented by their own icons. These icons are stored as separate JPEG markers using the techniques described previously in this application. The sound icons are linked to sound data by items having the same field name (e.g., "music"). As shown in fig. 14, the storage size of the text icon is actually larger than the stored text data, and thus will occupy several JPEG marks, while the text data will not occupy several JPEG marks.

The various icons 334, 336, 338, 340, and 342 are shown in fig. 15 with respect to the visual image 330. These icons are used to identify the individual metadata and, as previously mentioned, are non-destructible visual metadata. The user can choose to turn the icon off completely at any time using the appropriate input keys available in all receivers. The respective icons will be applied as "thumbnail" 332 provided below the image 330 shown in fig. 15. It can be appreciated that these thumbnail icons 332 can be provided anywhere within the visual image 330 or surrounding the image 330.

It is important to note that an enabler device must be provided in the receiving device to allow each icon to represent the metadata associated therewith. The enabler 344 is software downloaded from the internet or provided to the display device in various ways.

When any of the icons 334, 336, 338, 340, and 342 are used, code is sent to the enabler 334 to access and play or display the appropriate metadata in accordance with the teachings of the present invention. Note that the code to be used to play or display the appropriate metadata may also be embedded in the JPEG image.

Each icon will have multiple display states based on the condition of the icon or the condition of the receiver itself. For example, if the receiver does not have the appropriate enabler software code, the icon will be displayed in one way (e.g., appear shaded), or not displayed at all. Alternatively, if the display device does include the appropriate enabler 334, the icon will be illuminated or appear a different color than if the display device did not include the appropriate enabler software. Finally, once a particular icon is enabled, it may be illuminated in different ways. It is also important to note that more than one icon may be used at any one time. For example, if the user is provided with graphical information as well as voice information, the icons 336 and 342 can be used simultaneously or sequentially. The present invention will also have the ability to display a watermark using watermark icon 338.

Alternatively, with respect to watermarking, the present invention may use a system where "clicking" on the watermark icon 338 would not be sufficient to remove the watermark from the visual image 330 or add the watermark to the visual image 330. In this case, a prompt will appear on the display screen asking the user to enter a particular password. The password can be programmed to automatically match a particular code or number. For example, if the display device is a cellular telephone, the password would be a telephone number stored in the "SIM" chip of the telephone itself. This would therefore limit the use of the image to a single phone with a matching SIM chip therein. If the image is sent to a different phone, the watermark will not be enabled or disabled. It may therefore be the case that if a user with the correct telephone sends it to the second display device, the watermark will automatically appear or disappear again.

As can be appreciated, the present invention has many markets and applications. It can be used as an entertainment device in an individual's home, and can be used by retailers to distribute information to customers. Further, the teachings of the present invention can be used for auctions, stock imaging, image messaging, online learning, inventory, and law enforcement or other myriad uses. For example, when used in the real estate domain, real estate agents can embed property descriptions and update prices as well as property status. With respect to commerce, retailers are able to emphasize items offered for sale in detailed specifications and offers. Law enforcement agencies are able to transmit "portable" media databases to various handheld devices in the field as well as to radio-equipped vehicles.

All metadata and objects are stored in a standard JPEG file. All devices capable of receiving and viewing JPEG files are capable of receiving enabled images. Therefore, unlimited titles, text files or forms up to 64 kbytes may be used. The sound file and the graphic object can be provided in any file format. Devices that receive embedded JPEG images but cannot use them can still then forward them to other devices that can view the embedded content. Furthermore, the entire packet can be transmitted to the target device as standard JPEG using standard protocols.

It will be appreciated that, although embodiments of the invention have been described in detail for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims.

Claims (69)

1. A system for allowing an operator to create a watermark to be applied to a complete digital image comprising n pixels:

a first device for generating a first image file of said complete digital image comprising an image portion and a label portion; and

a watermark generation device associated with the first device for generating at least one watermark for semi-transparently covering a portion of the full digital image, the at least one watermark being composed of m pixels, wherein the m pixels are less than n pixels of the full digital image, the watermark generation device replacing m pixels of the full digital image with the m pixels of the at least one watermark to create an altered digital image, the altered digital image including n-m pixels of the full digital image and m pixels of the at least one watermark, the watermark generation device replacing m pixels of the full digital image corresponding to m pixels of the at least one watermark in the image portion of the image file with the m pixels of the at least one watermark, and including m pixels of the at least one watermark in the marked portion of the first image file Pixel-wise to the m pixels of the complete digital image to produce a second image file.

2. The system of claim 1, wherein the at least one watermark covers a significant portion of the complete digital image.

3. The system of claim 1, wherein the watermark generation device is provided with software that allows an operator to select a particular watermark to overlay the complete digital image, the software generating a plurality of the watermarks to overlay a portion of the complete image.

4. The system of claim 1, further providing a device that allows an operator to transfer the second image file to an end user.

5. The system of claim 3, further providing a device that allows an operator to transfer the second image file to an end user.

6. The system of claim 4, wherein a second device is provided for inserting a password in the marked portion of the second image file to prevent an end user from generating the complete digital image if the password is not communicated to the end user.

7. The system of claim 5, provided with a second device for inserting a password in said marked portion of said second image file to prevent an end user from generating said complete digital image if said password is not communicated to said end user.

8. The system of claim 6, wherein the password is transmitted directly to the second image file without any intervention by an end user.

9. The system of claim 7, wherein the password is transmitted directly to the second image file without any intervention by an end user.

10. The system of claim 6, wherein the password allows an end user to generate the complete digital image only once.

11. The system of claim 7, wherein the password allows an end user to generate the complete digital image only once.

12. The system of claim 6, wherein the operator generates an application that is transmitted to an end user, the application being provided with the password, thereby allowing the end user to use the password multiple times to generate the complete digital image.

13. The system of claim 7, wherein the operator generates an application that is transmitted to an end user, the application being provided with the password, thereby allowing the end user to use the password multiple times to generate the complete digital image.

14. The system of claim 6, wherein the operator can remotely modify a password included in the tagged portion of the retrieved image file.

15. The system of claim 7, wherein the operator can remotely modify a password included in the tagged portion of the retrieved image file.

16. The system of claim 1, wherein the operator is provided with a notification device for attaching a copyright notification to the full digital image, the copyright notification being included in the marked portion of the second image file.

17. The system of claim 16, further comprising a device for allowing said operator to transfer said second image file to an end user, said operator further provided with a second device for inserting a password in said marked portion of said second image file to provide an end user to generate said complete digital image if said password is not transferred to the end user.

18. The system of claim 17, further using means for transmitting a password to said second image file, thereby allowing said end user access to said complete digital image.

19. The system according to claim 18, provided with means allowing the end-user to download only a part of the complete digital image.

20. The system of claim 19, wherein the portion of the complete digital image includes the copyright notice.

21. The system of claim 2, wherein the at least one watermark covers 75% to 90% of the complete digital image.

22. The system of claim 1, wherein the first and second image files are JPEG files.

23. A method of allowing an operator to create a watermark to be applied to a complete digital image comprising n pixels, the method comprising the steps of:

generating a complete digital image;

generating a first image file of the complete digital image comprising an image portion and a label portion;

generating at least one watermark that semi-transparently covers m pixels of the complete digital image, where m < n;

replacing m pixels of the complete digital image with the m pixels of the at least one watermark to create an altered digital image, the altered digital image comprising n-m pixels of the complete digital image and m pixels of the at least one watermark;

including replacing the m pixels of the full digital image with the m pixels of the at least one watermark in the image portion of the first image file to produce a second image file including the altered digital image and a second marked portion.

24. The method of claim 23, wherein the first and second image files are JPEG files.

25. The method of claim 23, wherein the at least one watermark covers 75% to 90% of the complete digital image.

26. The method of claim 23, further comprising the step of transmitting said second image file to an end user.

27. The method of claim 26, further comprising the step of inserting a password in the second indicia portion to prevent unauthorized use of the complete digital image.

28. The method of claim 27, including the step of transmitting the password directly to the second image file.

29. The method of claim 27, further comprising the steps of generating an application including the password therein and transmitting the application to the end user.

30. The method of claim 23, further comprising the step of generating a series of watermarks to cover a portion of the complete digital image based on an operator selection of a single watermark pattern.

31. The method of claim 27, further comprising the step of having the operator modify the password remotely.

32. The method of claim 23, further comprising the step of inserting a copyright notice in the complete digital image.

33. A method for creating an augmented media file, comprising the steps of:

creating an original digital photograph having a number n of digital pixels, the number n of digital pixels including a first portion having m digital pixels;

transferring said original digital photograph to a first storage device provided in a microprocessor, said microprocessor being provided with a first data input device;

editing the original digital photograph using the data input device by replacing the first portion containing m digital pixels with a second portion of m digital pixels of non-photographic material, thereby producing a changed digital photograph;

creating an augmented media file by storing the altered digital photograph and the first portion of digital pixels as a single file, wherein the first portion is stored within a photo tag label appended to the altered digital photograph;

creating an icon associated with the second portion of m digital pixels;

storing the icon within the photo label tag.

34. The method of creating an augmented media file as recited in claim 33, further comprising the steps of:

creating the augmented media file by replacing portions of the m digital pixels comprising the original digital photograph with additional portions of m digital pixels of non-photographic material, the replaced portions of the original digital photograph and the augmented file being stored in the photograph tag label;

creating a separate icon for each of the additional portions of m digital pixels; and

storing each of the separate icons within the photo tag label.

35. The method of creating an augmented media file as recited in claim 33, further comprising the steps of:

generating an audio file associated with the original digital photograph;

storing the audio file in the photo tag markup file;

creating an icon associated with the audio file;

and storing the icon in the photo label mark file.

36. The method of creating an augmented media file as recited in claim 34, further comprising the steps of:

generating an audio file associated with the original digital photograph;

storing the audio file in the photo tag markup file;

creating an icon associated with the audio file;

and storing the icon in the photo label mark file.

37. The method of claim 33, comprising the steps of:

transferring the augmented media file to a storage device having a display device and software for displaying the icon.

38. The method of claim 37, comprising the steps of:

the icon is displayed as a thumbnail associated with the displayed visual image.

39. The method of claim 38, further comprising the step of activating the icon to display the second portion of m digital pixels.

40. The method of claim 39, wherein the icon is associated with a watermark, comprising the steps of:

providing a password to a user; and

using the password in conjunction with the icon to generate the watermark that appears on the augmented media file.

41. The method of claim 40, wherein the display device is a cellular telephone, comprising the steps of:

providing the password within the cellular telephone;

providing the password within the photo tag label;

comparing the password provided within the cellular telephone with the password provided within the photo tag label; and

when the password provided within the cell phone and the password provided within the photo tag label match, the watermark is automatically generated when the associated icon is activated.

42. The method of claim 33, further indicating the step of storing the second portion of m digital pixels within the photo tag label appended to the altered digital photo.

43. An augmented media file for displaying a modified photograph on a display device, the modified photograph originally containing a number n of digital pixels used to create an original photograph, the n digital pixels containing a first portion, the first portion within the n digital pixels containing m digital pixels, the augmented media file comprising:

an altered photograph having the first portion comprising m digital pixels replaced by a second portion of m digital pixels of non-photographic material;

a photo tag label affixed to the altered photo to create an augmented media file, the photo tag label further comprising a first portion of m digital pixels, wherein the altered photo and the photo tag label form a single augmented media file; and

an icon associated with the second portion of m digital pixels, the icon stored in the photo tag label and displayed as a thumbnail when a photo is displayed on a display device, activation of the icon displaying the second portion of m digital pixels.

44. The augmented media file of claim 43, further comprising:

a plurality of additional portions containing m digital pixels of the original photograph to be stored in the photograph label tag;

a plurality of icons, one for each of the plurality of additional portions containing m digital pixels, each of the plurality of icons being displayed as a thumbnail when a photograph is displayed on the display device, each of the icons being stored in the photograph label indicia, activation of each of the plurality of icons displaying its respective additional portion of m digital pixels.

45. The augmented media file of claim 43 further comprising audio material provided in the photo tag label and an icon associated with the audio material, the icon stored in the photo tag label.

46. The augmented media file of claim 43, wherein the icon is associated with a watermark and further comprising a password provided within the photo tag label.

47. The augmented media file of claim 43 wherein the augmented media file is a JPEG file.

48. A system for creating an augmented media file containing photo and non-photo material in a single file, the system comprising:

a camera that produces an original digital photograph containing n digital pixels;

means for transferring the original digital photograph to a first storage device, the first storage device including a microprocessor having a first display, the original digital photograph including a first portion including m digital pixels, the original digital photograph creating a media file within the first storage device;

a first data input device connected to the first microprocessor for replacing the first portion of the digital photograph with a second portion of m digital pixels of non-photographic material, thereby producing a modified digital photograph;

apparatus for creating an augmented media file, the augmented file comprising a modified digital photograph having the second portion of m pixels replacing the first portion of m digital pixels, the augmented media file comprising a photograph tag label comprising the first portion of m digital pixels, wherein the modified photograph and the photograph tag label form a single augmented media file, the apparatus further provided with an icon associated with the second portion of m digital pixels, the icon stored in the photograph tag label.

49. The system of claim 48, further comprising a transmission device for transmitting the augmented media file to a remote device provided with a second display for displaying a photograph and an icon.

50. The system of claim 49, further comprising m number of additional portions of a plurality of digital pixels and a plurality of additional icons, each associated with one of said additional portions of m digital pixels, each said additional portion and each said additional icon stored in said photo tag label.

51. The system of claim 50, wherein the remote device is provided with software for displaying the icon.

52. The system of claim 51, wherein each of the icons is displayed as a thumbnail associated with the displayed visual image.

53. The system of claim 52, wherein there is a watermark on the additional portion, and further wherein there is provided a password associated with a single user in the photo tagging.

54. The system of claim 50, further comprising audio material provided in the photo tag label and an icon associated with the audio material, the icon being stored in the photo tag label.

55. The system of claim 53, wherein the remote device is a cellular telephone provided with the password associated with a single user.

56. The system of claim 52, wherein each of the thumbnails changes its appearance after they are activated.

57. The system of claim 48, wherein the augmented file is a JPEG file.

58. A method of creating an augmented media file comprising the steps of:

creating an original digital photograph having a plurality of digital pixels;

forwarding the raw digital photograph to a first storage device provided in a microprocessor, the microprocessor being provided with a first date entry device, the raw digital photograph being stored as a JPEG file including a tag label;

editing the JPEG file by adding additional material to the JPEG file to create an augmented media file, the additional material being added to the tag label; and

forwarding the augmented file to a remote receiving device.

59. The method of claim 58, further comprising the step of:

creating an icon associated with the additional material; and

storing the icon in the tag label.

60. The method of claim 59, wherein the additional material is text.

61. The method of claim 58, wherein the additional material is audio.

62. An augmented media file for displaying a digital photograph containing a plurality of pixels, the digital photograph being stored in a JPEG file containing a plurality of tag markers, the augmented file comprising:

a digital photograph provided in the JPEG file containing a tag label;

additional non-photo material added to the tag label of the JPEG file; wherein the augmented file containing the digital photograph and the additional non-photographic material can be transferred in its entirety to a remote receiving device.

63. The augmented media file of claim 62 further comprising an icon associated with the additional non-photo material, the icon stored in the tag label of the JPEG file.

64. The augmented media file of claim 63 wherein the additional material is text.

65. The augmented media file of claim 63 wherein the additional material is audio.

66. A system for creating an augmented media file containing photo and non-photo material in a single JPEG file containing tag markers, the system comprising:

a camera that produces an original digital photograph containing a plurality of pixels;

means for transferring the raw digital photograph to a storage device, wherein the raw digital photograph is stored in a JPEG file containing a tag label; and

means for creating additional material to be added to the raw digital photograph and stored in the JPEG file, wherein the JPEG file including the raw digital photograph and the additional material is transferred to a remote receiving device as a single file.

67. The system of claim 66, further comprising:

means for adding an icon to the JPEG file, the icon associated with the additional material.

68. The system of claim 67, wherein the additional material is text.

69. The system of claim 67, wherein the additional material is audio.