WO2016069300A1 - Photobook and photoalbum having a flush photo montage on the front cover and methods for manufacre thereof - Google Patents
Photobook and photoalbum having a flush photo montage on the front cover and methods for manufacre thereof Download PDFInfo
- Publication number
- WO2016069300A1 WO2016069300A1 PCT/US2015/056130 US2015056130W WO2016069300A1 WO 2016069300 A1 WO2016069300 A1 WO 2016069300A1 US 2015056130 W US2015056130 W US 2015056130W WO 2016069300 A1 WO2016069300 A1 WO 2016069300A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- photobook
- photo
- cover
- strip
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/438—Presentation of query results
- G06F16/4387—Presentation of query results by the use of playlists
- G06F16/4393—Multimedia presentations, e.g. slide shows, multimedia albums
Definitions
- Embodiments of the present invention broadly relate to systems and methods for customized media-based projects.
- systems and methods are provided for generating a photo-based project that can be printed as a photobook, which has a photo strip in a channel on the front cover, so that an attached photo strip, having a photo montage or photo collage, is essentially flush with the front cover.
- Other embodiments of the present invention relate to customizations of content including multimedia files, photos, text, and stylistic parameters when ordering a media-based project online via an electronic commerce software application.
- photo-books In the field of electronic commerce (e-commerce), there are interactive websites that assist users in creating media-based projects such as photo-books, photo-calendars, photo-cards, and photo- invitations. Such interactive websites allow users to upload photos and interact with the websites to create photo-based projects that are customized to user's preferences.
- a media-based project creation utilizes photos, videos, audio-videos, pictures, and/or text.
- photobook shall be synonymous with any type of media-based product or project.
- Prior art photo-based project web sites come with various drawbacks.
- a common drawback with conventional photo-based project websites is the amount of time required by users (Internet users or customers) to create a photo-based project.
- Users must often participate in a variety of functions and processes in order to complete a photo-based project. For example, users usually must sort through the photos they desire to upload, remove the photos that are not adequate or properly focused, choose the photos to be uploaded, group photos together by topic or location, order the photos by chronological order, and crop or focus photos on the relevant portions of said photos. Performing the above processes can take undue time and labor. Further, there are corrections and modifications that users must make to photos before the photos can be used in a photo-based project.
- Prior art products for a photobook creation are often based on a theme or stylistic parameter. However, they often do not have any bearing on the inter-relationship of layouts, themes and designs so applied, and on how relevant photos are grouped together in the project. Prior art products also lack photo feature extraction, object detection, and image analysis to intelligently customize the photobook for the user.
- Prior art products also lack a layering of photos based on photo feature extraction, object detection, and image analysis.
- an Internet user or customer may need to spend undue time and labor in manually customizing layouts, colors, themes, stylistic parameters, and captions of photos and text to be able to finally design and print-by-order a satisfactory photobook.
- Undue time and labor in such a manual customization is a barrier to decide on an online purchase of a photobook.
- Such a barrier is also known as a "friction point.”
- Such barriers or friction points are further compounded by unfriendly graphical user interface (GUI) of prior art products and photo editing software, and result in additional inefficiencies and errors in printing a media-based product, such as a photobook.
- GUI graphical user interface
- prior art books may have a photo printed on a protective book jacket of the book or on a hardcover of the book itself.
- school textbooks need to withstand heavy use by students and may have a textbook cover photo concerning the textbook subject that is printed as an integral part of the protective plastic wrapping of the textbook cover.
- a protective clear plastic wrap may be used over a photo on a photobook cover.
- there is a printing of a photographic image directly on the protective cover material or the jacket of the book cover and as a result the printed photographic image contributes nothing to the thickness of the protective cover material or to the thickness of the book jacket.
- a personalized photograph may be inserted underneath a plastic window on top of the photobook cover, or may be inserted underneath a border frame on top of the book cover. In some cases, the personalized photograph is inserted underneath a plastic window bordered by a frame on the top of the photobook cover. If a photograph is attached to a photobook cover without a protective covering, then an object may catch an edge of the photograph and initiate a sudden detachment or gradual peeling of the photograph from the cover of the photobook cover.
- the present inventors have recognized that it would be an advancement in the state of the art to address these and other problems with prior art methods for attaching photographs to photo albums, photobooks, and the like.
- an embodiments of the present invention which include a photobook, made by a process comprising the steps of printing a photo-strip having one or more images; producing a front cover by compressing a compressible cover material with a stamp to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; wrapping the compressible cover material with the channel around a cover board material to make the front cover; and setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
- a photobook made by a process comprising the steps of printing a photo-strip having one or more images; producing a front cover by wrapping a compressible cover material around a cover board material to make the front cover; compressing the compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; and setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
- a preferred embodiment of the present invention is a photobook, comprising a front cover with a channel having an area less than the area of the front cover; a photo strip set into the channel; a back cover; a connection between the front cover and the back cover; and pages for photographic images, wherein the pages are situated between the front cover and the back cover, and wherein the photo strip set into the channel displays a single or multiple photographic images which are essentially flush with the front cover.
- the photo strip set into the channel can have a location anywhere on the front cover, for example a location selected from the group consisting of in a central location, a right side location, a left side location, an upper location, a lower location, and a combination thereof.
- the photo strip is wrapped around a front cover edge and terminates on a backside of the front cover. In other embodiments of the present invention, two or more ends of the photo strip are wrapped around edges of the front cover and terminate on the backside.
- the photo strip is set into a channel on the front cover and ends of the photo strip are wrapped around one to three edges of the front cover, and terminate on the back side of the front cover. In some embodiments of the invention, the channel from the cover continues around the cover edge so that the photo strip is essentially flush with the surface of the edge of the cover as well as with the cover.
- photobook embodiments of the invention preferably have a photo strip covering only part of a cover.
- a photo strip can be located on the spine or an interior side of a cover with this photo strip set into a channel on the spine or interior side of the cover so that the photo strip is essentially flush with the surface of the spine or the interior cover.
- a photo strip on a cover of the photobook can be of any imaginable shape, for example a square, a rectangle, a triangle, pentagon, a hexagon, a polygon, a circle, an ellipse, a rounded vertices polygon, a shape having a hole or a combination of these shapes.
- the front cover or the back cover or both of a photobook may contain a rigid board material to stiffen the cover.
- the front cover and/or the back cover of the photobook comprises a flexible cover material so that the cover is flexible.
- the front cover and the back cover of the photobook each contain at least a rigid board cover material wrapped over with a flexible cover material.
- the spine of the photobook flexibly connects the front cover to the back cover, the spine comprising the same flexible cover material as the front and back covers.
- the spine of the photobook contains a rigid board material beneath the flexible cover material which stiffens the spine.
- the front cover, the back cover or both covers are padded using a padding material between the rigid board cover material and the flexible cover material.
- the padding has been added only to the back cover of the photobook, and the front cover contains additional rigid cover board material so that the thickness of the front cover will match the thickness of the back cover.
- Some embodiments of the present invention are a photobook, comprising a front cover with a centrally located channel; a photo strip; a back cover; a spine comprising a flexible cover material; and pages for photographic images between the first cover and the second cover, wherein the front cover further comprises a first rigid board cover material over which is wrapped a flexible cover material, and wherein the back cover comprises a second rigid board cover material and further comprises a padding material over which has been wrapped the flexible cover material, and wherein the photo strip is set into the centrally located channel, the photo strip displaying one or multiple photographic images which are essentially flush with the cover.
- photobook embodiments of the preset invention have no spine which connects the front cover directly with the back cover, and the photobook has been made with its inside pages interconnected and connected to the covers.
- the photobook has a spine connecting the front cover and the back cover and also the inside pages are interconnected and fixed to the covers.
- the present invention also includes methods for making a photobook of the present invention.
- the present invention provides a method for making a photobook, the photobook comprising a photo strip set in a channel on a cover of the photobook, the channel having an area less than the area of the cover, the photo strip displaying one or multiple photographic images, so that the one or more photographic images is essentially flush with the cover, the method for making said photobook comprising (a) cutting pieces of a rigid cover board material for a front cover, and a back cover of the photobook; (b) cutting a piece of a flexible cover material to a size and to a shape for a wrapping of the piece of the flexible cover material over the pieces of the rigid board cover materials for the front cover and the back cover; (c) using a portion of the flexible cover material between the front cover and the back cover for making a spine connecting the front cover and the back cover; (d) making the channel by pressing a channel shaped stamp into the piece of the flexible cover material, the channel shaped stamp having so the channel has
- the flexible cover material is preferably pressed to form a channel for the photo strip before the flexible cover material is wrapped around the rigid cover board material.
- the pressing may require an elevated temperature to soften the flexible cover material.
- the photobook has a cover comprising a rigid cover board wrapped with a flexible cover material, wherein the flexible cover material is pressed to create a channel before the flexible cover material is wrapped around a rigid cover board of a cover so that one or more photographic images on the photo strip is essentially flush with the cover.
- the method for making the channel comprises pressing a piece of the flexible cover material with a stamp having the shape and the dimensions of the photo strip to be set into the channel.
- the stamp pressure is about 1 psi (pounds per square inch) to about 20,000 psi.
- one method for making the channel in a photobook cover further comprises adding rigid cover material strips to the rigid cover material beneath sides of the channel in the flexible cover material.
- methods for making the channel in a photobook cover comprise using one or more layers of same rigid cover materials alone or in combination with one or more different layers of flexible cover materials.
- the cover materials of the channel in a cover of the photobook may comprise rigid cover materials, flexible cover materials, or any combinations thereof. Any method for shaping, casting, or molding of a cover material as a means for creating a channel wall is contemplated for some embodiments of the present invention.
- Any known method for forming a channel in a flexible cover material is contemplated for some embodiments of the present invention.
- One or a plurality of cover materials may be employed in some embodiments of the present invention to create one or more channels in one or more covers of the photobook so that one or more photo strips can be set into the one or the more channels so that the one or more photographic images of the one or the more photo strips are essentially flush with the cover having the channel.
- the method for making the photobook further comprises padding a cover with a padding material to create a soft cover touch.
- Any form of padding may be suitable, including a flexible foam plastic sheet. Padding may be placed over the rigid cover board of the front cover or back cover. When padding is used in one cover, then other cover can be matched in thickness by a method of using a thicker rigid cover board material in the other cover.
- the method for making the photobook further comprises adding a flexible cover material strip along at least the width of the inside surface of the spine.
- Adding the cover strip along the inside of the spine helps to reduce or remove or smooth out any uneven aspect of a surface along the spine caused by the use of a flexible cover material, including for example, a thick cover material such as a synthetic leather cover material that is wrapped over a photobook edge such as from an outside surface to an inside surface of the photobook.
- a flexible cover material including for example, a thick cover material such as a synthetic leather cover material that is wrapped over a photobook edge such as from an outside surface to an inside surface of the photobook.
- the photobook has a feature selected from the group consisting of a photo strip which displays a collage of images, a back cover which contains a padding material, a spine which comprises a flexible cover material, a spine which is rounded, a binding decoration which is located between the spine and the binding of the pages, an end sheet before the binding of the pages, an end sheet after the binding of the pages, an end sheet which displays a collage of images, pages which display a collage of images, a page which displays a single image, two consecutive pages which have a portion of the same image, and any combination of features thereof.
- the spine may comprise a layer of a flexible cover material such as the kind used as a wrapping for the cover of the photobook.
- the decoration may be located between the spine and the ends of the inside pages of the photobook.
- the binding decoration may be flexible or stiff, of any shape, and of any material.
- the biding decoration may be a woven fabric of the same color as the flexible cover material and can give the photobook spine to have a traditional book binding appearance.
- the inventors were trying to figure out the best way to automatically insert content (photos, text, and so on) into a design in such a way as for it to appear the way the user intended in a book that they would want to purchase, with minimum input from the user.
- content would be placed in a default way that would not take into account photo information and saliency information.
- the prior art would also not aggregate this information in a useful way.
- Most automatic content insertion in auto-fill algorithms involved centering photos perfectly in photo slots regardless of what was in the photo, thus potentially cutting off important content.
- the present invention helps automate the building of photobooks into a buyable product from the get-go, with little need for tweaking.
- the present invention offers several unique advantages over the state of the art: (a) positions photos more accurately within photo slots; (b) determines a photo's importance and uses it in the decision making process for where to place the photo on the page; (c) grouping algorithm takes into account a photo's importance, and will single a photo out if it is so important that it should end up on its own page; and (d) once the combined saliency of a photo is properly determined, the system will add the appropriate buffer to the photo so the system can fit it into a photo slot in a natural way. Other features and benefits are described later.
- the present invention saves time and labor in creating media-based projects by automating content insertion into projects.
- the present invention eliminates singularly practiced manual methods in prior art photo projects and automates a plurality of novel methods for creating such projects.
- Such novel methods include sorting of photos, removing inadequate or undesirable photos, selecting correct photos, grouping of photos, ordering of photos in correct order, and cropping or focusing on an area of the photos, and presenting a book design to a user.
- Other embodiments of the present invention automate a plurality of photo edit processes that were traditionally not available in a single prior art product; such edit processes include orienting photos, re-coloring photos or performing a color correction on photos, correcting red-eyes, and correcting photos for brightness or contrast.
- the present invention places photos more accurately within photo slots of a design page in a photobook.
- the present invention also auto-detects areas of saliency and computes photo importance for using photos intelligently in grouping and highlighting purposes.
- the present invention presents to a user a final design of a photo project, without the user's involvement in the designing aspect of the project.
- the present invention is a method, a system, and a computer-readable storage medium storing executable program code for a process comprising the steps of automating a creation of a media project in a client-server environment via a graphical user interface (GUI) of a user device, after a user of said device uploads media files from said device to a server, the method comprising (1) inserting a first set of contents, said contents comprising media files, text and user inputs, into a design of said media project, wherein said first set of contents comprises information derived though an analysis performed by said sever, and wherein said first set of contents is automatically inserted into said design without said user's interaction to said design by using automated rules of inserting contents; (2) grouping of said media files into a group by using automated rules of grouping, and inserting said group of media files into said design, wherein said design is a spread, and wherein said rules of grouping comprise timestamps, image similarities, approximations of image sequence, image scores by repetition of images, and
- Another embodiment of the present invention includes a method, a system, and a computer- readable storage medium storing executable program code for a process comprising the steps of (1) establishing a client-server connection between the server and a computing device over said communications network; (2) receiving a plurality of images from said computing device through an upload to said server, and storing said plurality of images in a data repository connected to said server; (3) performing an image analysis on said plurality of images to derive automatic image information from said plurality of images; (4) grouping said plurality of images based on said automatic image information; (5) inserting content into said photo-project based on the grouping of said plurality of images to create an editable photo-project; (6) presenting said editable photo-project to a user of said computing device; (7) monitoring user interactions on said computing device while the user is manipulating said plurality of images to derive observed image information from said user interactions; and (8) automatically modifying an arrangement of content in said editable photo-project based on said observed image information to produce a final recommended design of the photo
- the automatic image information may be derived though one or more algorithms for saliency detection, face detection, photo similarity detection, and/or object detection.
- the grouping of said plurality of images may be achieved through at least one of timestamp analysis, image similarity analysis, and photo importance scoring.
- the observed image information may be derived through analysis of user interactions comprising areas of interest selection, photo importance selection, text input, panning, zooming, moving, and/or swapping of photos by said user.
- the insertion of content may be achieved through one or more algorithms for a ranking of images based on a plurality of image heuristics for calculating image importance, a buffer calculation for a best fitting content into a photo slot, an area of interest selection on a photo, a score for inserting a photo into a photo slot of a layout, a determination of a best design of a design page, a dynamic text insertion algorithm, and/or a positioning of a photo into a photo slot having a focal box.
- Yet another embodiment of the present invention includes a method, a system, and a computer-readable storage medium storing executable program code for a process comprising the steps of (1) inserting a first set of content into a design of said media project, wherein said first set of content is based on information derived though an analysis performed by the server, and wherein said first set of content is automatically inserted into said design without said user's interaction to said design by using automated rules for insertion of content; (2) grouping of said media files into a plurality of groups by using automated rules of content grouping, and inserting said groups of media files into said design, wherein said design is a spread, and wherein said rules of grouping comprise at least image similarity detection and image scores by repetition of images; (3) presenting to said user at said GUI an automated design of said media project, wherein said automated design is prepared through said automated rules of inserting content and said automated rules of content grouping, and wherein said rules comprise algorithms comprising at least saliency detection rules, face detection rules, object detection rules, photo similarity detection rules, and/
- the first content and the second content are inserted automatically by the server at a background computer process that is invisible to the user utilizing the GUI.
- the steps may be practiced in any sequence.
- the observed user interactions may comprise panning, zooming, moving, and/or swapping of said media files by the user.
- the process sends an order to print said automated design or said final recommended design to a printer, wherein said printer prints said automated design on paper, and wherein said paper is selected from a plurality of configurable print sizes.
- the processes described here may be practiced over the Internet network.
- Fig. 1A is a network configuration diagram in which the present invention may be practiced.
- Figs. lB-1 and 1B-2 comprise a flow diagram showing a method of background uploading of media files, according to one embodiment of the invention.
- Fig. 1C is a diagram showing a Graphical User Interface (GUI) of selecting media files, according to one embodiment of the present invention (this embodiment sometimes known as "MONTAGE").
- GUI Graphical User Interface
- Fig. ID is a diagram showing a GUI of uploading media files, according to said embodiment of the invention.
- Fig. IE is a diagram showing a GUI of building a media-based project, according to said embodiment of the invention.
- Fig. IF is a diagram showing a GUI of selecting a theme, according to said embodiment of the invention.
- Fig. 1 G is a diagram showing a GUI of designing a front cover, according to said embodiment of the invention.
- Fig. 1H is a diagram showing a GUI of arranging text and images, according to said embodiment of the invention.
- Fig. 1J is a diagram showing a GUI of previewing a design, according to said embodiment of the invention.
- Fig. IK is a diagram showing a GUI of inserting text, according to said embodiment of the invention.
- Fig. 1L is a diagram showing a GUI of ordering a photobook, according to said embodiment of the invention.
- Fig. 2A is a block diagram illustrating a network architecture of a system for facilitating a creation of photo-based projects over a communications network, in accordance with one embodiment of the present invention.
- Fig. 2B is a block diagram illustrating a data flow of the system for facilitating the creation of photo-based projects over a communications network, in accordance with one embodiment of the present invention.
- Fig. 2C is a flowchart depicting a general control flow of the process for facilitating photobook creation over a communications network, according to an embodiment of the present invention.
- Fig. 2D is an illustrative flowchart of an image analysis process, according to one embodiment of the present invention.
- Fig. 3 is a flowchart showing an automatic insertion of content into a design of a media-based project, according to an embodiment of the present invention.
- Fig. 4 is a flowchart showing a first automatic and then a second manual insertion of content into a design of a media-based project, according to an embodiment of the present invention.
- Fig. 5 is a flowchart showing a simultaneous automatic and manual insertion of contents into a design of a media-based project, according to an embodiment of the present invention.
- Fig. 6A illustrates a diagram showing a relationship between a canvas, a design, a layout, a designset, and a lay outset, according to an embodiment of the present invention.
- Fig. 6B illustrates a diagram showing an interrelationship of a theme, a layout, and a design, according to an embodiment of the present invention.
- Fig. 7 is an exemplary GUI of an editing interface displaying a photo spread across multiple pages (front cover, back cover, and spine), according to an embodiment of the present invention.
- Fig. 8 is a block diagram illustrating an embodiment of the present invention having an asset recommendation system, according to another embodiment of the present invention.
- Fig. 9 is an architectural diagram illustrating a printing network, according to an embodiment of the present invention.
- Fig. 10 depicts a frontal view of an example photobook embodiment of the present invention before a photo strip has been set into a channel on the cover of the photobook.
- the channel passes over a top edge of the front cover, traverses a central area on the front cover, and passes over a bottom edge of the front cover.
- Fig. 10A depicts an expanded frontal view of a section of the top edge of the example photobook embodiment of the invention depicted in FIG. 10.
- Fig. 10B depicts a front cover, a back spine, and back cover of an example photobook embodiment of the present invention before a photo strip has been set into the channel.
- Fig. 11 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover. A photographic image is not depicted on the surface of the photo strip.
- Fig. 12 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover.
- Lines in the photo strip are hypothetical borders for eight photographic images on the photo strip arranged as a a montage or a collage.
- the photo strip is wrapped over top and bottom edges of the front cover and is essentially flush with the bottom edge and top edges of the cover.
- Fig. 13 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover.
- Lines in the photo strip are hypothetical borders for twelve photographic images arranged as a montage or a collage.
- the photo strip depicted in Fig. 13 is wider and larger than the photo strip depicted in Fig. 12.
- Fig. 14 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into a channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for four photographic images.
- Fig. 15 depicts a frontal view of an example photobook embodiment of the present invention with a horizontally-oriented photo strip set into the channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover.
- Lines in the photo strip are hypothetical borders for eight photographic images arranged in a rectangular collage. The photo strip and channel continue over the right edge of the front cover so that the photo strip remains essentially flush with the edge.
- Fig. 16 depicts a frontal view of an example photobook embodiment of the present invention with a circular photo strip set into a circular channel in the center of the front cover.
- the photo strip is essentially flush with the cover.
- Figs. 17A, 17B, and 17C depict sequential steps of one method that could be used for making a channel into which a photo strip could be set.
- Fig. 17A depicts a stamp above a cover material.
- Fig. 17B depicts the stamp pressed into the cover material.
- Fig. 17C depicts a channel remaining after the stamp has been withdrawn from the cover material. The shape of the stamp, and the depth that the stamp presses into the cover material, will determine the shape and depth of the channel.
- Figs. 18A and 18B depict sequential steps in a method for adding a strip of a cover material to the inside surface of a spine of a photobook according to one embodiment of the present invention.
- Fig. 19 depicts a frontal view of a soft rounded spine photobook embodiment of the present invention before a photo strip has been set into a channel on the front cover.
- Fig. 20 depicts a frontal view of a soft spine photobook embodiment of the present invention with a photo strip set into the channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for six photographic images. The photo strip is wrapped over top and bottom edges of the front cover and is essentially flush with the bottom edge and top edges of the cover. DETAILED DESCRIPTION OF THE INVENTION
- Photobook and “photo-book” are names for a printed book with photo images.
- the photobook has a front and a back cover with one or more printed images on a photo strip.
- the photo strip is mounted on a cover of the photobook so that the photo strip is essentially flush with the cover.
- Example embodiments of photobooks of the present invention are depicted in Figs. 10-16, and Figs. 19-20.
- a media-based project is not limited to a photobook.
- a photobook is a physical manifestation of the project in the form of a printed product. A physical manifestation may not be necessary in other embodiments, such as, an online video album, an online photo album, and/or combinations thereof. In the description that follows, a photobook is used as a representative of any type of media-based project.
- Photo strip and “photo-strip” is a strip with a single or multiple printed images of people, places, or things on the surface of the strip.
- the strip is generally smaller than the area of a cover of a photobook.
- the photo strip in preferred embodiments of the present invention has a thickness of less than 5 millimeters, more preferably less than 2 millimeters.
- the photo strip is generally mounted into a channel on a cover of a photobook so that the photo strip is essentially flush with the surface of the cover of the photobook.
- Channel refers to a depression that is made in the cover, generally on the front cover's outside surface of the photobook as depicted in Figs. 10 and 19.
- the channel In the channel is mounted a photo strip and thus the channel needs to have a depth slightly more than the thickness of the "photo strip” to be mounted in the channel.
- the channel has a horizontal area with a shape matching the shape of the photo strip to be set in the channel.
- the channel can be made with suitable depth and area dimensions by stamping the front cover or its "cover material” using a "stamp” in a stamping method depicted in Fig. 17.
- stamp stamp
- stamp-press stamp-press
- channel-shaped stamp refer to a device with a flat surface in the shape of the area of the photo strip.
- a “stamp press” is a machine which can be used to exert a pressure through the stamp upon a surface which may be a cover board material, or a flexible cover material.
- An example stamp and stamping method are depicted in Fig. 17. The method for making the channel in the flexible cover material or another cover material of the photobook by stamping a shaped stamp into a deformable material is also sometimes called de-bossing.
- a stamp is but one illustrative method of compressing the compressible cover material to make the channel, and other methods are within the scope of the present invention.
- “Spine” is defined as the outermost "connection between the front cover and back cover” of the photobook and appears as the surface of the back of the photobook. It bridges the front cover to the back cover.
- the spine can be constructed of any material. In some embodiment of the present invention, the spine comprises a flexible material.
- the spine can be rounded by any means.
- An example of a flexible and rounded spine photobook is depicted as spine 160 in Figs. 19-20.
- the spine may have a stiff material as a part of the spine so that the spine only flexes only at its margins where it is attached to the front and back covers.
- An example embodiment of a stiff spine photobook is spine 102 depicted in Figs. 10-16.
- the spine is separate from the binding of the pages. The separation between the spine and the binding may increase when the photobook is opened.
- the spine may be directly connected to the binding of the pages of the photobook.
- Collage and montage refers to the appearance that results when the printed images are selected to appear in combination on the photo strip used on a cover as well as may also occur on the pages of the photobook or on end sheets.
- Cross board material refers to a structural material, for example, a thin wooden chip board which is used to stiffen the cover of a photobook. All known materials for making book covers are intended to be within the scope of the term “cover board material,” including but not limited to cardboard, wood, plastic, and so on.
- “Padding material” refers to a soft material, such as but not limited to a soft foam, that may be used in a cover of the photobook. It may be present between a cover board material and a flexible cover material to improve the softness and feel of a cover of the photobook.
- Flexible cover material is a flexible fabric, such as but not limited to a synthetic leather material, which can be used as a surface wrapping for covers of the photobook.
- Compressible cover material and “compressible material” is meant to include any compressible material, such as but not limited to plastic foam, foam, leather, synthetic leather, and so on, that is used on a cover of a photobook to create a channel in the compressible cover material for holding the photo-strip.
- MONTAGE”, MOSAIC, and MIXBOOK are trademark names carrying embodiments of the present invention, and hence, the aforementioned trademark names may be collectively or interchangeably used in the specification and drawings to refer to the products/services offered by embodiments of the present invention. Because all of the products are created from media-files using a graphical manipulation system provided by the inventors as a network-based software application, the relative terms descriptive of the overall process and for the providing company are MONTAGE, MOSAIC, and/or MIXBOOK.
- MONTAGE, MOSAIC, and/or MIXBOOK may be used in this specification to describe the overall media-based product creation and editing process of the invention, the website, mobile app, or web-app through which the process is made accessible to authorized users, the photobooks or printed media-based projects themselves, and/or the service- providing company.
- embodiments of the present invention relate to systems and methods for automating insertion of content into designs of media-based projects.
- Content refers to images/photos, text, and user inputs.
- a user needs to select photos from a user-device, such as a laptop or mobile device. Once the photos are selected, the photos are uploaded to a server. Upon image analysis of the uploaded photos by the server, the server provides essential tools to the user-device to build a photobook.
- a server operates within a client-server architecture of a computer system.
- a server comprises computer hardware and software to serve computational requests of other programs or clients.
- a client comprises a user-device, such as, a laptop, a smartphone, or a desktop PC.
- a server performs tasks on behalf of clients.
- a server often provides services via the Internet or other network.
- prior art products for a photobook creation are often based on a theme or stylistic parameter. They often do not have any bearing on the inter-relationship of layouts, themes, designs and content so applied, and on how relevant photos are grouped together in the project. Prior art products also lack photo feature extraction, object detection, and image analysis to intelligently customize a photobook for the user.
- Prior art products lack a layering of photos based on photo feature extraction, object detection, and image analysis.
- an Internet user or customer may need to spend undue time and labor in manually customizing layouts, colors, themes, stylistic parameters, and captions of photos and text, to be able to finally design and print-by-order a satisfactory photobook.
- Undue time and labor in such a manual customization is a barrier to decide on an online purchase of a photobook.
- Such a barrier is also known as a "friction point.”
- Such barriers or friction points are further compounded by unfriendly graphical user interface (GUI) of prior art products and photo editing software, and result in additional inefficiencies and errors in printing a media-based product, such as a photobook.
- GUI graphical user interface
- the present invention addresses and resolves these problems by automating designs of photo- based or media-based projects.
- the present invention automates photo feature extraction, object detection, image analysis, photo grouping, and/or content insertion.
- the present invention saves time and labor in creating media-based projects by automating content insertion into the projects.
- the present invention eliminates manual methods of prior art photo projects and automates efficient methods for such projects, the automated methods including sorting of photos, removing inadequate or undesirable photos, selecting correct photos, grouping of photos, ordering of photos in correct orders, and/or cropping or focusing on an area of photos.
- inventions of the present invention automate the processes that include orienting photos, re-coloring photos or performing a color correction on photos, correcting red-eyes, and/or correcting photos for brightness or contrast.
- the present invention places photos more accurately within photo slots of a page in a photobook.
- the present invention also auto-detects areas of saliency and computes photo importance for using photos intelligently for grouping and highlighting purposes.
- the present invention presents to a user a final recommended design of a photo project, without the user's involvement in the designing aspect of the project.
- a final design is printable. Once printed, the final design is shippable to the user.
- the media-based project comprises a physical manifestation of the project, resulting in a printed photobook, photo album, or other physical manifestation of the project.
- a photobook may be printed on a canvas or photo quality paper and bound together as a book.
- the media-based project also includes an online product comprising audio, video, image and text - all packaged together as a single cohesive item or product for display or sharing through online means or electronic means, or for ordering a physical version of said online product.
- Fig. 1A shows a schematic diagram of a network configuration 1A100 for practicing embodiments of the present invention (this embodiment is sometimes known as "MONTAGE").
- a user-device or devices may be connected to the Internet using a wireless network or wired network.
- a user-device may be a smartphone 1A102, laptop 1A104, desktop PC 1A106, or tablet 1A108.
- the wireless network comprises a cellular tower 1A110, or a wireless router 1A112.
- User-devices are connected to servers comprising a web server 1A114, an application server 1A116, and a database server 1A118.
- the servers are connected to a user-device through the wireless network, or the wired network 1A120.
- the wired network 1A120 or the wireless network may employ technologies and protocols comprising Ethernet technology, Local Area Network (LAN), Wide Area Network (WAN), and optical network, and the like.
- Figs. lB-1 and 1B-2 show a flow diagram of an exemplary embodiment 1B200 showing a method of background uploading of media-files while an Internet user is customizing a media-based project, which in this embodiment is a photobook.
- An Internet user (hereinafter "user") starts the process of creating a media-based project (hereinafter “project") using a user-device at step 1B202.
- the term photobook or project may be used interchangeably.
- the user logs in at step 1B204 using the user-device.
- the user selects original media files or images for creating a photobook.
- the user is provided a workspace (also called "editor") where the user can practice or utilize various embodiments of the present invention to customize, edit or create the photobook.
- a workspace also called "editor”
- the client- device analyses the images.
- the original images are resized to smaller sized image files and a meta-data is extracted from each original image.
- Meta-data is a type of data that is relevant to or embedded in an image. Meta-data comprises digital data about a media file.
- meta-data is an Exchangeable Image File format data, or "EXIF" data (also in some cases known as "Exif ' data).
- EXIF is a standard that specifies a format for images, sound, geolocation data (provided by a Global Positioning Receiver or GPS receiver attached to a camera), and ancillary tags used by digital cameras, smartphone cameras, scanners, and other systems that handle image and/or sound files recorded by digital cameras.
- EXIF data comprises a broad spectrum of data, including, but not limiting to, date and time information of images taken, GPS location of images, camera settings (camera make and model, image orientation or rotation, aperture, shutter speed, focal length, metering mode, and ISO information), a thumbnail of the image, a description of the images, and/or copyright information thereof.
- the resized image files and the meta-data are uploaded to the server.
- the resized image files are smaller in size as compared to the larger sized original images, but not so small as to be un-useable by the server for image analysis.
- the resized images are immediately provided to the user-device for building the project.
- step 1B214 After the resized images and the extracted meta-data of the original images are uploaded to the server, step 1B214 will follow to begin uploading the original images to the server in a background computer process at the user-device.
- the server uses the uploaded resized images and the meta-data, at step 1B216, the server performs a server-side image analysis.
- An image analysis comprises image optimization algorithms, image ranking algorithms, feature or object detection algorithms, use case formation techniques, layout, and/or photobook design formation techniques, as described in greater detail below.
- the server converts the resized uploaded media files into multiple sizes or multiple (plural) compression levels (the term “compressed media file” or “compressed image” is used herein to refer to a media file that has been in some way modified by the server and of a different size from the resized file uploaded from the client to the server).
- the server provides to the user- device a compressed image, or an image of a size relevant to the scope of the project (photobook).
- the server converts a resized image into many different sizes (called “compressed files” herein) so that each size may be called by the user-device depending on the scope of the project.
- the scope of the project may comprise different sizes and different resolutions of image files, wherein the server performs the server-side image reduction to prepare images of different sizes as needed by the project. While the server performs the server-side image reduction using the resized image files, the original image files are still uploading in the background computer process.
- the server provides analytical data to the user.
- the user pulls relevant image data comprising image files and analytical data from the server.
- the user continues customizing (building) the photobook or project.
- the original image files may or still may not be uploaded [completed upload] to the server. If the original image files are uploaded to the server, the user continues building the photobook as shown by step 1B224.
- the photobook customization still continues at step 1B234. If the original image files are uploaded and the photobook customization continues (and thus completes), the photobook will be ready to order at step 1B224. At step 1B232, the user can place an order to print the photobook. Once the photobook is ordered and printed at step 1B232, the embodiment 1B200 ends at step 1B246.
- the user still customizes (and thereby completes) the photobook at step 1B234, and is ready to order the photobook at step 1B236.
- the user places the order at step 1B238.
- the embodiment 1B200 provides the user a widget to upload [begin a new upload or complete the ongoing upload] the original image files that the user selects.
- Step 1B238 allows the user to order the photobook before the upload of the original image files completes.
- step 1B240 the embodiment 1B200 provides the user an optional uploading mechanism of original image files if the user does not have time to wait until the original images are uploaded to the server before placing an order to print the photobook.
- step 1B240 can happen before step 1B238 and/or before the step 1B236; that is, a widget to upload original images may be provided to the user-device before placing an order, or before the order is ready.
- steps discussed herein may be switched, overlapped, or altered as long as the spirit of the present invention is consistent with the plurality of embodiments described herein.
- An uploading widget may comprise a software application that the user can download from an embodiment of the present invention.
- the user can begin uploading or continue uploading the original image files using the installed widget.
- the original image files are uploaded to the server with the aid of the widget.
- the server can send the order to print the finished product. The embodiment thereby ends at step 1B246.
- Fig. 1C is a screenshot embodiment 1C300 showing a method of selecting original media files or images at a user-device.
- a user-device may select original images by clicking a button 1C302, labeled as, for example, pick photos.
- a button is a soft-button in a software embodiment.
- image(s) and photo(s) are used interchangeably in this specification.
- Fig. ID is a screenshot embodiment 1D400 showing an upload of original images to a workspace of a user-device after the user-device selects or picks images.
- a workspace at a user- device is also called an editor.
- a progress bar 1D402 shows the number of images uploaded or uploading to the workspace.
- An uploaded image 1D404 is displayed in the workspace.
- the embodiment 1D400 also shows an uploading image 1D410 when the image is being uploaded.
- the embodiment 1D400 demonstrates client-side resizing of original image files.
- the image 1D404 is a resized image which is smaller than the original image.
- a button 1D406 allows the user to add more images.
- the user can click a button 1D408 to create a design of a photobook using the uploaded photos in the workspace.
- the button 1D408 reads "create my Montage," where MONTAGE is a trademark name, but not a functional name, of the button 1D408.
- Fig. IE is a screenshot embodiment 1E500 in which the present invention does an analytical work or analysis work to build a photobook.
- An exemplary symbol 1E502 teaches that the present invention is building a photobook by performing the analytical work or analysis work, such as an image or video analysis.
- Fig. IF is a screenshot embodiment 1F600 that displays a theme selection of a project or photobook after images are uploaded to the workspace (editor) of a user-device. All pages of the photobook are displayed in the workspace. In a page 1F604 of the photobook, an arrangement of the images is shown. This arrangement is auto-populated by the present invention after an image analysis.
- the server performs the image analysis for auto-populating the arrangement. For auto- populating the design of the book, the server may use features such as, cropping, zooming, relevant images grouping, and other image editing processes.
- the embodiment 1F600 allows the user to select a theme 1F606.
- the instruction at 1F602 allows the user to select or choose the theme 1F606. Once the user is satisfied with the theme and arrangement of the images in all pages of the photobook, the user can click the button 1F608 to continue to the next step of the present invention.
- Fig. 1 G is a screenshot embodiment 1 G700 that shows a cover page of a photobook and an arrangement of images therein.
- the cover page 1G706 uses at least an image 1G704.
- a plurality of images can be used in the cover page 1G706.
- a server auto-populates a first design of the cover page 1G706.
- the user has an option to further customize the cover page 1G706 by using any of the uploaded images. Standard image editing features may be applied in customizing each image on the cover page 1G706 of the photobook.
- the workspace displays the theme of the design at 1G702.
- the workspace further allows the user to upload more photos at 1G708.
- the button 1G710 allows the user to scroll the uploaded images.
- a checkout button 1G712 allows the user to order and print the photobook.
- Fig. 1H is a screenshot embodiment 1H800 that shows a page of a photobook for rearranging text and images therein.
- the page 1H802 shows an arrangement of the uploaded images on the page.
- the user can shuffle images on the page 1H802 by using a shuffle button 1H804, insert text on the page 1H802 by using an insert-text button 1H806, and rearrange pages of the book by using a rearrange button 1H808.
- the user can additionally use one or more of the uploaded images to customize the design of the page 1H802.
- the user can flip to a next page of the book by clicking or pressing an arrow button 1H810.
- Fig. 1J is a screenshot embodiment 1J900 showing a method of previewing and rearranging design pages of a photobook.
- a design page 1J904 has a theme and at least an image. A plurality of images can be used in the design page 1J904 and the theme is selected by the user.
- the design page 1J904 comprises page 1 and page 2 of the photobook.
- the message at 1J902 explains that the workspace provides a method to rearrange the pages of the photobook.
- the user can click a done button 1J906 to save the rearrangement.
- Fig. IK is a screenshot embodiment 1K1000 showing an insertion of text to describe an image.
- Text can be inserted into a page of the photobook to describe an image or a group of images.
- a storyline or narrative of work may be added with images using text.
- a narrative can be added as shown in the region 1K1002.
- Fig. 1L is a screenshot embodiment 1L1100 showing a method of ordering a project or photobook.
- a user can select a specification of the project to order. If the user is not happy with the customization of the photobook 1L1102, the user can go back to the editor or workspace by clicking 1L1104 to further edit/customize the pages of the photobook.
- a specification may be a size or dimension of the photobook.
- a sample specification of the project by size or dimension is shown at 1L1106. Each specification may have a different price for order.
- the user can enter a shipping address as shown at 1L1108.
- the user can further continue with payment options and order a printed photobook.
- a standard payment processing gateway maybe implemented in the present invention to order a printed copy of the photobook.
- Fig. 2A shows an illustration of a block diagram showing a network architecture of a system 2A200 and method for facilitating the creation of photo-based projects over a communications network in accordance with one embodiment.
- Fig. 2A shows various components coupled with network 2A206, which can be a circuit switched network, such as the Public Service Telephone Network (PSTN), or a packet switched network, such as the Internet or the World Wide Web, the global telephone network, a cellular network, a mobile communications network, or any combination of the above.
- PSTN Public Service Telephone Network
- a prominent element of Fig. 2A is the server 2A202 associated with a data repository or server-side database 2A204 and further coupled with network 2A206.
- Server 2A202 collects, processes and manages data from the users 2A210 and further manages the server-side database 2A204, which holds the master copy of data that is served to fulfillment entity 2A250 and client device 2A220.
- Fig. 2A further includes user 2A210 and his or her client computing device 2A220, which may be a desktop computer, a common computer terminal, or mobile computing devices such as smart phones, mobile phones, tablet computers, handheld computers, laptops, or the like.
- Fig. 2A shows an embodiment of the present invention wherein networked client computing device 2A220 interacts with server 2A202 and server-side database 2A204 over the network 2A206.
- Server 2A202 may comprise one or more servers, workstations, desktop computers, computer terminals, workstations, mainframes, mobile computing devices, smart phones, mobile phones, handheld computers, laptops, or the like.
- Server 2A202 and client computing device 2A220 include program logic comprising computer source code, scripting language code, or interpreted language code that may be compiled to produce an executable file or computer instructions, or that may be interpreted at run-time, wherein the computer source code performs various functions of the present invention.
- Server 2A202 may include a software engine that executes applications as well as delivers applications, data, program code, and other information to networked computing devices, such as device 2A220.
- Fig. 2A further shows a database or repository 2A204, which may be a relational database comprising a Structured Query Language (SQL) database stored in a SQL server.
- the repository 2A204 serves data from a database, which is a repository for data used by server 2A202 and device 2A220 during the course of operation of the invention.
- Database 2A204 may be distributed over one or more nodes or locations that are connected via network 2A206.
- the database 2A204 may include user account records, user records, and multimedia records that include images, video, audio and like.
- Fig. 2A shows only one server 2A202, one database 2A204, one client device 2A220, and customer 2A210
- the system of the present invention supports any number of servers, databases, client devices, and customers connected via network 2A206.
- server 2A202 is shown as a single and independent entity; in one embodiment, the functions of server 2A202 may be integrated with another entity, such as one image provider 2A240, fulfillment entity 2A250, device 2A220, and/or the database 2A204.
- server 2A202 and its functionality can be realized in a centralized fashion in one computer system or in a distributed fashion wherein different elements are spread across several interconnected computer systems.
- Fig. 2A further shows fulfillment party 2A250, which performs product fulfillment or order fulfillment services, i.e., the process of fulfilling the obligation of server 2A202 (or the entity represented by server 2A202) to send user 2A210 one or more photo-based project products 2A252 that the customer has ordered, purchased, or requested from the server 2A202 (or the entity represented by server 2A202).
- Fulfillment party 2A250 may receive orders for the photo-based project products, print the photo-based project products, bind the photo-based project products, package the products, and then ship the ordered products 2A252 to the end customer, such as user 2A210.
- the server 2A202 may interface with fulfillment party 2A250 to effectuate the delivery of purchased products to the customers after payment has been effectuated.
- fulfillment party 2A250 is shown as a single and independent entity, in one embodiment of the present invention, the functions of fulfillment party 2A250 may be integrated with another entity, such as server 2A202. Further, the functionality of fulfillment party 2A250 may be realized in a centralized fashion in one computer system or in a distributed fashion wherein different elements are spread across several interconnected computer systems.
- Figs. 2B to 2C depict, among other things, the data flow (2A260) and control flow (2C200) in the process for facilitating the creation of photo-based projects over a communications network 2A206, according to one embodiment.
- the process 2C200 of the disclosed embodiments begins with step 2C202, wherein the user 2A210 provides, via his or her device 2A220 over the network 2A206, at least a plurality of images or photos 2B264 to the server 2A202 for storage in the database 2A204.
- the images or photos 2B264 are provided to server 2A202 via a graphical user interface executing on the device 2A220.
- the images or photos 2B264 are provided to server 2A202 for storage in the database 2A204 via TCP/IP and/or HTTP over network 2A206. Subsequently, server 2A202 stores the images or photos 2B264 in the database 2A204 as records 2B266. In one embodiment, the records 2B266 are stored in association with an identity for user 2A210 or in association with a user record for user 2A210.
- the user 2A210 provides, via his device 2A220 over the network 2A206, a use-case identifier 2B266 to the server 2A202 for storage in the database 2A204.
- the use-case identifier 2B266 is provided to server 2A202 via a graphical user interface executing on the device 2A220. In another embodiment, the use-case identifier 2B266 is provided to server 2A202 for storage in the database 2A204 via TCP/IP and/or HTTP over network 2A206. Subsequently, server 2A202 stores the use-case identifier 2B266 in the database 2A204 in association with records 2B266. In one embodiment, the use-case identifier 2B266 is stored in association with an identity for user 2A210 or in association with a user record for user 2A210.
- various photo analysis processes may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220.
- the photo analysis processes comprise identifying similar images, identifying faces in the images, identifying objects in the images, identifying undesirable images, and identifying relevant portions of the images.
- the identification of faces and objects may be accomplished via object recognition and face recognition libraries.
- the identification of similar images may be accomplished via an analysis and comparison of color, focus, brightness, faces, objects and the like in each image, as described in greater detail below.
- the identification of undesirable images may be accomplished by identifying images that are out of focus or contain too little light for a proper exposure.
- edge detection may be used to detect any sharp edges, if any.
- the identification of the relevant portions of an image may be accomplished by identifying the relevant portions of an image, such as faces and objects that may have been recognized. By applying saliency filters, a bounding box may be drawn around a focus of an image. Thus, if the relevant faces or objects are located on the sides of the image, the image may be zoomed or cropped to highlight the identified relevant area.
- the photo analysis processes may further include identifying images with an incorrect orientation, identifying images with incorrect color, brightness or contract, and/or identifying images with red-eye.
- various photo correction processes may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220.
- the photo correction processes comprise orienting images that have been identified as having an incorrect orientation to an appropriate orientation, adjusting images that have been identified as having an incorrect color, brightness or contract to the correct color, brightness or contract, and removing redeye from images that have been identified as having red-eye.
- various use-case specific heuristics may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, based on results of said photo analysis, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220.
- the use-case specific heuristics comprise grouping images that have previously been found to be similar, grouping images having identical or similar faces (based on the faces that were previously identified), grouping images having identical objects (based on the objects that were previously identified), removing undesirable images (based on the images that were previously identified as undesirable), and cropping images to highlight relevant portions of said images (based on the relevant portions of images that were previously identified).
- heuristics that correspond to the use-case identifier 2B266 are utilized. Different use-cases may require different heuristics. For example, a use- case identifier 2B266 that identifies a wedding would result in heuristics that group images having the faces of the bride and groom (based on the faces that were previously identified), group images having the wedding cake or the altar (based on the objects that were previously identified), and crop images to highlight portions of said images having the faces of the bride and groom (based on the relevant portions of images that were previously identified). Instructions 2B268 in database 2A204 may instruct server 2A202 on which heuristics correspond to each potential use-case identifier 2B266 provided by user 2A210.
- identifiers may be applied to each face to determine, for example, that person A shows up in 85% of the photos, person B shows up in 73% of the photos, and person C shows up in 20% of the photos. Therefore, person A and B are most likely the bride and groom.
- various additional functions may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220.
- the additional functions may comprise reading embedded date or location metadata from the images, performing a lookup of additional location data based on the embedded location metadata, and generating text boxes to be printed below said images upon creation of the photo-based project 2A252.
- the additional functions may also include ordering the images or photos 2B264 stored in the database 2A204 in chronological order, based on the time the images were taken or created.
- the server 2A202 reads an image in EXIF format, a standard that specifies the formats for images and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image files recorded by digital cameras.
- EXIF format may store date and time information of the image, camera settings of the image, thumbnails of the image, descriptions of the image, and copyright information for the image.
- the server 2A202 may read the location for the image from the file and generate a text box reflecting said location to be printed below said image upon creation of the photo- based project 2A252.
- the server 2A202 may read the location for the image from the file (in a GPS coordinate format), perform a lookup to find a corresponding text string that matches the GPS coordinate, and generate a text box having said text string to be printed below said image upon creation of the photo-based project 2A252.
- the server 2A202 may read the date and time for the images from their files and place the images or photos 2B264 in chronological order, based on the time the images were taken or created.
- step 2C214 the creation process is concluded and the resulting photo-based project product is ordered, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220.
- the server 2A202 sends a request 2B270 to the fulfillment party 2A250 for the photo-based project product 2A252.
- the fulfillment party 2A250 receives the request 2B270 for the photo-based project product 2A252, prints the photo-based project product, binds the photo-based project product, packages the product, and then ships the ordered product 2A252 to the end customer, such as user 2A210.
- Fig. 2D shows an illustrative image analysis or photo processing process according to one embodiment of the invention.
- the image analysis process shown in Fig. 2D is illustrative only, and is not intended to be limiting. Various other image analysis processes, some of which are described later, are also within the scope of the present invention.
- This image analysis process describes an image similarity detection process, as illustrative of the image analysis processes that are useable with the present invention.
- the process 2D200 examines the images for similarity by expressing a query to the image database 2D202 (all of the images) by requesting images that are similar to one of the images (query image 2D208).
- the process considers basic shape and color information of the photo when looking through the database for potential matches of similar photos. Several factors make this matching process difficult.
- the query image is typically different from the target image, so the retrieval method must allow for some distortions. Since these are natural photos taken in natural environments, the photos may suffer artifacts such as color shift, poor resolution, dithering effects, and mis-registration. Furthermore, it is important to perform the retrieval fast enough to not hamper user experience of the media-product creation process.
- a wavelet transform 2D204 is performed on every image in the image database 2D202. By collecting just the few largest coefficients from this transform, the process distills a small "signature" for each of the images. These signatures are saved in a signature database 2D206 so that it is computationally efficient to compare them all to each other.
- the process 2D200 When the process 2D200 requires photo(s) of similarity to a given query image 2D208, the process performs a wavelet transform 2D210 on the query image 2D208 to produce a signature 2D212 for the given query image 2D208.
- This query signature 2D212 is compared to the signatures of the database images 2D206, and the best matches are retrieved by the process 2D200 for use in automatically creating, organizing, and presenting the media-based project to the user.
- the wavelet transform is used to analyze functions at different levels of detail; it is somewhat similar to the Fourier transform, but encodes both frequency and spatial information. By saving the few largest wavelet coefficients for an image (and throwing away all of the smaller coefficients), it is possible to recover a fairly accurate representation of the image. This property may be exploited for efficiency gains to optimize image similarity comparisons. For example, a wavelet transformed image (“signature") that incorporates 400 coefficients would require about 3% as much disk space as the original image. In one embodiment, it is possible to take a wavelet transform and keep just a few (for example, 20) coefficients for each color channel and distill from them a small "signature" for each image. Because the signature is so small, it permits very fast searching in the database.
- Fig. 3 is a flowchart 300 of an embodiment showing an automatic insertion of content into a design.
- Content may comprise photos or images, text, and user inputs from a user.
- a user starts a process of automatic insertion of content by using a user-device.
- the user logs in at the user device at step 304. After log-in, the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device.
- GUI graphical user interface
- the user at step 306 selects images or photos from the user device and uploads them to a server.
- the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images.
- automatic image information is derived by using computer algorithms that include, but are not limited to, one or more of detecting areas of saliency, face detection, computer vision analysis, photo similarity detection, and analyzing photo information, text matrix, photo captions, and so on. Automatic image information is also derived from image meta-data.
- the server at step 312 outputs a default design preview (an automated design) of a photobook.
- the design preview at step 312 also comprises a layout that is the basis for the design.
- the server performs automatic content grouping of the uploaded images.
- the server utilizes, but is not limited to, one or more of the information in the list 320.
- the list 320 comprises timestamps, image similarities, image sequences, photo importance, aesthetics, and so on.
- the server automatically inserts content into the design of the photo project by using, but not limited to, one or more of the items (computer algorithms) in the list 324.
- the list 324 comprises image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, photo position in a photo slot, and so on.
- the user previews the design at step 318.
- the preview at step 318 is populated after completing the automatic image information extraction of one or more of the items in the list 310, after step 314 of automatic content grouping, and after step 316 of automatic content insertion.
- the preview at step 318 is thus automatically populated by the server by using one or more of the items (computer algorithms) as shown by the lists 310, 320, and 324.
- the user can order the photobook at step 322.
- the order is sent to a printer.
- the printer may be a third party printer, a social printer, or any industrial printer that is adapted to print photos on a photo quality paper.
- the ordered photobook may be printed on a preferred size of a photo paper.
- the ordered photobook may be a shippable product to the user.
- the printer may be further adapted to print a shipping label comprising the order information, and other relevant information.
- the process ends at step 328.
- Fig. 4 is a flowchart 400 of an alternative embodiment showing a first insertion based on automatic image information and a second insertion based on observed image information of content into a design of a photo project.
- the process starts at a user-device.
- the user logs in at the user device at step 404. After the log-in, the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device.
- GUI graphical user interface
- the user at step 406 selects images or photos from the user device and uploads them to a server.
- the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images.
- automatic image information is derived by using computer algorithms that are relevant to areas of saliency, face detection, computer vision analysis, photo similarities, photo information, text matrix, and photo captions. Automatic image information is also derived from image meta-data.
- the server at step 412 outputs a default design preview of a photobook.
- the default design generated at step 412 is based in part on step 416 of automatic content grouping and step 414 of automatic content insertion.
- the server performs automatic content grouping of the uploaded images.
- the server utilizes the information comprising timestamps, image similarities, image sequences, photo importance, and aesthetics.
- the server automatically inserts content into the design of the photo project by using items (computer algorithms) comprising image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, and photo position in a photo slot.
- the default design at step 412 is based on a combination of computer algorithms in lists 410,
- the user previews the default design created by the server at the user device at step 418. Furthermore, the user customizes the design at step 420 by adding user interactions or user inputs. User interactions or user inputs comprise area of interest selection, photo importance determination, and so on. User interactions or user inputs are performed manually by the user. By observing and monitoring the user interactions at the GUI of the user device, the system is able to collect observed image information, which the system can use to improve on the automatic image information obtained by the system from image analysis. The system can monitor for such user interactions as zooming, panning, swapping photos, adding captions to photos, and so on.
- a final recommended design based on the automatic image information and the observed image information is generated by the system, and can be previewed by the user at step 422.
- the user can then perform additional customizations to the final recommended design, and then order the photobook at step 424.
- the order is sent to a printer as described previously. The process ends at step 428.
- Fig. 5 is a flowchart 500 of an alternative embodiment showing an insertion of content based on automatic and observed image information.
- the process starts at a user-device.
- the user logs in at the user device at step 504.
- the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device.
- GUI graphical user interface
- the user at step 506 selects images or photos from the user device and uploads them to a server.
- the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images. As shown in the list 512, automatic image information is derived by using computer algorithms that are relevant to areas of saliency, face detection, computer vision analysis, photo similarities, photo information, text matrix, photo captions, and so on. Automatic image information may also be derived from image meta-data. While the user is uploading original images to the server, the user can interact with a layout and a design of a photobook, and thus contribute to creating user inputs or user interactions with the design. At step 508, the user interacts with the images that are already uploaded to the server. User interactions are also called manual interactions, manually observed information, user inputs, or user's behavioral observation.
- user interactions comprise one or more of, but are not limited to, areas of interest in a photo, photo importance, panning, zooming, switching photos, and so on. Not every possible user interaction is shown in list 518 for simplicity of explanation.
- User inputs may also be called tweaks, and give rise to observed image information.
- a default preview of the design is generated at step 514.
- the automated design is subject to manual edit by the user at any time before an order to print is placed and confirmed.
- automatic content grouping of images is performed by using both observed image information from the user and automatic image information from the server. Errors made by the server in automatic content grouping are manually corrected by the user at step 520.
- the server utilizes one or more of, but is not limited to, the items or information in the list 516.
- the list 516 comprises timestamps, image similarities, image sequences, photo importance, aesthetics, and so on.
- User inputs at step 520 come from, but are not limited to, one or more items on the list 518.
- manual tweaks are performed to further customize content.
- the server automatically inserts content into the design of the photo project by using, but is not limited to, one or more of the items (computer algorithms) in the list 522.
- the list 522 comprises image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, photo position in a photo slot, and so on.
- Manual tweaks come from the list 518, but are not limited to the items shown.
- the user previews a final recommended design of the photo project at step 526.
- the user can order a photobook at step 528. The order may then be sent to a printer as previously described.
- the process completes at step 530.
- Fig. 6A illustrates a diagram showing a relationship between a canvas, a design, a layout, a designset, and a lay outset, according to an embodiment of the present invention.
- a lay outset 650 gives rise to a layout 654, which itself gives rise to a design 656.
- the design 656 inherits from both a layout 654, and indirectly its layoutset 650.
- the design 656 is selected from a designset 652.
- Each design 656 specifies a style and formatting of content in a layout 654 from which it depends, while each layout 654 specifies a location of photo slots and/or text slots without reference to styling.
- Each designset 652 is always tied to a layout 654, which itself is always tied to a layoutset 650.
- the design 656 gives rise to a canvas 658 through an addition of content by a user, automatically by the server, or both.
- This inter-relationship of a canvas, a design, and a layout ensures system flexibility and efficiency whenever designs have to be modified.
- Fig. 6B illustrates a diagram 600 showing an inter-relationship of a theme, a layout, and a design, as well as their corresponding sets.
- a user selects images to build a photobook. Once the images are selected, the user can select a theme 616.
- the theme 616 corresponds to designsets 614 and 618.
- Each designset is a collection of designs.
- the designset 614 comprises designs 608, 610, and 612.
- the designset 618 comprises designs 620, 622, and other designs not shown.
- Each design depends on a layout.
- the design 608 depends on a layout 604.
- Each layout depends on a layoutset.
- the layout 604 depends on a layoutset 602.
- a layout can have or create many designs.
- layout 606 can create designs 620, 622, and another designs (not labeled).
- a design at the user device may be presented in a canvas 626.
- a canvas 626 represents how a design will be printed on paper.
- a project 624 can be represented by a canvas 626 or a canvas 628.
- the design 622 is reflected on canvas 628.
- Each layoutset serves as a basis for many designsets, and each layout serves as a basis for many designs.
- a theme 616 selected is consistent with a layout as well as a design, thereby maintaining an inter-relationship between a theme, a layout, and a design.
- this interrelationship offers greater control for a responsive design update and for auto-flow/auto-fill algorithms, thereby saving time and labor in creating a photobook.
- Fig. 7 is an exemplary screenshot of an editing interface 700 displaying a photo-spread including a front cover, back cover, and spine according to an embodiment of the present invention.
- Interface 700 has a storyboard display 701 that includes all of the pages of the photobook being created.
- the inventors provide a unique paradigm for relating items across multiple pages including special pages, in this case, the front cover 703, the back cover 704, and the spine of the book 705.
- the photos, shapes and stickers can be "overflowed" from one page to other pages within the spread.
- the items are overflowing, they are linked together such that when an item changes on one page, its linked item gets updated reflectively on the other page.
- Fig. 8 is a block diagram illustrating an asset recommendation system according to an embodiment of the present invention.
- asset recommendations are made to a user who is editing a current project using the MIXBOOK collage editor.
- System architecture 800 includes a user 801 that is currently online with runtime server 804 running an asset recommendation engine (ARE) 805.
- ARE asset recommendation engine
- the system of the invention has access to all of the assets that have been designed by system designers, users whom may be registered as asset contributors, and any other user contributions to the system.
- Creative assets 803 include all of the stickers, shapes, backgrounds, layouts, photos, and any other accessible imagery.
- Fig. 9 is an architectural diagram illustrating a printing network 900 according to an embodiment of the present invention.
- Printing network 900 includes a user 901 connected online to a runtime server 902.
- Runtime server 902 is analogous to runtime servers previously described in this specification.
- Runtime server 902 includes a digital medium coupled thereto and adapted to contain all of the SW and data required to enable server function.
- MBE 906 MIXBOOK editor
- the user is connected in session with server 902.
- the user may decide to print a finished project such as a photobook by invoking a printing option such as print project option 908 illustrated in exploded view.
- a printing engine 907 that is responsible for rendering the user's project in a ready state for printing according to a final end product selected by the user.
- the printing engine sends output files for printing to a third-party printing service 903.
- Printing service 903 represents any online printing service.
- Third-party printer 903 includes a printing server 904 and a connected printer 905 representing printing capability.
- Printing engine 907 is a flash based that is enhanced to produce high resolution output files to print.
- Print engine 907 is a single print engine that is involved from the point of creating a project to the point of printing out the project.
- PRINTED PHOTOBOOK AND PHOTOALBUM EMBODIMENTS Example printed embodiments of the present invention are photobooks with one or more photo strips, having one or more photographic images inserted (mounted or set) into one or more cover surfaces of the photobook, wherein the one or more photographic images of each photo strip is essentially flush with the cover surface.
- a photo strip of an embodiment of the present invention may be inserted into a channel on a cover surface and may be attached using an adhesive or other bonding method.
- Photo strips for the present invention may be created in any manner and can have images of any kind arranged in any pattern, shape, or form.
- the one or more photographic images of the photo strip on the cover occupy only part of the cover surface of the photobook and are essentially flush with the photobook cover.
- the observer obtains a crisp visual experience of one or more photographic images with a photobook cover, providing a border flush with the one or more photographic images of each photo strip.
- This embodiment of the invention solves a visual depreciation problem common in the prior art that is felt when the photobook cover is very thick and relatively cumbersome, if the front cover is a thick picture frame mounting.
- Another problem with prior art photobooks occurs when the cover images are significantly recessed below the photobook cover's surface since this causes the cover image to appear walled-in.
- Another prior art problem is the cover image of a photobook can become dim or too light-reflecting if the image is placed behind a protective window of plastic or glass, or is laminated, or embedded in plastic.
- FIG. 10 An example photobook embodiment of the present invention is depicted in Fig. 10, which has a front cover 114; and a back cover edge 106.
- a stiff flat spine 102 connects the front cover to the back cover.
- Fig. 10 depicts a frontal view of a photobook cover channel 112. This channel runs from the top edge 108 to the bottom edge 118 of the front cover 114.
- Channel 112 is the location where a photo strip is attached (mounted or set) using an adhesive or other means, so that the photo strip and its images are essentially flush with the front cover 114.
- Channel edge 116 indicates the depth of channel 112. The channel depth from the cover surface is made to be the thickness of the photo strip plus the thickness of an adhesive layer or other attachment means that has a thickness.
- Fig. 10A depicts in greater detail various features of an example photobook of the present invention. Depicted are top edge 108 of channel 112, vertical edge 116 of channel 112, front cover 114, top edge 104 of front cover 114, top edge 106 of back cover (not shown), and top view of inside pages 110. Fig. 10A also provides details of three front cover channel surfaces, recessed top edge 108 of channel 112; channel side wall (edge) 116; and front cover channel 112, into which the photo strip would be mounted. When a photo strip is set into the channel 112 and wraps over top edge 108, the photographic images of the photo strip are essentially flush with top edge surface 104 as well as with cover surface 114. Fig.
- FIG. 10B depicts a view of the example photobook front cover 114, back cover 119, top edge 116 of back cover 119, flat stiff spine 102, channel 112, top edge 104 of front cover 114, top edge 108 of channel 112, bottom edge 118 of channel 112, side 116 of channel 112, and inside pages 110.
- Top edge 104 of the front cover has a top edge channel 108 in the mid-section of top edge 104.
- Front cover 104 continues with a channel 112 from the top edge channel 108, and channel 112 traverses a central area or mid-section of the front cover, with channel 112 continuing to bottom edge channel 118 of the front cover.
- a photo strip (not shown) could be set down into channel 112 and photo strip's top end could be wrapped over recessed top edge 108, while photo strip's bottom end could be wrapped over recessed bottom edge 118, so that the photo strip is essentially flush with the front cover 114, and top edge 104 surface, as well as bottom edge surface of the front cover 114.
- Fig. 11 depicts a frontal view of an example embodiment of a photobook of the present invention, similar to the photobook depicted in Fig. 10, with a hypothetical single image on a photo strip 122 set in a channel 112 (not visible), so that the image and photo strip with the image are essentially flush with front cover 114, as suggested at locations 124, 126, and 126, where an edge of the photo strip in adjacent to a part of front cover 114 and at location 120 for top edge 104 of the front cover.
- Fig. 12 depicts a frontal view of an example embodiment of a photobook of the present invention similar to the photobook depicted in Fig. 11, however the photo strip depicts hypothetical borders for eight photographic images as a collage 128.
- the photo strip is essentially flush with front cover 114, as suggested at locations 121, 125, and 126 of front cover 114.
- Fig. 13 depicts a frontal view of another example embodiment of a photobook of the present invention, with a photo strip 127 depicting hypothetical borders for twelve photographic images as a collage set in the single channel on the front cover 114 (not visible), so that photo strip 127 is essentially flush with cover 114, as indicated at locations 120, 124, 126, and 129 of front cover 114.
- Fig. 13 photo strip 127 is wider than collage 128 of Fig. 12.
- Fig. 14 depicts a frontal view of an example embodiment of a photobook of the present invention with a photo strip 131 , depicting hypothetical borders for four photographic images as a collage set in the single channel on the front cover 114 (not visible), so that photo strip 127 is essentially flush with the front cover, as indicated at locations 126 and 133 of front cover 114.
- Fig. 15 depicts a frontal view of another example embodiment of a photobook of the present invention, with a horizontal photo strip 135 in the single channel on the front cover 114 (not visible), so that photo strip 135 is essentially flush with cover 114 as indicated at locations 120, 136, and 137 of front cover 114.
- Photo strip 135 depicts hypothetical borders for a collage (montage) of eight photographic images. Photo strip 135 wraps over the right side edge 120 of the front cover 114.
- Fig. 16 depicts a frontal view of another example embodiment of a photobook of the present invention, with circular photo strip 141 set into a circular front cover channel (not visible), so that the depicted hypothetical photographic image is essentially flush with the front cover 114, as suggested at locations 139 and 143. It is contemplated that there could be multiple images in circular photo strip 141 as a montage, and there could be multiple circular photo strips set in multiple circular channels on cover 114 of the present invention.
- the present invention further includes one or any combination of the following embodiments (1) embodiments wherein the photo strip is straight and has a constant width; (2) embodiments wherein the photo strip is located centrally on the first cover; (3) embodiments wherein at least two edges of the photo strip are located at edges of the first cover; (4) embodiments wherein edges of the photo strip cross an edge of the first cover, and a part of the photo strip is located on an outside face of the first cover and a part of the photo strip is located on an inside face of the first cover; (5) embodiments wherein edges of the photo strip are within edges of the first cover; (6) embodiments wherein the photo strip displays the one or more photographic images on the photo strip as a collage; (7) embodiments further comprising one or more printed photographic images essentially on a cover material of the photobook; (8) embodiments wherein the one or more printed photographic images is essentially on a material comprising a paper; (9) embodiments wherein one or more of the covers of the photobook comprises a rigid board cover material; (10) embodiments wherein one or
- the photobook has a feature selected from the group consisting of a photo strip which displays a collage of images, a back cover which contains a padding material, a spine which comprises a flexible cover material, a spine which is rounded, a binding decoration which is located between the spine and the binding of the pages, an end sheet before the binding of the pages, an end sheet after the binding of the pages, an end sheet which displays a collage of images, pages which display a collage of images, a page which displays a single image, two consecutive pages which display a portion of the same image, and any combination of features thereof.
- the front cover, the spine, and the back cover are wrapped with a flexible cover material.
- a part of the front cover, a part of the spine, and a part of the back cover have a flexible cover material surface.
- a part of the front cover, a part of the spine, and a part of the back cover have a flexible cover material surface.
- Figs. 17A, 17B, and 17C depict in cross-section three sequential steps of one method embodiment of the invention for making a channel 112 in a photobook cover 114 (or more simply cover material 114). Any method for making a channel 112 in a cover or a cover material may be used, including cutting, hammering, abrading, dissolving, or melting (depending upon characteristics of cover material 114).
- Fig. 17A depicts press 132 with a stamp 134 above a cover material 114. Stamping area of stamp 134 is selected to have a shape similar to the shape of the photo strip intended so that the channel 112 has the shape of the photo strip.
- One or more channel shaped stamps used in conjunction to make a complex shaped area channel may be complex, when the channel needs to have a complex shape.
- the channel may have a shape selected from the group consisting of a square, a rectangle, a triangle, a pentagon, a hexagon, a polygon, a circle, an ellipse, a rounded vertices polygon, a shape having a hole, and a combination thereof.
- a channel needs to be made to have a depth matching the total thickness of the photo strip and its adhesive attaching it in the channel.
- press stamp 136 is compressing an area of the cover material.
- press stamp 136 has been withdrawn from cover material 114, and cover material 114 has a resulting channel 112 therein.
- the channel can be formed in a particular flexible cover material, with a depth matching the thickness of the photo strip and its underlying adhesive layer, so that the photo strip can be set into the channel and is essentially flush with the cover material surface.
- the time required and amount of pressure needed to form the channel varies with the properties of the cover material, as is known in the art; sometimes termed de-bossing.
- the pressure needed by the stamp press to form the channel in a cover material may vary considerably. It is expected that for some soft materials, the stamp pressure would need to be only 0.1 psi.
- the stamp pressure might need to be as high as 20,000 psi (pounds per square inch).
- Another factor is the conditioning of the cover material, such as its temperature, stretch and hydration. A temperature may be increased to soften some synthetic cover materials before their pressing to form the channel. Natural fibers and leathers may be softened by a controlled aqueous hydration of the material prior to pressing the material. In addition, a softer material surface may be placed beneath the cover material when stamping the cover material to make the channel.
- the channel in the cover material such as a flexible cover material
- One method is to then attach the photo strip to the flexible cover material. It is contemplated, however, for some embodiments of the present invention that the strip is first attached by some means such as an adhesive, and then the photo strip attached to the cover material is pressed so that the photo strip becomes essentially flush with adjacent cover material. This method assumes that the photographic images on the photo strip can tolerate such stamping.
- the method for making the channel in the cover of the photobook further comprises applying two strips of a hard material to the sides of the cover to make cover walls for a channel.
- the channel-making method may combine this method of building up the walls channel with stamp press compression of a flexible cover material being wrapped over the two strips of the hard material.
- the photobook has a cover comprising a rigid cover board material wrapped with a flexible cover material.
- the flexible cover material in such embodiments has been pressed (or de-bossed) using a stamping process as exemplified in Figs. 17A, 17B, and 17C to create the needed channel.
- the flexible cover material is wrapped around the rigid cover board material to make the cover and then the photo strip is mounted (set) into the channel of the flexible cover material.
- the photo strip on a cover of a photobook of some embodiments of the present invention does not cover the entire cover.
- the cover-mounted photo strip and the cover's channel may be the length the photobook cover.
- the photo strip and the sub-surface channel may have only a central location on the cover with a cover margin on all sides of the cover.
- a preferred horizontal width for the photo strip mounted on a cover of the photobook is between about 20 percent to about 80 percent of the horizontal length of the cover, more preferably between about 40 percent to about 70 percent of the horizontal length of the cover, and most preferably between about 50 percent to about 65 percent of the horizontal length of the cover.
- Preferred embodiments of the photobook have a photo strip with a plurality of images in the form of a collage or montage.
- the photo strip and the channel run the length of the cover and both the strip and the channel continue around edges of the photobook with the photo strip terminating on the inside of the cover.
- the photo strip material could be mounted in a long channel so that the images of the photo strip mounted in the channel continue from one cover, across the spine, and onto the other cover.
- a preferred flexible cover material, for some embodiments of the photo book is thick synthetic leather. Any cover material, including glass, metal, plastic, or wood could be used provided that a suitable method is used to make the channel for the photo strip.
- Figs. 18A and 18B illustrate an option for some embodiments of the present invention in which an extra strip 150 of the cover material may be attached to the inside 140 of the spine 148 of the photobook 142.
- This extra strip 150 can be used to cover up unevenness along the spine 148 caused by the flexible cover material being wrapped from an outside cover surface to an inside cover surface 144.
- This extra strip 150 may also help to make the inside of the spine flatter.
- Additional cover materials, such as padding 146 may be added to the front or back covers as needed to enhance the feel of the photobook.
- a rigid cover board used for the present invention may comprise a material such as a high impact fiberboard, laminated plywood, a chip board, a metal sheet, a plastic sheet, and any combination thereof.
- a chip board is a preferred rigid board cover material.
- a rigid cover board has a thickness between 0.01 to about 1 inch, and preferably between 0.1 to about 0.5 inches.
- the front cover has the same thickness as the back cover, and uses a 0.125 inch thick chip board with a 0.0625 inch thick strip on top of the sides of the chip board.
- a cover material which may be used for the present invention may be selected from the group consisting of a wood, a wood chip board, a plastic, a metal foil, a foam material, a soft fabric, a metallic fabric, a fiberglass, a polymer, a fiber, a laminate, a glass, a ceramic, a leather, an animal skin, a mineral, a stone, a transparent material, a reflective material, an opaque material, a translucent material, a soft material, a padded material, a woven material, a waterproof material, a synthetic material, a flexible material, a rigid material, a rough material, a smooth material, a fragrant material, a velour material, a silk material, and a combination thereof.
- an adhesive, a bonding agent, or another attaching means is used for making some embodiments of the present invention, with the proviso that adhesive, bonding agent, or other attaching means has a compatibility with attaching the photo strip to the cover material.
- An adhesive, a bonding agent, or another attaching means with photobook materials should not be used which may stain, degrade, or discolor the photobook cover material or the photo strip.
- one method for attaching the photo strip to a channel on a cover of the photobook comprises printing the photo strip on an adhesive backed paper or applying an adhesive to the back of the photo strip; and setting the photo strip in the channel so that the adhesive holds the photo strip to surfaces of the channel.
- One method for securing the photo strip and creating a clean finish is to terminate a photo strip on the inside face of the front cover after the photo strip has been wrapped over edges of a cover.
- the photo strip is terminated on the inside face of a cover after the photo strip has been wrapped over the cover edges by approximately 0.5 inches from a cover edge.
- the photo strip can of course be permitted to wrap further on the back side of the front cover.
- the photo strip may display photographic images only on an outer side of the cover, with distal ends of the photo strip displaying no photographic images but colored to match or contrast with the coloration of the photobook cover.
- the present invention preferably uses end sheets, which are a material layer that is attached to inside surfaces of the front and back cover of the photobook.
- An end sheet can be decorated in any manner for the present invention.
- a photo collage or a montage of some of the photographic images from pages of the photobook may be printed on the end sheets of a photobook of the present invention.
- End sheets may be comprised of a cover material, including a paper material.
- the end sheets and photographic images of the photo strip can be a collage of images, symbols, designs, and the like, and can have transparency, filters, overlays, and any other effects.
- Cover surface and end sheet surface designs can, for example, include various grids, equal sized boxes, rectangles, polygons and rounded shapes, fractal patterns, and the like, in a repeat or random pattern.
- One method embodiment of the present invention is a method for making a photobook, the photobook comprising a photo strip set in a channel on a cover of the photobook, the channel having an area less than the area of the cover, the photo strip displaying one or multiple photographic images, so that the one or more photographic images are essentially flush with the cover, the method for making said photobook comprising (a) cutting pieces of a rigid cover board material for a front cover, and a back cover of the photo book; (b) cutting a piece of a flexible cover material to a size and to a shape for a wrapping of the piece of the flexible cover material over the pieces of the rigid board cover materials for making the front cover, and for making the back cover; (c) making a connection between the front cover and the back cover; (d) making the channel by pressing a channel shaped stamp into the piece of the flexible cover material, the channel shaped stamp having a shape and dimensions of the photo strip to be set into the channel; (e) holding the channel pressed into the piece of the flexible cover material over the rigid
- the photobook has a rounded spine 16CT which comprises a flexible cover material. See rounded spine 160 in Figs. 19-20.
- the spine 160 may be made of the flexible cover material used on the front and back covers of the photobook.
- One aesthetic advantage of the soft spine photobook embodiments of the present invention is that the photo strip on the front cover can have equal-width panels and be centered on the photobook at the same time.
- the flat stiff spine 102 photobook (as depicted in Fig. 11) can have equal-width front cover panels, but at the same time cannot be centered with respect to the overall photobook.
- FIG. 19 depicts a frontal view example of a photobook embodiment of the present invention prior to insertion of a photo strip into channel 170, which in this particular example vertically traverses the front cover, and over upper edge 166 and lower edge 174 of front cover 172. Additionally, Fig. 19 depicts front cover 172 connected to back cover 162 via spine 160. There is a flat area 164 on each cover side that smoothly transitions to become the rounded shoulders of the soft spine, unlike the embodiments with a flat stiff spine which are depicted in Figs. 10, 11, 12, 13, 14, 15, and 16. Fig. 19 also depicts that inside pages 168 near the spine are functionally interconnected as a binding. The first and last inside pages of some photobooks are connected to the front and back covers of the photobook.
- Channel 170 has a depth which can be inferred from the depiction of the channel depth at channel edge 176.
- Channel depth is made to be the same as the sum of the photo strip thickness plus the thickness of the adhesive that is used to attach the photo strip to the channel. In this way, the photo strip when mounted in the channel using the adhesive, is essentially flush with the cover of the photobook.
- Fig. 20 depicts a soft rounded spine photobook embodiment of the present invention, with a photo strip 184 set into channel 170 (not visible here) between front cover panels 172 and 192.
- Related Fig. 19 depicts channel 170.
- the example photobook embodiment of the present invention has a photo strip which traverses the front cover vertically after wrapping around top edge 180 and the bottom edge.
- the depicted photo strip 184 is depicted as a hypothetical collage of six photographic images (specific photographic images not depicted), where only hypothetical borders have been drawn within the photo strip 184.
- the hypothetical six photographic images of the photo strip 184 are depicted set in a channel 170 and are essentially flush with the surfaces of cover panels 172 and 192.
- Fig. 20 depicts a soft rounded spine photobook embodiment of the present invention, with a photo strip 184 set into channel 170 (not visible here) between front cover panels 172 and 192.
- Related Fig. 19 depicts channel 170.
- the photo strip is essentially flush with the cover, for example, at upper right corner 188 of the photo strip, at mid right side 190 of the photo strip, at mid-left side 182 of the photo strip, and at bottom left edge 186 of the photo strip.
- the photobook can be of any size, and dimensions, such as in terms of its height, width, and the thickness overall, and of its covers.
- a reference to “a method” includes one or more methods, and/or steps of the type described herein and/or which will become apparent to those persons skilled in the art upon reading this disclosure.
- the use of the word "or" between a first item and a second item shall mean the group of items equaling a first item, a second item, and both the first item and the second item.
- all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, the preferred methods, constructs and materials are now described. All publications mentioned herein are incorporated herein by reference in their entirety.
- a user-device is a hardware that includes at least one processor coupled to a memory.
- the processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory e.g., cache memories, non-volatile or back-up memories (e.g. programmable or flash memories), read-only memories, etc.
- the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device.
- the hardware of a user-device also typically receives a number of inputs and outputs for communicating information externally.
- the hardware may include one or more user input devices (e.g., a keyboard, a mouse, a scanner, a microphone, a web camera, etc.) and a display (e.g., a Liquid Crystal Display (LCD) panel).
- the hardware my also include one or more mass storage devices, e.g., a floppy or other removable disk drive, a hard disk drive, a Direct Access Storage Device (DASD), an optical drive (e.g.
- DASD Direct Access Storage Device
- the hardware may include an interface with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others) to permit the communication of information with other computers coupled to the networks.
- networks e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others.
- LAN local area network
- WAN wide area network
- wireless network e.g., a wireless network
- the hardware typically includes suitable analog and/or digital interfaces to communicate with each other.
- the method executed to implement the embodiments of the invention may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as "computer program(s)" or “computer code(s).”
- the computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations necessary to execute elements involving the various aspects of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
A photo-based project and associated printed photobook having a flush photo collage, or montage, on the front cover are disclosed. In one embodiment, a photobook is made by receiving a plurality of images corresponding to a photo-project; storing said plurality of images in a data repository; receiving a print order for a printed photobook corresponding to the photo-project; in response to the print order, printing a photo-strip having one or more of the images; producing a front cover by compressing a compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; wrapping the compressible cover material with the channel around a cover board material to make the front cover; and setting the photo-strip into the channel and flush with the front cover to create the photobook. An aesthetically pleasing photobook is thereby manufactured.
Description
PHOTOBOOK AND PHOTOALBUM HAVING A FLUSH PHOTO MONTAGE ON THE FRONT COVER AND METHODS FOR MANUFACTURE THEREOF
REFERENCE TO RELATED APPLICATIONS
This application is an International Application under the Patent Cooperation Treaty (PCT) which claims priority from U.S. Serial No. 14/529,777, filed on 31 October 2014, which issued as U.S. Patent No. 9,077,823, issued on 7 July 2015, and entitled "SYSTEMS AND METHODS FOR A UTOMATICALLY GENERATING A PHOTO-BASED PROJECT HA VING A FLUSH PHOTO MONTAGE ON THE FRONT COVER " as well as claims priority from provisional application U.S. Serial No. 62/079,424, filed on 13 November 2014, and entitled "MIXBOOK PATENTS 3.1," the entire disclosures of which are hereby incorporated by reference in their entireties herein.
FIELD OF THE INVENTION
Embodiments of the present invention broadly relate to systems and methods for customized media-based projects. In particular, systems and methods are provided for generating a photo-based project that can be printed as a photobook, which has a photo strip in a channel on the front cover, so that an attached photo strip, having a photo montage or photo collage, is essentially flush with the front cover. Other embodiments of the present invention relate to customizations of content including multimedia files, photos, text, and stylistic parameters when ordering a media-based project online via an electronic commerce software application.
BACKGROUND OF THE INVENTION
The statements in this section may be useful background to understanding the invention, but may not constitute prior art.
In the field of electronic commerce (e-commerce), there are interactive websites that assist users in creating media-based projects such as photo-books, photo-calendars, photo-cards, and photo- invitations. Such interactive websites allow users to upload photos and interact with the websites to create photo-based projects that are customized to user's preferences. A media-based project creation utilizes photos, videos, audio-videos, pictures, and/or text. As used herein, the phrase "photobook" shall be synonymous with any type of media-based product or project. Prior art photo-based project web sites come with various drawbacks.
A common drawback with conventional photo-based project websites is the amount of time required by users (Internet users or customers) to create a photo-based project. Users must often participate in a variety of functions and processes in order to complete a photo-based project. For example, users usually must sort through the photos they desire to upload, remove the photos that are not adequate or properly focused, choose the photos to be uploaded, group photos together by topic or location, order the photos by chronological order, and crop or focus photos on the relevant portions of
said photos. Performing the above processes can take undue time and labor. Further, there are corrections and modifications that users must make to photos before the photos can be used in a photo-based project. For example, users often must correctly orient photos (landscape or portrait view), perform color corrections on photos, remove red-eye from the eyes of photo subjects, and correct photos for brightness or contrast. All of the corrections and modifications mentioned above require that the users additionally open a second program or application in a user device such as a personal computer, choose the photos to be corrected or modified, perform corrections or modifications, and then save the modified photos before using them in the project. Again, performing the corrections and modifications mentioned above may require additional skills, and can be time consuming, labor intensive, and create displeasure for users. As a result, many users that log-on to prior art photo-based project websites often discontinue the process, without completing a purchase transaction of the project.
Prior art products for a photobook creation are often based on a theme or stylistic parameter. However, they often do not have any bearing on the inter-relationship of layouts, themes and designs so applied, and on how relevant photos are grouped together in the project. Prior art products also lack photo feature extraction, object detection, and image analysis to intelligently customize the photobook for the user.
Prior art products also lack a layering of photos based on photo feature extraction, object detection, and image analysis. With such shortcomings in the prior art, an Internet user or customer may need to spend undue time and labor in manually customizing layouts, colors, themes, stylistic parameters, and captions of photos and text to be able to finally design and print-by-order a satisfactory photobook. Undue time and labor in such a manual customization is a barrier to decide on an online purchase of a photobook. Such a barrier is also known as a "friction point." Such barriers or friction points are further compounded by unfriendly graphical user interface (GUI) of prior art products and photo editing software, and result in additional inefficiencies and errors in printing a media-based product, such as a photobook. Such friction points result in loss of potential customers before the customers reach to the point where they can order the media-based product for printing. A significant fraction of potential customers is lost in such friction points.
Because of the aforementioned issues in customizing and printing a photobook through a web application or electronic commerce application, customer or user experience in creating and ordering a photobook is inefficient.
With respect to the physical books themselves, prior art books may have a photo printed on a protective book jacket of the book or on a hardcover of the book itself. For example, school textbooks need to withstand heavy use by students and may have a textbook cover photo concerning the textbook subject that is printed as an integral part of the protective plastic wrapping of the textbook cover. Similarly, a protective clear plastic wrap may be used over a photo on a photobook cover. In these cases, there is a printing of a photographic image directly on the protective cover material or the
jacket of the book cover, and as a result the printed photographic image contributes nothing to the thickness of the protective cover material or to the thickness of the book jacket.
Other personalized photo albums and photobooks are generally used to hold photographs printed on photographic paper, and the cover of the album or book may display a photograph printed on photographic paper as well. A personalized photograph may be inserted underneath a plastic window on top of the photobook cover, or may be inserted underneath a border frame on top of the book cover. In some cases, the personalized photograph is inserted underneath a plastic window bordered by a frame on the top of the photobook cover. If a photograph is attached to a photobook cover without a protective covering, then an object may catch an edge of the photograph and initiate a sudden detachment or gradual peeling of the photograph from the cover of the photobook cover. The present inventors have recognized that it would be an advancement in the state of the art to address these and other problems with prior art methods for attaching photographs to photo albums, photobooks, and the like.
It is against this background that the various embodiments of the present invention were developed.
BRIEF SUMMARY OF THE INVENTION
Accordingly, the inventors have developed an embodiments of the present invention, which include a photobook, made by a process comprising the steps of printing a photo-strip having one or more images; producing a front cover by compressing a compressible cover material with a stamp to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; wrapping the compressible cover material with the channel around a cover board material to make the front cover; and setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
Other embodiments of the present invention include a photobook, made by a process comprising the steps of printing a photo-strip having one or more images; producing a front cover by wrapping a compressible cover material around a cover board material to make the front cover; compressing the compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; and setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
With respect to the physical photobook itself, a preferred embodiment of the present invention is a photobook, comprising a front cover with a channel having an area less than the area of the front cover; a photo strip set into the channel; a back cover; a connection between the front cover and the back cover; and pages for photographic images, wherein the pages are situated between the front cover and the back cover, and wherein the photo strip set into the channel displays a single or multiple photographic images which are essentially flush with the front cover.
The photo strip set into the channel can have a location anywhere on the front cover, for example a location selected from the group consisting of in a central location, a right side location, a left side location, an upper location, a lower location, and a combination thereof.
In some embodiments of the present invention, the photo strip is wrapped around a front cover edge and terminates on a backside of the front cover. In other embodiments of the present invention, two or more ends of the photo strip are wrapped around edges of the front cover and terminate on the backside. For other embodiments of the photobook, the photo strip is set into a channel on the front cover and ends of the photo strip are wrapped around one to three edges of the front cover, and terminate on the back side of the front cover. In some embodiments of the invention, the channel from the cover continues around the cover edge so that the photo strip is essentially flush with the surface of the edge of the cover as well as with the cover.
In other embodiments of the photobook of the present invention, there is a channel also on the back cover of the photobook with a photo strip is set into the back cover channel so that this photo strip is essentially flush with the back cover. Photobook embodiments of the invention preferably have a photo strip covering only part of a cover.
Furthermore, in other embodiments of the photobook of the present invention there are several channels on a cover of the photobook. Optionally a photo strip can be located on the spine or an interior side of a cover with this photo strip set into a channel on the spine or interior side of the cover so that the photo strip is essentially flush with the surface of the spine or the interior cover. Notably, a photo strip on a cover of the photobook can be of any imaginable shape, for example a square, a rectangle, a triangle, pentagon, a hexagon, a polygon, a circle, an ellipse, a rounded vertices polygon, a shape having a hole or a combination of these shapes. When the present invention is practiced to make a photobook, photo strip images appear as a collage or montage of images.
For some embodiments of the present invention, the front cover or the back cover or both of a photobook may contain a rigid board material to stiffen the cover. For other embodiments of the present invention, the front cover and/or the back cover of the photobook comprises a flexible cover material so that the cover is flexible. In preferred embodiments of the present invention, the front cover and the back cover of the photobook each contain at least a rigid board cover material wrapped over with a flexible cover material.
In some embodiments of the present invention, the spine of the photobook flexibly connects the front cover to the back cover, the spine comprising the same flexible cover material as the front and back covers. In alternative embodiments, the spine of the photobook contains a rigid board material beneath the flexible cover material which stiffens the spine. For some embodiments of the present invention, the front cover, the back cover or both covers are padded using a padding material between the rigid board cover material and the flexible cover material. When in some embodiments of the present invention, the padding has been added only to the back cover of the photobook, and the
front cover contains additional rigid cover board material so that the thickness of the front cover will match the thickness of the back cover.
Some embodiments of the present invention are a photobook, comprising a front cover with a centrally located channel; a photo strip; a back cover; a spine comprising a flexible cover material; and pages for photographic images between the first cover and the second cover, wherein the front cover further comprises a first rigid board cover material over which is wrapped a flexible cover material, and wherein the back cover comprises a second rigid board cover material and further comprises a padding material over which has been wrapped the flexible cover material, and wherein the photo strip is set into the centrally located channel, the photo strip displaying one or multiple photographic images which are essentially flush with the cover. Optionally some photobook embodiments of the preset invention have no spine which connects the front cover directly with the back cover, and the photobook has been made with its inside pages interconnected and connected to the covers. In preferred embodiments of the present invention, the photobook has a spine connecting the front cover and the back cover and also the inside pages are interconnected and fixed to the covers.
The present invention also includes methods for making a photobook of the present invention. In one embodiment, the present invention provides a method for making a photobook, the photobook comprising a photo strip set in a channel on a cover of the photobook, the channel having an area less than the area of the cover, the photo strip displaying one or multiple photographic images, so that the one or more photographic images is essentially flush with the cover, the method for making said photobook comprising (a) cutting pieces of a rigid cover board material for a front cover, and a back cover of the photobook; (b) cutting a piece of a flexible cover material to a size and to a shape for a wrapping of the piece of the flexible cover material over the pieces of the rigid board cover materials for the front cover and the back cover; (c) using a portion of the flexible cover material between the front cover and the back cover for making a spine connecting the front cover and the back cover; (d) making the channel by pressing a channel shaped stamp into the piece of the flexible cover material, the channel shaped stamp having so the channel has a shape and dimensions of the photo strip to be set into the channel; (e) holding the channel pressed into the piece of the flexible cover material over the rigid cover board materials at a cover location selected for the channel while wrapping the piece of flexible cover material over the front cover and the back cover; (f) using a portion of the flexible cover material for making a flexible spine comprising the flexible cover material connecting the front cover to the back cover of the photobook; (g) setting the photo strip having one or more photographic images into the channel of the piece of the flexible cover material so that the one or more photographic images of the photo strip are essentially flush with the cover having the channel; (h) printing photographic images on pages to be situated in the photobook between the front cover and the back cover; and (i) situating the pages with the photographic images in the photobook between the front cover and the back cover, thereby creating said photobook.
The flexible cover material is preferably pressed to form a channel for the photo strip before the flexible cover material is wrapped around the rigid cover board material. For synthetic flexible cover materials, the pressing may require an elevated temperature to soften the flexible cover material. For some embodiments of the present invention, the photobook has a cover comprising a rigid cover board wrapped with a flexible cover material, wherein the flexible cover material is pressed to create a channel before the flexible cover material is wrapped around a rigid cover board of a cover so that one or more photographic images on the photo strip is essentially flush with the cover. In such cases as when the photo strip is intended to continue from a cover surface to over a cover edge, and thence to the other side of the cover, then these edge surfaces needed require pressing to create a channel on the cover edge so that the photo strip is essentially flush with the cover edges.
For some embodiments of the present invention, the method for making the channel comprises pressing a piece of the flexible cover material with a stamp having the shape and the dimensions of the photo strip to be set into the channel. For some materials stamped to form a channel, the stamp pressure is about 1 psi (pounds per square inch) to about 20,000 psi.
For some embodiments of the present invention, one method for making the channel in a photobook cover further comprises adding rigid cover material strips to the rigid cover material beneath sides of the channel in the flexible cover material. In other embodiments of the present invention, methods for making the channel in a photobook cover comprise using one or more layers of same rigid cover materials alone or in combination with one or more different layers of flexible cover materials. Thus the cover materials of the channel in a cover of the photobook may comprise rigid cover materials, flexible cover materials, or any combinations thereof. Any method for shaping, casting, or molding of a cover material as a means for creating a channel wall is contemplated for some embodiments of the present invention. For example, contemplated are methods for making embodiments of the present invention having a channel in one or more cover materials of the photobook by a process of shaping, or a molding, or a casting a cover channel in a rigid cover board material and placing said channel of the rigid cover board material beneath a channel in a flexible cover material made by a process of pressing the flexible cover material. Any known method for forming a channel in a flexible cover material is contemplated for some embodiments of the present invention. One or a plurality of cover materials may be employed in some embodiments of the present invention to create one or more channels in one or more covers of the photobook so that one or more photo strips can be set into the one or the more channels so that the one or more photographic images of the one or the more photo strips are essentially flush with the cover having the channel.
For some embodiments of the present invention, the method for making the photobook further comprises padding a cover with a padding material to create a soft cover touch. Any form of padding may be suitable, including a flexible foam plastic sheet. Padding may be placed over the rigid cover board of the front cover or back cover. When padding is used in one cover, then other cover can be matched in thickness by a method of using a thicker rigid cover board material in the other cover.
For some embodiments of the present invention, the method for making the photobook further comprises adding a flexible cover material strip along at least the width of the inside surface of the spine. Adding the cover strip along the inside of the spine helps to reduce or remove or smooth out any uneven aspect of a surface along the spine caused by the use of a flexible cover material, including for example, a thick cover material such as a synthetic leather cover material that is wrapped over a photobook edge such as from an outside surface to an inside surface of the photobook.
In some photobook embodiments of the present invention, the photobook has a feature selected from the group consisting of a photo strip which displays a collage of images, a back cover which contains a padding material, a spine which comprises a flexible cover material, a spine which is rounded, a binding decoration which is located between the spine and the binding of the pages, an end sheet before the binding of the pages, an end sheet after the binding of the pages, an end sheet which displays a collage of images, pages which display a collage of images, a page which displays a single image, two consecutive pages which have a portion of the same image, and any combination of features thereof. In embodiments of the present invention, wherein the spine does not have a rigid board material, the spine may comprise a layer of a flexible cover material such as the kind used as a wrapping for the cover of the photobook.
In embodiments of the present invention, wherein the spine of a photobook has a binding decoration, the decoration may be located between the spine and the ends of the inside pages of the photobook. The binding decoration may be flexible or stiff, of any shape, and of any material. For example, the biding decoration may be a woven fabric of the same color as the flexible cover material and can give the photobook spine to have a traditional book binding appearance.
With respect to the electronic commerce software application for ordering the photobook, and according to another aspect of the invention, the inventors were trying to figure out the best way to automatically insert content (photos, text, and so on) into a design in such a way as for it to appear the way the user intended in a book that they would want to purchase, with minimum input from the user. In the prior art, content would be placed in a default way that would not take into account photo information and saliency information. The prior art would also not aggregate this information in a useful way. Most automatic content insertion in auto-fill algorithms, for example, involved centering photos perfectly in photo slots regardless of what was in the photo, thus potentially cutting off important content.
In the prior art, a combination of saliency detection, face detection, computer vision analysis, etc. was not used because all these technologies present many false positives that can make inserting the content into a layout worse. For example, with saliency detection, if one could potentially detect the sky of a photo as being the important part since it's bright and big, which would cause the photo to be positioned improperly, cutting off the people standing on the beach. Embodiments of the present invention solves such problems.
For grouping of photos, the prior art did not combine timestamp data along with photo similarity and photo meta-data information to do a better job at grouping the photos. The present invention is able to use this information to create a more appealing book with less user interaction. The present invention, from user feedback, presents a better finished product than the prior art. Feedback from users has been strong that the system is in fact automatically cropping and grouping photos intelligently.
The present invention helps automate the building of photobooks into a buyable product from the get-go, with little need for tweaking. The present invention offers several unique advantages over the state of the art: (a) positions photos more accurately within photo slots; (b) determines a photo's importance and uses it in the decision making process for where to place the photo on the page; (c) grouping algorithm takes into account a photo's importance, and will single a photo out if it is so important that it should end up on its own page; and (d) once the combined saliency of a photo is properly determined, the system will add the appropriate buffer to the photo so the system can fit it into a photo slot in a natural way. Other features and benefits are described later.
Accordingly, the present invention saves time and labor in creating media-based projects by automating content insertion into projects. In one embodiment, the present invention eliminates singularly practiced manual methods in prior art photo projects and automates a plurality of novel methods for creating such projects. Such novel methods include sorting of photos, removing inadequate or undesirable photos, selecting correct photos, grouping of photos, ordering of photos in correct order, and cropping or focusing on an area of the photos, and presenting a book design to a user. Other embodiments of the present invention automate a plurality of photo edit processes that were traditionally not available in a single prior art product; such edit processes include orienting photos, re-coloring photos or performing a color correction on photos, correcting red-eyes, and correcting photos for brightness or contrast. Furthermore, the present invention places photos more accurately within photo slots of a design page in a photobook. The present invention also auto-detects areas of saliency and computes photo importance for using photos intelligently in grouping and highlighting purposes. In one embodiment, the present invention presents to a user a final design of a photo project, without the user's involvement in the designing aspect of the project.
In one embodiment, the present invention is a method, a system, and a computer-readable storage medium storing executable program code for a process comprising the steps of automating a creation of a media project in a client-server environment via a graphical user interface (GUI) of a user device, after a user of said device uploads media files from said device to a server, the method comprising (1) inserting a first set of contents, said contents comprising media files, text and user inputs, into a design of said media project, wherein said first set of contents comprises information derived though an analysis performed by said sever, and wherein said first set of contents is automatically inserted into said design without said user's interaction to said design by using automated rules of inserting contents; (2) grouping of said media files into a group by using
automated rules of grouping, and inserting said group of media files into said design, wherein said design is a spread, and wherein said rules of grouping comprise timestamps, image similarities, approximations of image sequence, image scores by repetition of images, and/or aesthetics; and (3) presenting to said user at said GUI an automated design of said media project, wherein said presented automated design is prepared through said automated insertion rules of said contents of step (1) and said automated rules of grouping of step (2), and wherein said both rule types comprise computer algorithms of content insertion rules having saliency detection rules, face detection rules, computer vision detection rules, photo similarity detection rules, photo information detection rules, text matrix detection rules, and/or photo caption detection rules.
Another embodiment of the present invention includes a method, a system, and a computer- readable storage medium storing executable program code for a process comprising the steps of (1) establishing a client-server connection between the server and a computing device over said communications network; (2) receiving a plurality of images from said computing device through an upload to said server, and storing said plurality of images in a data repository connected to said server; (3) performing an image analysis on said plurality of images to derive automatic image information from said plurality of images; (4) grouping said plurality of images based on said automatic image information; (5) inserting content into said photo-project based on the grouping of said plurality of images to create an editable photo-project; (6) presenting said editable photo-project to a user of said computing device; (7) monitoring user interactions on said computing device while the user is manipulating said plurality of images to derive observed image information from said user interactions; and (8) automatically modifying an arrangement of content in said editable photo-project based on said observed image information to produce a final recommended design of the photo- project.
In some embodiments, the automatic image information may be derived though one or more algorithms for saliency detection, face detection, photo similarity detection, and/or object detection. The grouping of said plurality of images may be achieved through at least one of timestamp analysis, image similarity analysis, and photo importance scoring.
In some embodiments, the observed image information may be derived through analysis of user interactions comprising areas of interest selection, photo importance selection, text input, panning, zooming, moving, and/or swapping of photos by said user.
In some embodiments, the insertion of content may be achieved through one or more algorithms for a ranking of images based on a plurality of image heuristics for calculating image importance, a buffer calculation for a best fitting content into a photo slot, an area of interest selection on a photo, a score for inserting a photo into a photo slot of a layout, a determination of a best design of a design page, a dynamic text insertion algorithm, and/or a positioning of a photo into a photo slot having a focal box.
Yet another embodiment of the present invention includes a method, a system, and a computer-readable storage medium storing executable program code for a process comprising the steps of (1) inserting a first set of content into a design of said media project, wherein said first set of content is based on information derived though an analysis performed by the server, and wherein said first set of content is automatically inserted into said design without said user's interaction to said design by using automated rules for insertion of content; (2) grouping of said media files into a plurality of groups by using automated rules of content grouping, and inserting said groups of media files into said design, wherein said design is a spread, and wherein said rules of grouping comprise at least image similarity detection and image scores by repetition of images; (3) presenting to said user at said GUI an automated design of said media project, wherein said automated design is prepared through said automated rules of inserting content and said automated rules of content grouping, and wherein said rules comprise algorithms comprising at least saliency detection rules, face detection rules, object detection rules, photo similarity detection rules, and/or photo caption detection rules; (4) inserting a second set of content into said automated design based on observed user interactions with said first content, wherein said second set of content is based on information derived from observing said user interactions with said media project, and wherein said insertion of said second content alters said first set of content or adds additional content into said automated design; (5) automatically rearranging said media files based on said observed user interactions, wherein said automated design is automatically customized based on said observed user interactions via said GUI, and wherein said user interactions over-ride a photo arrangement logic specifying how photos are arranged in the media project; and (6) presenting to said user a final recommended design of said media project, wherein said final recommended design comprises said first content, said second content, and said automatic re-arranging, and wherein said final recommended design may be further customized by said user prior to ordering.
In some embodiments, the first content and the second content are inserted automatically by the server at a background computer process that is invisible to the user utilizing the GUI. In some embodiments, the steps may be practiced in any sequence. In some embodiments, the observed user interactions may comprise panning, zooming, moving, and/or swapping of said media files by the user. In some embodiments, the process sends an order to print said automated design or said final recommended design to a printer, wherein said printer prints said automated design on paper, and wherein said paper is selected from a plurality of configurable print sizes. In some embodiments, the processes described here may be practiced over the Internet network.
Other embodiments of the invention will become apparent from the detailed description below.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the present invention described herein are exemplary, and not restrictive. Embodiments will now be described, by way of examples, with reference to the accompanying drawings.
Fig. 1A is a network configuration diagram in which the present invention may be practiced.
Figs. lB-1 and 1B-2 comprise a flow diagram showing a method of background uploading of media files, according to one embodiment of the invention.
Fig. 1C is a diagram showing a Graphical User Interface (GUI) of selecting media files, according to one embodiment of the present invention (this embodiment sometimes known as "MONTAGE").
Fig. ID is a diagram showing a GUI of uploading media files, according to said embodiment of the invention.
Fig. IE is a diagram showing a GUI of building a media-based project, according to said embodiment of the invention.
Fig. IF is a diagram showing a GUI of selecting a theme, according to said embodiment of the invention.
Fig. 1 G is a diagram showing a GUI of designing a front cover, according to said embodiment of the invention.
Fig. 1H is a diagram showing a GUI of arranging text and images, according to said embodiment of the invention.
Fig. 1J is a diagram showing a GUI of previewing a design, according to said embodiment of the invention.
Fig. IK is a diagram showing a GUI of inserting text, according to said embodiment of the invention.
Fig. 1L is a diagram showing a GUI of ordering a photobook, according to said embodiment of the invention.
Fig. 2A is a block diagram illustrating a network architecture of a system for facilitating a creation of photo-based projects over a communications network, in accordance with one embodiment of the present invention.
Fig. 2B is a block diagram illustrating a data flow of the system for facilitating the creation of photo-based projects over a communications network, in accordance with one embodiment of the present invention.
Fig. 2C is a flowchart depicting a general control flow of the process for facilitating photobook creation over a communications network, according to an embodiment of the present invention.
Fig. 2D is an illustrative flowchart of an image analysis process, according to one embodiment of the present invention.
Fig. 3 is a flowchart showing an automatic insertion of content into a design of a media-based project, according to an embodiment of the present invention.
Fig. 4 is a flowchart showing a first automatic and then a second manual insertion of content into a design of a media-based project, according to an embodiment of the present invention.
Fig. 5 is a flowchart showing a simultaneous automatic and manual insertion of contents into a design of a media-based project, according to an embodiment of the present invention.
Fig. 6A illustrates a diagram showing a relationship between a canvas, a design, a layout, a designset, and a lay outset, according to an embodiment of the present invention.
Fig. 6B illustrates a diagram showing an interrelationship of a theme, a layout, and a design, according to an embodiment of the present invention.
Fig. 7 is an exemplary GUI of an editing interface displaying a photo spread across multiple pages (front cover, back cover, and spine), according to an embodiment of the present invention.
Fig. 8 is a block diagram illustrating an embodiment of the present invention having an asset recommendation system, according to another embodiment of the present invention.
Fig. 9 is an architectural diagram illustrating a printing network, according to an embodiment of the present invention.
Fig. 10 depicts a frontal view of an example photobook embodiment of the present invention before a photo strip has been set into a channel on the cover of the photobook. In this embodiment, the channel passes over a top edge of the front cover, traverses a central area on the front cover, and passes over a bottom edge of the front cover.
Fig. 10A depicts an expanded frontal view of a section of the top edge of the example photobook embodiment of the invention depicted in FIG. 10.
Fig. 10B depicts a front cover, a back spine, and back cover of an example photobook embodiment of the present invention before a photo strip has been set into the channel.
Fig. 11 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover. A photographic image is not depicted on the surface of the photo strip.
Fig. 12 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for eight photographic images on the photo strip arranged as a a montage or a collage. The photo strip is wrapped over top and bottom edges of the front cover and is essentially flush with the bottom edge and top edges of the cover.
Fig. 13 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into the channel on the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for twelve
photographic images arranged as a montage or a collage. The photo strip depicted in Fig. 13 is wider and larger than the photo strip depicted in Fig. 12.
Fig. 14 depicts a frontal view of an example photobook embodiment of the present invention with a photo strip set into a channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for four photographic images.
Fig. 15 depicts a frontal view of an example photobook embodiment of the present invention with a horizontally-oriented photo strip set into the channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for eight photographic images arranged in a rectangular collage. The photo strip and channel continue over the right edge of the front cover so that the photo strip remains essentially flush with the edge.
Fig. 16 depicts a frontal view of an example photobook embodiment of the present invention with a circular photo strip set into a circular channel in the center of the front cover. The photo strip is essentially flush with the cover.
Figs. 17A, 17B, and 17C depict sequential steps of one method that could be used for making a channel into which a photo strip could be set. Fig. 17A depicts a stamp above a cover material. Fig. 17B depicts the stamp pressed into the cover material. Fig. 17C depicts a channel remaining after the stamp has been withdrawn from the cover material. The shape of the stamp, and the depth that the stamp presses into the cover material, will determine the shape and depth of the channel.
Figs. 18A and 18B depict sequential steps in a method for adding a strip of a cover material to the inside surface of a spine of a photobook according to one embodiment of the present invention.
Fig. 19 depicts a frontal view of a soft rounded spine photobook embodiment of the present invention before a photo strip has been set into a channel on the front cover.
Fig. 20 depicts a frontal view of a soft spine photobook embodiment of the present invention with a photo strip set into the channel located in the center of the front cover of the photobook so that the photo strip is essentially flush with the cover. Lines in the photo strip are hypothetical borders for six photographic images. The photo strip is wrapped over top and bottom edges of the front cover and is essentially flush with the bottom edge and top edges of the cover.
DETAILED DESCRIPTION OF THE INVENTION
DEFINITIONS
The following definitions are illustrative only and are not to be read as limiting the scope of the present invention or claims. The terms may be used as nouns (singular and plural), verbs, adjectives, etc. as the context requires.
"Photobook" and "photo-book" are names for a printed book with photo images. The photobook has a front and a back cover with one or more printed images on a photo strip. The photo strip is mounted on a cover of the photobook so that the photo strip is essentially flush with the cover. Example embodiments of photobooks of the present invention are depicted in Figs. 10-16, and Figs. 19-20. A media-based project is not limited to a photobook. A photobook is a physical manifestation of the project in the form of a printed product. A physical manifestation may not be necessary in other embodiments, such as, an online video album, an online photo album, and/or combinations thereof. In the description that follows, a photobook is used as a representative of any type of media-based project.
"Photo strip" and "photo-strip" is a strip with a single or multiple printed images of people, places, or things on the surface of the strip. The strip is generally smaller than the area of a cover of a photobook. The photo strip in preferred embodiments of the present invention has a thickness of less than 5 millimeters, more preferably less than 2 millimeters. The photo strip is generally mounted into a channel on a cover of a photobook so that the photo strip is essentially flush with the surface of the cover of the photobook.
"Channel" refers to a depression that is made in the cover, generally on the front cover's outside surface of the photobook as depicted in Figs. 10 and 19. In the channel is mounted a photo strip and thus the channel needs to have a depth slightly more than the thickness of the "photo strip" to be mounted in the channel. The channel has a horizontal area with a shape matching the shape of the photo strip to be set in the channel. Thus in some embodiments of the present invention, the channel can be made with suitable depth and area dimensions by stamping the front cover or its "cover material" using a "stamp" in a stamping method depicted in Fig. 17.
"Flush" and "essentially flush" refer to the surface of the photo strip being at the essentially same level as the surface of the front cover, after the photo strip has been mounted in the channel.
"Stamp", "stamp-press", and "channel-shaped stamp" refer to a device with a flat surface in the shape of the area of the photo strip. A "stamp press" is a machine which can be used to exert a pressure through the stamp upon a surface which may be a cover board material, or a flexible cover material. An example stamp and stamping method are depicted in Fig. 17. The method for making the channel in the flexible cover material or another cover material of the photobook by stamping a shaped stamp into a deformable material is also sometimes called de-bossing. A stamp is but one
illustrative method of compressing the compressible cover material to make the channel, and other methods are within the scope of the present invention.
"Spine" is defined as the outermost "connection between the front cover and back cover" of the photobook and appears as the surface of the back of the photobook. It bridges the front cover to the back cover. The spine can be constructed of any material. In some embodiment of the present invention, the spine comprises a flexible material. The spine can be rounded by any means. An example of a flexible and rounded spine photobook is depicted as spine 160 in Figs. 19-20. The spine may have a stiff material as a part of the spine so that the spine only flexes only at its margins where it is attached to the front and back covers. An example embodiment of a stiff spine photobook is spine 102 depicted in Figs. 10-16. In preferred embodiments of the present invention, the spine is separate from the binding of the pages. The separation between the spine and the binding may increase when the photobook is opened. In some embodiments the spine may be directly connected to the binding of the pages of the photobook.
"Collage" and "montage" refers to the appearance that results when the printed images are selected to appear in combination on the photo strip used on a cover as well as may also occur on the pages of the photobook or on end sheets.
"Cover board material" refers to a structural material, for example, a thin wooden chip board which is used to stiffen the cover of a photobook. All known materials for making book covers are intended to be within the scope of the term "cover board material," including but not limited to cardboard, wood, plastic, and so on.
"Padding material" refers to a soft material, such as but not limited to a soft foam, that may be used in a cover of the photobook. It may be present between a cover board material and a flexible cover material to improve the softness and feel of a cover of the photobook.
"Flexible cover material" is a flexible fabric, such as but not limited to a synthetic leather material, which can be used as a surface wrapping for covers of the photobook.
"Compressible cover material" and "compressible material" is meant to include any compressible material, such as but not limited to plastic foam, foam, leather, synthetic leather, and so on, that is used on a cover of a photobook to create a channel in the compressible cover material for holding the photo-strip.
"MONTAGE", "MOSAIC", and MIXBOOK are trademark names carrying embodiments of the present invention, and hence, the aforementioned trademark names may be collectively or interchangeably used in the specification and drawings to refer to the products/services offered by embodiments of the present invention. Because all of the products are created from media-files using a graphical manipulation system provided by the inventors as a network-based software application, the relative terms descriptive of the overall process and for the providing company are MONTAGE, MOSAIC, and/or MIXBOOK. The terms MONTAGE, MOSAIC, and/or MIXBOOK may be used in this specification to describe the overall media-based product creation and editing process of the
invention, the website, mobile app, or web-app through which the process is made accessible to authorized users, the photobooks or printed media-based projects themselves, and/or the service- providing company.
Reference in this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearance of the phrases "in one embodiment" in various places in the specification is not necessarily all referring to the same embodiment, nor is a separate or alternative embodiment mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.
OVERVIEW
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures, devices, activities, and methods are shown using schematic, use case, and/or flow diagrams in order to avoid obscuring the invention. Although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to suggested details are within the scope of the present invention. Similarly, although many of the features of the present invention are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the invention is set forth without any loss of generality to, and without imposing limitations upon, the invention.
Broadly, embodiments of the present invention relate to systems and methods for automating insertion of content into designs of media-based projects. Content refers to images/photos, text, and user inputs. Generally, to create a photobook online, a user needs to select photos from a user-device, such as a laptop or mobile device. Once the photos are selected, the photos are uploaded to a server. Upon image analysis of the uploaded photos by the server, the server provides essential tools to the user-device to build a photobook. A server operates within a client-server architecture of a computer system. A server comprises computer hardware and software to serve computational requests of other programs or clients. A client comprises a user-device, such as, a laptop, a smartphone, or a desktop PC. A server performs tasks on behalf of clients. A server often provides services via the Internet or other network.
However, prior art products for a photobook creation are often based on a theme or stylistic parameter. They often do not have any bearing on the inter-relationship of layouts, themes, designs and content so applied, and on how relevant photos are grouped together in the project. Prior art
products also lack photo feature extraction, object detection, and image analysis to intelligently customize a photobook for the user.
Prior art products lack a layering of photos based on photo feature extraction, object detection, and image analysis. With such shortcomings in the prior art, an Internet user or customer may need to spend undue time and labor in manually customizing layouts, colors, themes, stylistic parameters, and captions of photos and text, to be able to finally design and print-by-order a satisfactory photobook. Undue time and labor in such a manual customization is a barrier to decide on an online purchase of a photobook. Such a barrier is also known as a "friction point." Such barriers or friction points are further compounded by unfriendly graphical user interface (GUI) of prior art products and photo editing software, and result in additional inefficiencies and errors in printing a media-based product, such as a photobook. Such friction points result in loss of potential customers before the customers reach to the point where they can order the media-based product for printing. A significant fraction of potential customers is lost in such friction points.
The present invention addresses and resolves these problems by automating designs of photo- based or media-based projects. The present invention automates photo feature extraction, object detection, image analysis, photo grouping, and/or content insertion. The present invention saves time and labor in creating media-based projects by automating content insertion into the projects. In one embodiment, the present invention eliminates manual methods of prior art photo projects and automates efficient methods for such projects, the automated methods including sorting of photos, removing inadequate or undesirable photos, selecting correct photos, grouping of photos, ordering of photos in correct orders, and/or cropping or focusing on an area of photos.
Other embodiments of the present invention automate the processes that include orienting photos, re-coloring photos or performing a color correction on photos, correcting red-eyes, and/or correcting photos for brightness or contrast. The present invention places photos more accurately within photo slots of a page in a photobook. The present invention also auto-detects areas of saliency and computes photo importance for using photos intelligently for grouping and highlighting purposes. In one embodiment, the present invention presents to a user a final recommended design of a photo project, without the user's involvement in the designing aspect of the project. A final design is printable. Once printed, the final design is shippable to the user.
The present invention reduces the number of Internet users who, because of undue time and labor required by prior art media-based projects, drop off or discontinue a photobook creation process before completing a purchase of the photobook. In some embodiments, the media-based project comprises a physical manifestation of the project, resulting in a printed photobook, photo album, or other physical manifestation of the project. A photobook may be printed on a canvas or photo quality paper and bound together as a book. The media-based project also includes an online product comprising audio, video, image and text - all packaged together as a single cohesive item or product
for display or sharing through online means or electronic means, or for ordering a physical version of said online product.
SYSTEM ARCHITECTURE
Fig. 1A shows a schematic diagram of a network configuration 1A100 for practicing embodiments of the present invention (this embodiment is sometimes known as "MONTAGE"). A user-device or devices may be connected to the Internet using a wireless network or wired network. A user-device may be a smartphone 1A102, laptop 1A104, desktop PC 1A106, or tablet 1A108. The wireless network comprises a cellular tower 1A110, or a wireless router 1A112. User-devices are connected to servers comprising a web server 1A114, an application server 1A116, and a database server 1A118. The servers are connected to a user-device through the wireless network, or the wired network 1A120. The wired network 1A120 or the wireless network may employ technologies and protocols comprising Ethernet technology, Local Area Network (LAN), Wide Area Network (WAN), and optical network, and the like.
Figs. lB-1 and 1B-2 show a flow diagram of an exemplary embodiment 1B200 showing a method of background uploading of media-files while an Internet user is customizing a media-based project, which in this embodiment is a photobook. An Internet user (hereinafter "user") starts the process of creating a media-based project (hereinafter "project") using a user-device at step 1B202. The term photobook or project may be used interchangeably. The user logs in at step 1B204 using the user-device. After the user logs in at the user-device, at step 1B206 the user selects original media files or images for creating a photobook. At step 1B206, following the log-in process at the user- device, the user is provided a workspace (also called "editor") where the user can practice or utilize various embodiments of the present invention to customize, edit or create the photobook. After the selection of images is made, then at step 1B208, optionally and in some embodiments, the client- device analyses the images. Once the optional image analysis is completed at the client-device, at step 1B210 the original images are resized to smaller sized image files and a meta-data is extracted from each original image. Meta-data is a type of data that is relevant to or embedded in an image. Meta-data comprises digital data about a media file. In one embodiment, meta-data is an Exchangeable Image File format data, or "EXIF" data (also in some cases known as "Exif ' data). EXIF is a standard that specifies a format for images, sound, geolocation data (provided by a Global Positioning Receiver or GPS receiver attached to a camera), and ancillary tags used by digital cameras, smartphone cameras, scanners, and other systems that handle image and/or sound files recorded by digital cameras. EXIF data comprises a broad spectrum of data, including, but not limiting to, date and time information of images taken, GPS location of images, camera settings (camera make and model, image orientation or rotation, aperture, shutter speed, focal length, metering mode, and ISO information), a thumbnail of the image, a description of the images, and/or copyright information thereof.
At step 1B212, the resized image files and the meta-data are uploaded to the server. The resized image files are smaller in size as compared to the larger sized original images, but not so small as to be un-useable by the server for image analysis. The resized images are immediately provided to the user-device for building the project. After the resized images and the extracted meta-data of the original images are uploaded to the server, step 1B214 will follow to begin uploading the original images to the server in a background computer process at the user-device. Using the uploaded resized images and the meta-data, at step 1B216, the server performs a server-side image analysis. An image analysis comprises image optimization algorithms, image ranking algorithms, feature or object detection algorithms, use case formation techniques, layout, and/or photobook design formation techniques, as described in greater detail below. Once the server-side image analysis is completed, at step 1B218, the server converts the resized uploaded media files into multiple sizes or multiple (plural) compression levels (the term "compressed media file" or "compressed image" is used herein to refer to a media file that has been in some way modified by the server and of a different size from the resized file uploaded from the client to the server). At step 1B220, the server provides to the user- device a compressed image, or an image of a size relevant to the scope of the project (photobook). The server converts a resized image into many different sizes (called "compressed files" herein) so that each size may be called by the user-device depending on the scope of the project. The scope of the project may comprise different sizes and different resolutions of image files, wherein the server performs the server-side image reduction to prepare images of different sizes as needed by the project. While the server performs the server-side image reduction using the resized image files, the original image files are still uploading in the background computer process. At step 1B226, the server provides analytical data to the user. At step 1B222, the user pulls relevant image data comprising image files and analytical data from the server. At step 1B224, the user continues customizing (building) the photobook or project. At step 1B228, the original image files may or still may not be uploaded [completed upload] to the server. If the original image files are uploaded to the server, the user continues building the photobook as shown by step 1B224. If the original image files are not uploaded, the photobook customization still continues at step 1B234. If the original image files are uploaded and the photobook customization continues (and thus completes), the photobook will be ready to order at step 1B224. At step 1B232, the user can place an order to print the photobook. Once the photobook is ordered and printed at step 1B232, the embodiment 1B200 ends at step 1B246.
In the event that the upload of the original images is not complete, the user still customizes (and thereby completes) the photobook at step 1B234, and is ready to order the photobook at step 1B236. The user places the order at step 1B238. Once the order is placed at step 1B238 and the original images are still not uploaded, the embodiment 1B200 provides the user a widget to upload [begin a new upload or complete the ongoing upload] the original image files that the user selects. Step 1B238 allows the user to order the photobook before the upload of the original image files completes. At step 1B240, the embodiment 1B200 provides the user an optional uploading
mechanism of original image files if the user does not have time to wait until the original images are uploaded to the server before placing an order to print the photobook. However, in another embodiment, step 1B240 can happen before step 1B238 and/or before the step 1B236; that is, a widget to upload original images may be provided to the user-device before placing an order, or before the order is ready. In other words, steps discussed herein may be switched, overlapped, or altered as long as the spirit of the present invention is consistent with the plurality of embodiments described herein. An uploading widget may comprise a software application that the user can download from an embodiment of the present invention. Once the widget is downloaded and installed at the user-device, the user can begin uploading or continue uploading the original image files using the installed widget. At step 1B242, the original image files are uploaded to the server with the aid of the widget. After the original image files are uploaded to the server, at step 1B244 the server can send the order to print the finished product. The embodiment thereby ends at step 1B246.
MONTAGE SYSTEM EMBODIMENTS
Fig. 1C is a screenshot embodiment 1C300 showing a method of selecting original media files or images at a user-device. In the embodiment 1C300, a user-device may select original images by clicking a button 1C302, labeled as, for example, pick photos. A button is a soft-button in a software embodiment. The words "image(s)" and "photo(s)" are used interchangeably in this specification.
Fig. ID is a screenshot embodiment 1D400 showing an upload of original images to a workspace of a user-device after the user-device selects or picks images. A workspace at a user- device is also called an editor. A progress bar 1D402 shows the number of images uploaded or uploading to the workspace. An uploaded image 1D404 is displayed in the workspace. The embodiment 1D400 also shows an uploading image 1D410 when the image is being uploaded. The embodiment 1D400 demonstrates client-side resizing of original image files. The image 1D404 is a resized image which is smaller than the original image. A button 1D406 allows the user to add more images. After the selected images are uploaded to the workspace of the client-device, the user can click a button 1D408 to create a design of a photobook using the uploaded photos in the workspace. The button 1D408 reads "create my Montage," where MONTAGE is a trademark name, but not a functional name, of the button 1D408.
Fig. IE is a screenshot embodiment 1E500 in which the present invention does an analytical work or analysis work to build a photobook. An exemplary symbol 1E502 teaches that the present invention is building a photobook by performing the analytical work or analysis work, such as an image or video analysis.
Fig. IF is a screenshot embodiment 1F600 that displays a theme selection of a project or photobook after images are uploaded to the workspace (editor) of a user-device. All pages of the photobook are displayed in the workspace. In a page 1F604 of the photobook, an arrangement of the
images is shown. This arrangement is auto-populated by the present invention after an image analysis. The server performs the image analysis for auto-populating the arrangement. For auto- populating the design of the book, the server may use features such as, cropping, zooming, relevant images grouping, and other image editing processes. The embodiment 1F600 allows the user to select a theme 1F606. The instruction at 1F602 allows the user to select or choose the theme 1F606. Once the user is satisfied with the theme and arrangement of the images in all pages of the photobook, the user can click the button 1F608 to continue to the next step of the present invention.
Fig. 1 G is a screenshot embodiment 1 G700 that shows a cover page of a photobook and an arrangement of images therein. The cover page 1G706 uses at least an image 1G704. A plurality of images can be used in the cover page 1G706. A server auto-populates a first design of the cover page 1G706. The user has an option to further customize the cover page 1G706 by using any of the uploaded images. Standard image editing features may be applied in customizing each image on the cover page 1G706 of the photobook. The workspace displays the theme of the design at 1G702. The workspace further allows the user to upload more photos at 1G708. The button 1G710 allows the user to scroll the uploaded images. A checkout button 1G712 allows the user to order and print the photobook.
Fig. 1H is a screenshot embodiment 1H800 that shows a page of a photobook for rearranging text and images therein. The page 1H802 shows an arrangement of the uploaded images on the page. The user can shuffle images on the page 1H802 by using a shuffle button 1H804, insert text on the page 1H802 by using an insert-text button 1H806, and rearrange pages of the book by using a rearrange button 1H808. The user can additionally use one or more of the uploaded images to customize the design of the page 1H802. The user can flip to a next page of the book by clicking or pressing an arrow button 1H810.
Fig. 1J is a screenshot embodiment 1J900 showing a method of previewing and rearranging design pages of a photobook. A design page 1J904 has a theme and at least an image. A plurality of images can be used in the design page 1J904 and the theme is selected by the user. In the embodiment 1J900, the design page 1J904 comprises page 1 and page 2 of the photobook. The message at 1J902 explains that the workspace provides a method to rearrange the pages of the photobook. There is a cancel button 1J908 to cancel a rearrangement if desired. Once a rearrangement of pages is completed, the user can click a done button 1J906 to save the rearrangement.
Fig. IK is a screenshot embodiment 1K1000 showing an insertion of text to describe an image. Text can be inserted into a page of the photobook to describe an image or a group of images. A storyline or narrative of work may be added with images using text. A narrative can be added as shown in the region 1K1002.
Fig. 1L is a screenshot embodiment 1L1100 showing a method of ordering a project or photobook. Once a photobook 1L1102 is customized and ready to order, a user can select a
specification of the project to order. If the user is not happy with the customization of the photobook 1L1102, the user can go back to the editor or workspace by clicking 1L1104 to further edit/customize the pages of the photobook. A specification may be a size or dimension of the photobook. A sample specification of the project by size or dimension is shown at 1L1106. Each specification may have a different price for order. When the user agrees to a price, the user can enter a shipping address as shown at 1L1108. The user can further continue with payment options and order a printed photobook. A standard payment processing gateway maybe implemented in the present invention to order a printed copy of the photobook. AUTO-CREATION OF PROJECTS UTILIZING USE CASE DATA
Fig. 2A shows an illustration of a block diagram showing a network architecture of a system 2A200 and method for facilitating the creation of photo-based projects over a communications network in accordance with one embodiment. Fig. 2A shows various components coupled with network 2A206, which can be a circuit switched network, such as the Public Service Telephone Network (PSTN), or a packet switched network, such as the Internet or the World Wide Web, the global telephone network, a cellular network, a mobile communications network, or any combination of the above. A prominent element of Fig. 2A is the server 2A202 associated with a data repository or server-side database 2A204 and further coupled with network 2A206. Server 2A202 collects, processes and manages data from the users 2A210 and further manages the server-side database 2A204, which holds the master copy of data that is served to fulfillment entity 2A250 and client device 2A220. Fig. 2A further includes user 2A210 and his or her client computing device 2A220, which may be a desktop computer, a common computer terminal, or mobile computing devices such as smart phones, mobile phones, tablet computers, handheld computers, laptops, or the like.
Fig. 2A shows an embodiment of the present invention wherein networked client computing device 2A220 interacts with server 2A202 and server-side database 2A204 over the network 2A206. Server 2A202 may comprise one or more servers, workstations, desktop computers, computer terminals, workstations, mainframes, mobile computing devices, smart phones, mobile phones, handheld computers, laptops, or the like. Server 2A202 and client computing device 2A220 include program logic comprising computer source code, scripting language code, or interpreted language code that may be compiled to produce an executable file or computer instructions, or that may be interpreted at run-time, wherein the computer source code performs various functions of the present invention. Server 2A202 may include a software engine that executes applications as well as delivers applications, data, program code, and other information to networked computing devices, such as device 2A220. Fig. 2A further shows a database or repository 2A204, which may be a relational database comprising a Structured Query Language (SQL) database stored in a SQL server. The repository 2A204 serves data from a database, which is a repository for data used by server 2A202 and device 2A220 during the course of operation of the invention. Database 2A204 may be distributed
over one or more nodes or locations that are connected via network 2A206. The database 2A204 may include user account records, user records, and multimedia records that include images, video, audio and like.
It should be noted that although Fig. 2A shows only one server 2A202, one database 2A204, one client device 2A220, and customer 2A210, the system of the present invention supports any number of servers, databases, client devices, and customers connected via network 2A206. Also note that server 2A202 is shown as a single and independent entity; in one embodiment, the functions of server 2A202 may be integrated with another entity, such as one image provider 2A240, fulfillment entity 2A250, device 2A220, and/or the database 2A204. Further, server 2A202 and its functionality, according to a preferred embodiment, can be realized in a centralized fashion in one computer system or in a distributed fashion wherein different elements are spread across several interconnected computer systems.
Fig. 2A further shows fulfillment party 2A250, which performs product fulfillment or order fulfillment services, i.e., the process of fulfilling the obligation of server 2A202 (or the entity represented by server 2A202) to send user 2A210 one or more photo-based project products 2A252 that the customer has ordered, purchased, or requested from the server 2A202 (or the entity represented by server 2A202). Fulfillment party 2A250 may receive orders for the photo-based project products, print the photo-based project products, bind the photo-based project products, package the products, and then ship the ordered products 2A252 to the end customer, such as user 2A210. In the course of a transaction, the server 2A202 may interface with fulfillment party 2A250 to effectuate the delivery of purchased products to the customers after payment has been effectuated. Note that although fulfillment party 2A250 is shown as a single and independent entity, in one embodiment of the present invention, the functions of fulfillment party 2A250 may be integrated with another entity, such as server 2A202. Further, the functionality of fulfillment party 2A250 may be realized in a centralized fashion in one computer system or in a distributed fashion wherein different elements are spread across several interconnected computer systems.
Figs. 2B to 2C depict, among other things, the data flow (2A260) and control flow (2C200) in the process for facilitating the creation of photo-based projects over a communications network 2A206, according to one embodiment. The process 2C200 of the disclosed embodiments begins with step 2C202, wherein the user 2A210 provides, via his or her device 2A220 over the network 2A206, at least a plurality of images or photos 2B264 to the server 2A202 for storage in the database 2A204. In one embodiment, the images or photos 2B264 are provided to server 2A202 via a graphical user interface executing on the device 2A220. In another embodiment, the images or photos 2B264 are provided to server 2A202 for storage in the database 2A204 via TCP/IP and/or HTTP over network 2A206. Subsequently, server 2A202 stores the images or photos 2B264 in the database 2A204 as records 2B266. In one embodiment, the records 2B266 are stored in association with an identity for user 2A210 or in association with a user record for user 2A210.
Next, in step 2C204, the user 2A210 provides, via his device 2A220 over the network 2A206, a use-case identifier 2B266 to the server 2A202 for storage in the database 2A204. In one embodiment, the use-case identifier 2B266 is provided to server 2A202 via a graphical user interface executing on the device 2A220. In another embodiment, the use-case identifier 2B266 is provided to server 2A202 for storage in the database 2A204 via TCP/IP and/or HTTP over network 2A206. Subsequently, server 2A202 stores the use-case identifier 2B266 in the database 2A204 in association with records 2B266. In one embodiment, the use-case identifier 2B266 is stored in association with an identity for user 2A210 or in association with a user record for user 2A210.
In the following step 2C206, various photo analysis processes may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220. The photo analysis processes comprise identifying similar images, identifying faces in the images, identifying objects in the images, identifying undesirable images, and identifying relevant portions of the images. The identification of faces and objects may be accomplished via object recognition and face recognition libraries. The identification of similar images may be accomplished via an analysis and comparison of color, focus, brightness, faces, objects and the like in each image, as described in greater detail below. The identification of undesirable images may be accomplished by identifying images that are out of focus or contain too little light for a proper exposure. For blurry photos, edge detection may be used to detect any sharp edges, if any. The identification of the relevant portions of an image may be accomplished by identifying the relevant portions of an image, such as faces and objects that may have been recognized. By applying saliency filters, a bounding box may be drawn around a focus of an image. Thus, if the relevant faces or objects are located on the sides of the image, the image may be zoomed or cropped to highlight the identified relevant area. The photo analysis processes may further include identifying images with an incorrect orientation, identifying images with incorrect color, brightness or contract, and/or identifying images with red-eye.
In the following step 2C208, various photo correction processes may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220. The photo correction processes comprise orienting images that have been identified as having an incorrect orientation to an appropriate orientation, adjusting images that have been identified as having an incorrect color, brightness or contract to the correct color, brightness or contract, and removing redeye from images that have been identified as having red-eye.
In the following step 2C210, various use-case specific heuristics may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, based on results of said photo analysis, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220. The use-case specific heuristics comprise grouping images that have previously been found to be similar, grouping images having identical or similar faces (based on the
faces that were previously identified), grouping images having identical objects (based on the objects that were previously identified), removing undesirable images (based on the images that were previously identified as undesirable), and cropping images to highlight relevant portions of said images (based on the relevant portions of images that were previously identified).
In executing the use-case specific heuristics, heuristics that correspond to the use-case identifier 2B266 are utilized. Different use-cases may require different heuristics. For example, a use- case identifier 2B266 that identifies a wedding would result in heuristics that group images having the faces of the bride and groom (based on the faces that were previously identified), group images having the wedding cake or the altar (based on the objects that were previously identified), and crop images to highlight portions of said images having the faces of the bride and groom (based on the relevant portions of images that were previously identified). Instructions 2B268 in database 2A204 may instruct server 2A202 on which heuristics correspond to each potential use-case identifier 2B266 provided by user 2A210.
As an example, with regard to grouping images having the faces of the bride and groom (based on the faces that were previously identified), using face detection, identifiers may be applied to each face to determine, for example, that person A shows up in 85% of the photos, person B shows up in 73% of the photos, and person C shows up in 20% of the photos. Therefore, person A and B are most likely the bride and groom.
In the following step 2C212, various additional functions may be executed on the images or photos 2B264 stored in the database 2A204 as records 2B266, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220. The additional functions may comprise reading embedded date or location metadata from the images, performing a lookup of additional location data based on the embedded location metadata, and generating text boxes to be printed below said images upon creation of the photo-based project 2A252. The additional functions may also include ordering the images or photos 2B264 stored in the database 2A204 in chronological order, based on the time the images were taken or created.
In one example of the execution of the process of step 2C212, the server 2A202 reads an image in EXIF format, a standard that specifies the formats for images and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image files recorded by digital cameras. The EXIF format may store date and time information of the image, camera settings of the image, thumbnails of the image, descriptions of the image, and copyright information for the image. In this example, the server 2A202 may read the location for the image from the file and generate a text box reflecting said location to be printed below said image upon creation of the photo- based project 2A252. In another example, the server 2A202 may read the location for the image from the file (in a GPS coordinate format), perform a lookup to find a corresponding text string that matches the GPS coordinate, and generate a text box having said text string to be printed below said image upon creation of the photo-based project 2A252. In this example, the server 2A202 may read
the date and time for the images from their files and place the images or photos 2B264 in chronological order, based on the time the images were taken or created.
In the following step 2C214, the creation process is concluded and the resulting photo-based project product is ordered, at the direction of the user 2A210 via input provided via a graphical user interface executing on the device 2A220. In this step, the server 2A202 sends a request 2B270 to the fulfillment party 2A250 for the photo-based project product 2A252. In the following step 2C216, the fulfillment party 2A250 receives the request 2B270 for the photo-based project product 2A252, prints the photo-based project product, binds the photo-based project product, packages the product, and then ships the ordered product 2A252 to the end customer, such as user 2A210.
Fig. 2D shows an illustrative image analysis or photo processing process according to one embodiment of the invention. The image analysis process shown in Fig. 2D is illustrative only, and is not intended to be limiting. Various other image analysis processes, some of which are described later, are also within the scope of the present invention. This image analysis process describes an image similarity detection process, as illustrative of the image analysis processes that are useable with the present invention.
In one illustrative image analysis process shown in Fig. 2D, the process 2D200 examines the images for similarity by expressing a query to the image database 2D202 (all of the images) by requesting images that are similar to one of the images (query image 2D208). The process considers basic shape and color information of the photo when looking through the database for potential matches of similar photos. Several factors make this matching process difficult. The query image is typically different from the target image, so the retrieval method must allow for some distortions. Since these are natural photos taken in natural environments, the photos may suffer artifacts such as color shift, poor resolution, dithering effects, and mis-registration. Furthermore, it is important to perform the retrieval fast enough to not hamper user experience of the media-product creation process.
When the images are first received by the server, a wavelet transform 2D204 is performed on every image in the image database 2D202. By collecting just the few largest coefficients from this transform, the process distills a small "signature" for each of the images. These signatures are saved in a signature database 2D206 so that it is computationally efficient to compare them all to each other.
When the process 2D200 requires photo(s) of similarity to a given query image 2D208, the process performs a wavelet transform 2D210 on the query image 2D208 to produce a signature 2D212 for the given query image 2D208. This query signature 2D212 is compared to the signatures of the database images 2D206, and the best matches are retrieved by the process 2D200 for use in automatically creating, organizing, and presenting the media-based project to the user.
The wavelet transform is used to analyze functions at different levels of detail; it is somewhat similar to the Fourier transform, but encodes both frequency and spatial information. By saving the few largest wavelet coefficients for an image (and throwing away all of the smaller coefficients), it is
possible to recover a fairly accurate representation of the image. This property may be exploited for efficiency gains to optimize image similarity comparisons. For example, a wavelet transformed image ("signature") that incorporates 400 coefficients would require about 3% as much disk space as the original image. In one embodiment, it is possible to take a wavelet transform and keep just a few (for example, 20) coefficients for each color channel and distill from them a small "signature" for each image. Because the signature is so small, it permits very fast searching in the database.
AUTOMATIC INSERTION OF CONTENT INTO PROJECTS
Fig. 3 is a flowchart 300 of an embodiment showing an automatic insertion of content into a design. Content may comprise photos or images, text, and user inputs from a user. At step 302, a user starts a process of automatic insertion of content by using a user-device. The user logs in at the user device at step 304. After log-in, the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device. The user at step 306 selects images or photos from the user device and uploads them to a server. At step 308, the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images. As shown in the list 310, automatic image information is derived by using computer algorithms that include, but are not limited to, one or more of detecting areas of saliency, face detection, computer vision analysis, photo similarity detection, and analyzing photo information, text matrix, photo captions, and so on. Automatic image information is also derived from image meta-data. After using the automatic image information derived by image analysis, the server at step 312 outputs a default design preview (an automated design) of a photobook. The design preview at step 312 also comprises a layout that is the basis for the design.
At step 314, the server performs automatic content grouping of the uploaded images. For automatic content grouping, the server utilizes, but is not limited to, one or more of the information in the list 320. The list 320 comprises timestamps, image similarities, image sequences, photo importance, aesthetics, and so on. At step 316, the server automatically inserts content into the design of the photo project by using, but not limited to, one or more of the items (computer algorithms) in the list 324. The list 324 comprises image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, photo position in a photo slot, and so on.
The user previews the design at step 318. The preview at step 318 is populated after completing the automatic image information extraction of one or more of the items in the list 310, after step 314 of automatic content grouping, and after step 316 of automatic content insertion. The preview at step 318 is thus automatically populated by the server by using one or more of the items (computer algorithms) as shown by the lists 310, 320, and 324. After the user previews the design, the user can order the photobook at step 322. At step 326, the order is sent to a printer. The printer may be a third party printer, a social printer, or any industrial printer that is adapted to print photos on
a photo quality paper. The ordered photobook may be printed on a preferred size of a photo paper. The ordered photobook may be a shippable product to the user. The printer may be further adapted to print a shipping label comprising the order information, and other relevant information. The process ends at step 328.
Fig. 4 is a flowchart 400 of an alternative embodiment showing a first insertion based on automatic image information and a second insertion based on observed image information of content into a design of a photo project. At step 402, the process starts at a user-device. The user logs in at the user device at step 404. After the log-in, the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device. The user at step 406 selects images or photos from the user device and uploads them to a server. At step 408, the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images. As shown in the list 410, automatic image information is derived by using computer algorithms that are relevant to areas of saliency, face detection, computer vision analysis, photo similarities, photo information, text matrix, and photo captions. Automatic image information is also derived from image meta-data. After using the automatic image information derived by image analysis, the server at step 412 outputs a default design preview of a photobook.
The default design generated at step 412 is based in part on step 416 of automatic content grouping and step 414 of automatic content insertion. At step 416, the server performs automatic content grouping of the uploaded images. For automatic content grouping, the server utilizes the information comprising timestamps, image similarities, image sequences, photo importance, and aesthetics. At step 414, the server automatically inserts content into the design of the photo project by using items (computer algorithms) comprising image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, and photo position in a photo slot.
The default design at step 412 is based on a combination of computer algorithms in lists 410,
414, and 416. The user previews the default design created by the server at the user device at step 418. Furthermore, the user customizes the design at step 420 by adding user interactions or user inputs. User interactions or user inputs comprise area of interest selection, photo importance determination, and so on. User interactions or user inputs are performed manually by the user. By observing and monitoring the user interactions at the GUI of the user device, the system is able to collect observed image information, which the system can use to improve on the automatic image information obtained by the system from image analysis. The system can monitor for such user interactions as zooming, panning, swapping photos, adding captions to photos, and so on. After the customization by the user, a final recommended design based on the automatic image information and the observed image information is generated by the system, and can be previewed by the user at step 422. The user can then perform additional customizations to the final recommended design, and then
order the photobook at step 424. At step 426, the order is sent to a printer as described previously. The process ends at step 428.
Fig. 5 is a flowchart 500 of an alternative embodiment showing an insertion of content based on automatic and observed image information. At step 502, the process starts at a user-device. The user logs in at the user device at step 504. After the log-in, the user enters a workspace, an editor, or a graphical user interface (GUI) at the user device. The user at step 506 selects images or photos from the user device and uploads them to a server.
At step 510, the server performs an image analysis on the uploaded images. After the images are analyzed, the server derives automatic image information from the uploaded images. As shown in the list 512, automatic image information is derived by using computer algorithms that are relevant to areas of saliency, face detection, computer vision analysis, photo similarities, photo information, text matrix, photo captions, and so on. Automatic image information may also be derived from image meta-data. While the user is uploading original images to the server, the user can interact with a layout and a design of a photobook, and thus contribute to creating user inputs or user interactions with the design. At step 508, the user interacts with the images that are already uploaded to the server. User interactions are also called manual interactions, manually observed information, user inputs, or user's behavioral observation. As shown in the list 518, user interactions comprise one or more of, but are not limited to, areas of interest in a photo, photo importance, panning, zooming, switching photos, and so on. Not every possible user interaction is shown in list 518 for simplicity of explanation. User inputs may also be called tweaks, and give rise to observed image information. By using user's manual inputs or user interactions (observed image information) along with the automatic image information derived though image analysis by the server, a default preview of the design (an automated design) is generated at step 514. The automated design is subject to manual edit by the user at any time before an order to print is placed and confirmed.
At step 520, automatic content grouping of images is performed by using both observed image information from the user and automatic image information from the server. Errors made by the server in automatic content grouping are manually corrected by the user at step 520. For automatic content grouping, the server utilizes one or more of, but is not limited to, the items or information in the list 516. The list 516 comprises timestamps, image similarities, image sequences, photo importance, aesthetics, and so on. User inputs at step 520 come from, but are not limited to, one or more items on the list 518. At step 524, manual tweaks are performed to further customize content. The server automatically inserts content into the design of the photo project by using, but is not limited to, one or more of the items (computer algorithms) in the list 522. The list 522 comprises image ranking, buffer calculation, areas of interest in an image, scores for insertion, best design determination, dynamic text insertion, photo position in a photo slot, and so on. Manual tweaks come from the list 518, but are not limited to the items shown.
After performing manual tweaks at steps 520 and 524, the user previews a final recommended design of the photo project at step 526. When the user is happy with the final recommended design, the user can order a photobook at step 528. The order may then be sent to a printer as previously described. The process completes at step 530.
Fig. 6A illustrates a diagram showing a relationship between a canvas, a design, a layout, a designset, and a lay outset, according to an embodiment of the present invention. A lay outset 650 gives rise to a layout 654, which itself gives rise to a design 656. The design 656 inherits from both a layout 654, and indirectly its layoutset 650. The design 656 is selected from a designset 652. Each design 656 specifies a style and formatting of content in a layout 654 from which it depends, while each layout 654 specifies a location of photo slots and/or text slots without reference to styling. Each designset 652 is always tied to a layout 654, which itself is always tied to a layoutset 650. Finally, the design 656 gives rise to a canvas 658 through an addition of content by a user, automatically by the server, or both. This inter-relationship of a canvas, a design, and a layout ensures system flexibility and efficiency whenever designs have to be modified.
Fig. 6B illustrates a diagram 600 showing an inter-relationship of a theme, a layout, and a design, as well as their corresponding sets. At a user device, a user selects images to build a photobook. Once the images are selected, the user can select a theme 616. The theme 616 corresponds to designsets 614 and 618. Each designset is a collection of designs. For example, the designset 614 comprises designs 608, 610, and 612. Similarly, the designset 618 comprises designs 620, 622, and other designs not shown. Each design depends on a layout. For example, the design 608 depends on a layout 604. Each layout depends on a layoutset. For example, the layout 604 depends on a layoutset 602. A layout can have or create many designs. For example, layout 606 can create designs 620, 622, and another designs (not labeled). A design at the user device may be presented in a canvas 626. A canvas 626 represents how a design will be printed on paper. A project 624 can be represented by a canvas 626 or a canvas 628. For example, the design 622 is reflected on canvas 628. Each layoutset serves as a basis for many designsets, and each layout serves as a basis for many designs. A theme 616 selected is consistent with a layout as well as a design, thereby maintaining an inter-relationship between a theme, a layout, and a design. Thus, this interrelationship offers greater control for a responsive design update and for auto-flow/auto-fill algorithms, thereby saving time and labor in creating a photobook.
OTHER EXEMPLARY EMBODIMENTS OF THE PRESENT INVENTION
Fig. 7 is an exemplary screenshot of an editing interface 700 displaying a photo-spread including a front cover, back cover, and spine according to an embodiment of the present invention. Interface 700 has a storyboard display 701 that includes all of the pages of the photobook being created. The inventors provide a unique paradigm for relating items across multiple pages including special pages, in this case, the front cover 703, the back cover 704, and the spine of the book 705.
When a user edits a spread that contains more than one page, the photos, shapes and stickers can be "overflowed" from one page to other pages within the spread. When the items are overflowing, they are linked together such that when an item changes on one page, its linked item gets updated reflectively on the other page.
Fig. 8 is a block diagram illustrating an asset recommendation system according to an embodiment of the present invention. In one embodiment of the present invention, asset recommendations are made to a user who is editing a current project using the MIXBOOK collage editor. System architecture 800 includes a user 801 that is currently online with runtime server 804 running an asset recommendation engine (ARE) 805. The system of the invention has access to all of the assets that have been designed by system designers, users whom may be registered as asset contributors, and any other user contributions to the system. Creative assets 803 include all of the stickers, shapes, backgrounds, layouts, photos, and any other accessible imagery.
Finally, Fig. 9 is an architectural diagram illustrating a printing network 900 according to an embodiment of the present invention. Printing network 900 includes a user 901 connected online to a runtime server 902. Runtime server 902 is analogous to runtime servers previously described in this specification. Runtime server 902 includes a digital medium coupled thereto and adapted to contain all of the SW and data required to enable server function.
When user 901 is editing a project using a MIXBOOK editor (MBE) such as MBE 906, for example, the user is connected in session with server 902. As such, the user may decide to print a finished project such as a photobook by invoking a printing option such as print project option 908 illustrated in exploded view. Such an action invokes a printing engine 907 that is responsible for rendering the user's project in a ready state for printing according to a final end product selected by the user.
The printing engine sends output files for printing to a third-party printing service 903. Printing service 903 represents any online printing service. Third-party printer 903 includes a printing server 904 and a connected printer 905 representing printing capability. Printing engine 907 is a flash based that is enhanced to produce high resolution output files to print. Print engine 907 is a single print engine that is involved from the point of creating a project to the point of printing out the project.
PRINTED PHOTOBOOK AND PHOTOALBUM EMBODIMENTS Example printed embodiments of the present invention are photobooks with one or more photo strips, having one or more photographic images inserted (mounted or set) into one or more cover surfaces of the photobook, wherein the one or more photographic images of each photo strip is essentially flush with the cover surface. A photo strip of an embodiment of the present invention may be inserted into a channel on a cover surface and may be attached using an adhesive or other bonding method. Photo strips for the present invention may be created in any manner and can have images of any kind arranged in any pattern, shape, or form.
In one preferred photobook embodiment of the present invention, the one or more photographic images of the photo strip on the cover occupy only part of the cover surface of the photobook and are essentially flush with the photobook cover. The observer obtains a crisp visual experience of one or more photographic images with a photobook cover, providing a border flush with the one or more photographic images of each photo strip. This embodiment of the invention solves a visual depreciation problem common in the prior art that is felt when the photobook cover is very thick and relatively cumbersome, if the front cover is a thick picture frame mounting. Another problem with prior art photobooks occurs when the cover images are significantly recessed below the photobook cover's surface since this causes the cover image to appear walled-in. Another prior art problem is the cover image of a photobook can become dim or too light-reflecting if the image is placed behind a protective window of plastic or glass, or is laminated, or embedded in plastic.
An example photobook embodiment of the present invention is depicted in Fig. 10, which has a front cover 114; and a back cover edge 106. A stiff flat spine 102 connects the front cover to the back cover. There are inside pages 110 for photographic images, and the pages are interconnected with first and last inner pages attached to the inside of the front and back covers. Also, Fig. 10 depicts a frontal view of a photobook cover channel 112. This channel runs from the top edge 108 to the bottom edge 118 of the front cover 114. Channel 112 is the location where a photo strip is attached (mounted or set) using an adhesive or other means, so that the photo strip and its images are essentially flush with the front cover 114. Channel edge 116 indicates the depth of channel 112. The channel depth from the cover surface is made to be the thickness of the photo strip plus the thickness of an adhesive layer or other attachment means that has a thickness.
Fig. 10A depicts in greater detail various features of an example photobook of the present invention. Depicted are top edge 108 of channel 112, vertical edge 116 of channel 112, front cover 114, top edge 104 of front cover 114, top edge 106 of back cover (not shown), and top view of inside pages 110. Fig. 10A also provides details of three front cover channel surfaces, recessed top edge 108 of channel 112; channel side wall (edge) 116; and front cover channel 112, into which the photo strip would be mounted. When a photo strip is set into the channel 112 and wraps over top edge 108, the photographic images of the photo strip are essentially flush with top edge surface 104 as well as with cover surface 114.
Fig. 10B depicts a view of the example photobook front cover 114, back cover 119, top edge 116 of back cover 119, flat stiff spine 102, channel 112, top edge 104 of front cover 114, top edge 108 of channel 112, bottom edge 118 of channel 112, side 116 of channel 112, and inside pages 110. Top edge 104 of the front cover has a top edge channel 108 in the mid-section of top edge 104. Front cover 104 continues with a channel 112 from the top edge channel 108, and channel 112 traverses a central area or mid-section of the front cover, with channel 112 continuing to bottom edge channel 118 of the front cover. A photo strip (not shown) could be set down into channel 112 and photo strip's top end could be wrapped over recessed top edge 108, while photo strip's bottom end could be wrapped over recessed bottom edge 118, so that the photo strip is essentially flush with the front cover 114, and top edge 104 surface, as well as bottom edge surface of the front cover 114.
Fig. 11 depicts a frontal view of an example embodiment of a photobook of the present invention, similar to the photobook depicted in Fig. 10, with a hypothetical single image on a photo strip 122 set in a channel 112 (not visible), so that the image and photo strip with the image are essentially flush with front cover 114, as suggested at locations 124, 126, and 126, where an edge of the photo strip in adjacent to a part of front cover 114 and at location 120 for top edge 104 of the front cover.
Fig. 12 depicts a frontal view of an example embodiment of a photobook of the present invention similar to the photobook depicted in Fig. 11, however the photo strip depicts hypothetical borders for eight photographic images as a collage 128. The photo strip is essentially flush with front cover 114, as suggested at locations 121, 125, and 126 of front cover 114.
Fig. 13 depicts a frontal view of another example embodiment of a photobook of the present invention, with a photo strip 127 depicting hypothetical borders for twelve photographic images as a collage set in the single channel on the front cover 114 (not visible), so that photo strip 127 is essentially flush with cover 114, as indicated at locations 120, 124, 126, and 129 of front cover 114. Fig. 13 photo strip 127 is wider than collage 128 of Fig. 12.
Fig. 14 depicts a frontal view of an example embodiment of a photobook of the present invention with a photo strip 131 , depicting hypothetical borders for four photographic images as a collage set in the single channel on the front cover 114 (not visible), so that photo strip 127 is essentially flush with the front cover, as indicated at locations 126 and 133 of front cover 114.
Fig. 15 depicts a frontal view of another example embodiment of a photobook of the present invention, with a horizontal photo strip 135 in the single channel on the front cover 114 (not visible), so that photo strip 135 is essentially flush with cover 114 as indicated at locations 120, 136, and 137 of front cover 114. Photo strip 135 depicts hypothetical borders for a collage (montage) of eight photographic images. Photo strip 135 wraps over the right side edge 120 of the front cover 114.
Fig. 16 depicts a frontal view of another example embodiment of a photobook of the present invention, with circular photo strip 141 set into a circular front cover channel (not visible), so that the depicted hypothetical photographic image is essentially flush with the front cover 114, as suggested at
locations 139 and 143. It is contemplated that there could be multiple images in circular photo strip 141 as a montage, and there could be multiple circular photo strips set in multiple circular channels on cover 114 of the present invention.
Additionally, the present invention further includes one or any combination of the following embodiments (1) embodiments wherein the photo strip is straight and has a constant width; (2) embodiments wherein the photo strip is located centrally on the first cover; (3) embodiments wherein at least two edges of the photo strip are located at edges of the first cover; (4) embodiments wherein edges of the photo strip cross an edge of the first cover, and a part of the photo strip is located on an outside face of the first cover and a part of the photo strip is located on an inside face of the first cover; (5) embodiments wherein edges of the photo strip are within edges of the first cover; (6) embodiments wherein the photo strip displays the one or more photographic images on the photo strip as a collage; (7) embodiments further comprising one or more printed photographic images essentially on a cover material of the photobook; (8) embodiments wherein the one or more printed photographic images is essentially on a material comprising a paper; (9) embodiments wherein one or more of the covers of the photobook comprises a rigid board cover material; (10) embodiments wherein one or more of the covers of the photobook comprises a flexible cover material; (11) embodiments wherein one or more of the covers of the photobook comprises a rigid board cover material wrapped with a flexible cover material; and (12) embodiments wherein the flexible cover material has one or more channels into which one or more photo strips are inserted so that the one or more photographic images of one or more inserted photo strips are essentially flush with the cover surfaces next to the inserted photo strips.
In some photobook embodiments of the present invention, the photobook has a feature selected from the group consisting of a photo strip which displays a collage of images, a back cover which contains a padding material, a spine which comprises a flexible cover material, a spine which is rounded, a binding decoration which is located between the spine and the binding of the pages, an end sheet before the binding of the pages, an end sheet after the binding of the pages, an end sheet which displays a collage of images, pages which display a collage of images, a page which displays a single image, two consecutive pages which display a portion of the same image, and any combination of features thereof. In some photobook embodiments of the present invention, the front cover, the spine, and the back cover are wrapped with a flexible cover material. In some photobook embodiments of the present invention, a part of the front cover, a part of the spine, and a part of the back cover have a flexible cover material surface. In some embodiments of the present invention a part of the front cover, a part of the spine, and a part of the back cover have a flexible cover material surface.
METHODS OF MANUFACTURE
Figs. 17A, 17B, and 17C depict in cross-section three sequential steps of one method embodiment of the invention for making a channel 112 in a photobook cover 114 (or more simply cover material 114). Any method for making a channel 112 in a cover or a cover material may be used, including cutting, hammering, abrading, dissolving, or melting (depending upon characteristics of cover material 114). Fig. 17A depicts press 132 with a stamp 134 above a cover material 114. Stamping area of stamp 134 is selected to have a shape similar to the shape of the photo strip intended so that the channel 112 has the shape of the photo strip. One or more channel shaped stamps used in conjunction to make a complex shaped area channel may be complex, when the channel needs to have a complex shape. For example, as one of skill in the art can infer from Figs. 10-16, and 19-20 depictions of channel shape (or photo strip shape), that the channel may have a shape selected from the group consisting of a square, a rectangle, a triangle, a pentagon, a hexagon, a polygon, a circle, an ellipse, a rounded vertices polygon, a shape having a hole, and a combination thereof. Fundamentally, a channel needs to be made to have a depth matching the total thickness of the photo strip and its adhesive attaching it in the channel. Next, as depicted in Fig. 17B, press stamp 136 is compressing an area of the cover material. Then, as depicted in Fig. 17C, press stamp 136 has been withdrawn from cover material 114, and cover material 114 has a resulting channel 112 therein.
By experimentation with the stamping machine variables, including stamp pressure, temperature, moisture, and duration of compression, the channel can be formed in a particular flexible cover material, with a depth matching the thickness of the photo strip and its underlying adhesive layer, so that the photo strip can be set into the channel and is essentially flush with the cover material surface. The time required and amount of pressure needed to form the channel varies with the properties of the cover material, as is known in the art; sometimes termed de-bossing. For different photobook cover materials, the pressure needed by the stamp press to form the channel in a cover material may vary considerably. It is expected that for some soft materials, the stamp pressure would need to be only 0.1 psi. It is expected that for very hard materials, the stamp pressure might need to be as high as 20,000 psi (pounds per square inch). Another factor is the conditioning of the cover material, such as its temperature, stretch and hydration. A temperature may be increased to soften some synthetic cover materials before their pressing to form the channel. Natural fibers and leathers may be softened by a controlled aqueous hydration of the material prior to pressing the material. In addition, a softer material surface may be placed beneath the cover material when stamping the cover material to make the channel.
For some photo book embodiments of the present invention, it is preferred to make the channel in the cover material (such as a flexible cover material) before wrapping the cover material onto the front cover and back cover of the photobook. One method is to then attach the photo strip to the flexible cover material. It is contemplated, however, for some embodiments of the present invention that the strip is first attached by some means such as an adhesive, and then the photo strip
attached to the cover material is pressed so that the photo strip becomes essentially flush with adjacent cover material. This method assumes that the photographic images on the photo strip can tolerate such stamping.
In other embodiments of the invention, the method for making the channel in the cover of the photobook further comprises applying two strips of a hard material to the sides of the cover to make cover walls for a channel. The channel-making method may combine this method of building up the walls channel with stamp press compression of a flexible cover material being wrapped over the two strips of the hard material. In some preferred embodiments of the present invention, the photobook has a cover comprising a rigid cover board material wrapped with a flexible cover material. The flexible cover material in such embodiments has been pressed (or de-bossed) using a stamping process as exemplified in Figs. 17A, 17B, and 17C to create the needed channel. The flexible cover material is wrapped around the rigid cover board material to make the cover and then the photo strip is mounted (set) into the channel of the flexible cover material.
In general, the photo strip on a cover of a photobook of some embodiments of the present invention does not cover the entire cover. As depicted in Fig. 11, the cover-mounted photo strip and the cover's channel may be the length the photobook cover. As depicted in Figs. 14 and 16 the photo strip and the sub-surface channel may have only a central location on the cover with a cover margin on all sides of the cover. A preferred horizontal width for the photo strip mounted on a cover of the photobook is between about 20 percent to about 80 percent of the horizontal length of the cover, more preferably between about 40 percent to about 70 percent of the horizontal length of the cover, and most preferably between about 50 percent to about 65 percent of the horizontal length of the cover.
Preferred embodiments of the photobook have a photo strip with a plurality of images in the form of a collage or montage. In another embodiment of the present invention, the photo strip and the channel run the length of the cover and both the strip and the channel continue around edges of the photobook with the photo strip terminating on the inside of the cover. It is contemplated that in some embodiments the photo strip material could be mounted in a long channel so that the images of the photo strip mounted in the channel continue from one cover, across the spine, and onto the other cover. A preferred flexible cover material, for some embodiments of the photo book, is thick synthetic leather. Any cover material, including glass, metal, plastic, or wood could be used provided that a suitable method is used to make the channel for the photo strip.
Figs. 18A and 18B illustrate an option for some embodiments of the present invention in which an extra strip 150 of the cover material may be attached to the inside 140 of the spine 148 of the photobook 142. This extra strip 150 can be used to cover up unevenness along the spine 148 caused by the flexible cover material being wrapped from an outside cover surface to an inside cover surface 144. This extra strip 150 may also help to make the inside of the spine flatter. Additional cover materials, such as padding 146, may be added to the front or back covers as needed to enhance the feel of the photobook.
A rigid cover board used for the present invention may comprise a material such as a high impact fiberboard, laminated plywood, a chip board, a metal sheet, a plastic sheet, and any combination thereof. A chip board is a preferred rigid board cover material. For some embodiments of the present invention, a rigid cover board has a thickness between 0.01 to about 1 inch, and preferably between 0.1 to about 0.5 inches. In one embodiment of the present invention, the front cover has the same thickness as the back cover, and uses a 0.125 inch thick chip board with a 0.0625 inch thick strip on top of the sides of the chip board.
It is contemplated that a cover material which may be used for the present invention may be selected from the group consisting of a wood, a wood chip board, a plastic, a metal foil, a foam material, a soft fabric, a metallic fabric, a fiberglass, a polymer, a fiber, a laminate, a glass, a ceramic, a leather, an animal skin, a mineral, a stone, a transparent material, a reflective material, an opaque material, a translucent material, a soft material, a padded material, a woven material, a waterproof material, a synthetic material, a flexible material, a rigid material, a rough material, a smooth material, a fragrant material, a velour material, a silk material, and a combination thereof.
To attach a photo strip to a cover material, such as, for example, a flexible or rigid cover material, an adhesive, a bonding agent, or another attaching means is used for making some embodiments of the present invention, with the proviso that adhesive, bonding agent, or other attaching means has a compatibility with attaching the photo strip to the cover material. An adhesive, a bonding agent, or another attaching means with photobook materials should not be used which may stain, degrade, or discolor the photobook cover material or the photo strip. For some embodiments of the present invention, one method for attaching the photo strip to a channel on a cover of the photobook comprises printing the photo strip on an adhesive backed paper or applying an adhesive to the back of the photo strip; and setting the photo strip in the channel so that the adhesive holds the photo strip to surfaces of the channel.
One method for securing the photo strip and creating a clean finish is to terminate a photo strip on the inside face of the front cover after the photo strip has been wrapped over edges of a cover. In some embodiments of the present invention, the photo strip is terminated on the inside face of a cover after the photo strip has been wrapped over the cover edges by approximately 0.5 inches from a cover edge. The photo strip can of course be permitted to wrap further on the back side of the front cover. Optionally, the photo strip may display photographic images only on an outer side of the cover, with distal ends of the photo strip displaying no photographic images but colored to match or contrast with the coloration of the photobook cover.
The present invention preferably uses end sheets, which are a material layer that is attached to inside surfaces of the front and back cover of the photobook. An end sheet can be decorated in any manner for the present invention. Instead of normal solid color end sheets, optionally, a photo collage or a montage of some of the photographic images from pages of the photobook may be printed on the end sheets of a photobook of the present invention. End sheets may be comprised of a cover material,
including a paper material. The end sheets and photographic images of the photo strip can be a collage of images, symbols, designs, and the like, and can have transparency, filters, overlays, and any other effects. In addition, there can be borders between photographic images within a photo strip and within end sheets. Any kind of variation in photo collages known in collage art methods can be used on a cover surface or end sheet. Cover surface and end sheet surface designs can, for example, include various grids, equal sized boxes, rectangles, polygons and rounded shapes, fractal patterns, and the like, in a repeat or random pattern.
One method embodiment of the present invention is a method for making a photobook, the photobook comprising a photo strip set in a channel on a cover of the photobook, the channel having an area less than the area of the cover, the photo strip displaying one or multiple photographic images, so that the one or more photographic images are essentially flush with the cover, the method for making said photobook comprising (a) cutting pieces of a rigid cover board material for a front cover, and a back cover of the photo book; (b) cutting a piece of a flexible cover material to a size and to a shape for a wrapping of the piece of the flexible cover material over the pieces of the rigid board cover materials for making the front cover, and for making the back cover; (c) making a connection between the front cover and the back cover; (d) making the channel by pressing a channel shaped stamp into the piece of the flexible cover material, the channel shaped stamp having a shape and dimensions of the photo strip to be set into the channel; (e) holding the channel pressed into the piece of the flexible cover material over the rigid cover board materials at a cover location selected for the channel, while wrapping the piece of flexible cover material over the front cover and over the back cover; (f) setting the photo strip having one or more photographic images into the channel of the piece of the flexible cover material so that the one or more photographic images of the photo strip are essentially flush with the cover adjacent to the cover location of the channel; (g) printing photographic images on pages to be situated in the photobook between the front cover and the back cover; and (h) situating the pages with the photographic images in the photobook between the front cover and the back cover.
PHOTOBOOK WITH ROUNDED OR SOFT SPINE EMBODIMENTS
In some preferred photobook embodiments of the present invention, the photobook has a rounded spine 16CT which comprises a flexible cover material. See rounded spine 160 in Figs. 19-20. The spine 160, for example, may be made of the flexible cover material used on the front and back covers of the photobook. One aesthetic advantage of the soft spine photobook embodiments of the present invention is that the photo strip on the front cover can have equal-width panels and be centered on the photobook at the same time. In contrast, the flat stiff spine 102 photobook (as depicted in Fig. 11) can have equal-width front cover panels, but at the same time cannot be centered with respect to the overall photobook.
Fig. 19 depicts a frontal view example of a photobook embodiment of the present invention prior to insertion of a photo strip into channel 170, which in this particular example vertically traverses the front cover, and over upper edge 166 and lower edge 174 of front cover 172. Additionally, Fig. 19 depicts front cover 172 connected to back cover 162 via spine 160. There is a flat area 164 on each cover side that smoothly transitions to become the rounded shoulders of the soft spine, unlike the embodiments with a flat stiff spine which are depicted in Figs. 10, 11, 12, 13, 14, 15, and 16. Fig. 19 also depicts that inside pages 168 near the spine are functionally interconnected as a binding. The first and last inside pages of some photobooks are connected to the front and back covers of the photobook. The pages are connected together and to covers of the photobook. Channel 170 has a depth which can be inferred from the depiction of the channel depth at channel edge 176. Channel depth is made to be the same as the sum of the photo strip thickness plus the thickness of the adhesive that is used to attach the photo strip to the channel. In this way, the photo strip when mounted in the channel using the adhesive, is essentially flush with the cover of the photobook.
Fig. 20 depicts a soft rounded spine photobook embodiment of the present invention, with a photo strip 184 set into channel 170 (not visible here) between front cover panels 172 and 192. Related Fig. 19 depicts channel 170. In Fig. 20, the example photobook embodiment of the present invention has a photo strip which traverses the front cover vertically after wrapping around top edge 180 and the bottom edge. In Fig. 20, the depicted photo strip 184 is depicted as a hypothetical collage of six photographic images (specific photographic images not depicted), where only hypothetical borders have been drawn within the photo strip 184. The hypothetical six photographic images of the photo strip 184 are depicted set in a channel 170 and are essentially flush with the surfaces of cover panels 172 and 192. In Fig. 20, the photo strip is essentially flush with the cover, for example, at upper right corner 188 of the photo strip, at mid right side 190 of the photo strip, at mid-left side 182 of the photo strip, and at bottom left edge 186 of the photo strip. In other embodiments of the present invention, it is contemplated that there be multiple photo strips set into each cover channel with each photo strip having one or more photographic images. In addition, it is contemplated for the present invention that the photobook can be of any size, and dimensions, such as in terms of its height, width, and the thickness overall, and of its covers. CONCLUSIONS
Certain terminology may be used in the specification for convenience only and is not limiting. The words "lower" and "upper" and "top" and "bottom" designate directions in the drawings to which reference is made. There is also a front cover and a back cover designation for a photobook of the present invention with each cover having an outside and an inside. The terminology includes the words above specifically mentioned, derivatives thereof and words of similar import. Where a term is provided in the singular, the inventors also contemplate aspects of the invention described by the plural of that term. As used in this specification and in the appended claims, the singular forms "a",
"an", and "the" include plural references unless the context clearly dictates otherwise, e.g., "a surface" includes a plurality of surfaces. Thus, for example, a reference to "a method" includes one or more methods, and/or steps of the type described herein and/or which will become apparent to those persons skilled in the art upon reading this disclosure. The use of the word "or" between a first item and a second item shall mean the group of items equaling a first item, a second item, and both the first item and the second item. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, the preferred methods, constructs and materials are now described. All publications mentioned herein are incorporated herein by reference in their entirety. Where there are discrepancies in terms and definitions used in references that are incorporated by reference, the terms used in this application shall have the definitions given herein. It will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular embodiments disclosed, but it is intended to cover modifications within the spirit and scope of the present invention as defined by the appended claims.
One of ordinary skill in the art knows that the use cases, structures, schematics, and flow diagrams may be performed in other orders or combinations, but the inventive concept of the present invention remains without departing from the broader spirit of the invention. Every embodiment may be unique, and methods/steps may be either shortened or lengthened, overlapped with the other activities, postponed, delayed, and continued after a time gap, such that every user is accommodated to practice the methods of the present invention.
The present invention may be implemented in hardware and/or in software. Many components of the system, for example, network interfaces etc., have not been shown, so as not to obscure the present invention. However, one of ordinary skill in the art would appreciate that the system necessarily includes these components. A user-device is a hardware that includes at least one processor coupled to a memory. The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory e.g., cache memories, non-volatile or back-up memories (e.g. programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device.
The hardware of a user-device also typically receives a number of inputs and outputs for communicating information externally. For interface with a user, the hardware may include one or more user input devices (e.g., a keyboard, a mouse, a scanner, a microphone, a web camera, etc.) and
a display (e.g., a Liquid Crystal Display (LCD) panel). For additional storage, the hardware my also include one or more mass storage devices, e.g., a floppy or other removable disk drive, a hard disk drive, a Direct Access Storage Device (DASD), an optical drive (e.g. a Compact Disk (CD) drive, a Digital Versatile Disk (DVD) drive, etc.) and/or a tape drive, among others. Furthermore, the hardware may include an interface with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others) to permit the communication of information with other computers coupled to the networks. It should be appreciated that the hardware typically includes suitable analog and/or digital interfaces to communicate with each other.
In general, the method executed to implement the embodiments of the invention, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as "computer program(s)" or "computer code(s)." The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations necessary to execute elements involving the various aspects of the invention. Moreover, while the invention has been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of machine or computer-readable media used to actually effect the distribution. Examples of computer-readable media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.), and digital and analog communication media.
Although the present invention has been described with reference to specific exemplary embodiments, it will be evident that the various modification and changes can be made to these embodiments without departing from the broader spirit of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than in a restrictive sense. It will also be apparent to the skilled artisan that the embodiments described above are specific examples of a single broader invention which may have greater scope than any of the singular descriptions taught. There may be many alterations made in the descriptions without departing from the spirit and scope of the present invention.
Claims
1. A non-transitory, physical storage medium storing program code, which when executed by a processor of a server, causes the processor to perform a process for creating a photo-project over a communications network, the process comprising steps of:
establishing a client-server connection between the server and a computing device over the communications network;
receiving a plurality of images from a user of said computing device;
storing said plurality of images in a data repository connected to the server;
presenting the photo-project to the user of said computing device;
receiving a print order from the user for a printed photobook corresponding to the photo- project; and
in response to the print order, instructing a print vendor to print the photobook,
wherein said photobook comprises a front cover with a channel therein, and a photo-strip permanently set into the channel, wherein the photo-strip displays one or more of the images and is flush with the front cover, and
wherein the front cover of the photobook is made by compressing a compressible cover material to form the channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip, wrapping the compressible cover material with the channel around a cover board material to make the front cover, and setting the photo-strip permanently into the channel to create the photobook.
2. The non-transitory, physical storage medium of claim 1, further comprising program code for process steps of:
monitoring user interactions on said computing device while the user is manipulating said plurality of images to derive observed image information from said user interactions; and
modifying automatically an arrangement of content in said photo-project based on said observed image information to produce a recommended design of the photo-project.
3. The non-transitory, physical storage medium of claim 2, wherein said observed image information is derived through analysis of said user interactions comprising areas of interest selection and photo importance selection.
4. The non-transitory, physical storage medium of claim 1, further comprising program code for process steps of:
deriving automatic image information though one or more algorithms for saliency detection, face detection, and photo similarity detection in order to automatically arrange said plurality of images.
5. The non-transitory, physical storage medium of claim 1, further comprising program code for process steps of:
automatically grouping said plurality of images through at least one of timestamp analysis, image similarity analysis, and photo importance scoring.
6. The non-transitory, physical storage medium of claim 1, further comprising program code for process steps of:
automatically inserting one or more of the plurality of images through one or more algorithms for a ranking of images, an area of interest selection on an image, a score for inserting an image into a photo slot, and a determination of a best design of a page.
7. The non-transitory, physical storage medium of claim 6, wherein said ranking of images comprises a plurality of image heuristics for calculating image importance.
8. The non-transitory, physical storage medium of claim 1, wherein the photo-strip of the photobook wraps over a portion of a top inside edge and a bottom inside edge of the front cover of the photobook.
9. A method for manufacturing a photobook, comprising:
instructing a printer to print a photo-strip having one or more images;
producing a front cover of the photobook by compressing a compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip;
wrapping the compressible cover material with the channel around a cover board material to make the front cover; and
setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
10. The method of claim 9, wherein the photo-strip of the photobook wraps over a portion of a top inside edge and a bottom inside edge of the front cover of the photobook.
11. The method of claim 9, further comprising:
instructing a stamping machine to create the channel in the compressible cover material.
12. A system for creating and ordering a photo-project and an associated photobook, the system comprising:
a server having a non-transitory, physical storage medium and a processor; and
a plurality of program code embodied on the non-transitory, physical storage medium and executable by said processor, said plurality of program code causing said processor to execute a process to:
establish a client-server connection between the server and at least one user device over a communications network;
receive a plurality of images from a user of said user device;
store said plurality of images in a data repository connected to the server;
present the photo-project to the user of said user device;
receive a print order from the user for the associated photobook corresponding to the photo- project;
in response to the print order, instructing a print vendor to print a photo-strip having one or more of the images;
instruct the print vendor to produce a front cover of the photobook by compressing a compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip;
instruct the print vendor to wrap the compressible cover material with the channel around a cover board material to make the front cover; and
instruct the print vendor to set the photo-strip permanently into the channel and flush with the front cover to create the photobook for delivery to one or more recipients.
13. The system of claim 12, wherein the photo-strip of the photobook wraps over a portion of a top inside edge and a bottom inside edge of the front cover of the photobook.
14. The system of claim 12, wherein the print order from the user to print the photobook is used to instruct a stamping machine to create the channel in the compressible cover material.
15. The system of claim 12, wherein the photo-strip displays a collage comprising a subset of the plurality of images.
16. The system of claim 12, wherein a back cover of the photobook contains a padding material.
17. The system of claim 12, wherein a spine of the photobook comprises the compressible cover material.
18. The system of claim 12, wherein the front cover, a back cover, and a spine of the photobook comprise a continuous piece of the compressible cover material.
19. The system of claim 12, wherein said sever, after performing an automatic image analysis on said plurality of images, is adapted to auto-populate a layout and a design of the photo-project for recommendation to the user.
20. A photobook, made by a process comprising the steps of:
printing a photo-strip having one or more images;
producing a front cover by compressing a compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo- strip;
wrapping the compressible cover material with the channel around a cover board material to make the front cover; and
setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
21. The photobook of claim 20, wherein the photo-strip set into the channel continues over a top edge and over a bottom edge of the front cover, and wherein the photo-strip has ends terminating on backsides of the front cover.
22. The photobook of claim 20, wherein the photo-strip is located on the front cover in a location selected from the group consisting of a center location, a right side location, a left side location, an upper location, and a lower location.
23. The photobook of claim 20, wherein the photo-strip displays a subset of the plurality of images comprising an image collage.
24. The photobook of claim 20, wherein the photo-strip displays a subset of the plurality of images comprising an image collage according to decisions of a user ordering the photobook.
25. The photobook of claim 20, further comprising:
a back cover; and
a spine between the front cover and the back cover,
wherein the front cover, the back cover, and the spine are composed of separate pieces of the cover board material.
26. The photobook of claim 25, further comprising:
a strip located along an inside part of the spine situated so that the inside part of the spine is flat.
27. The photobook of claim 20, wherein the front cover or a back cover of the photobook comprises a padding material.
28. The photobook of claim 20, wherein a connection between the front cover and a back cover comprises the compressible cover material.
29. The photobook of claim 20, further comprising:
one or more printed images as a decoration on a cover surface of the front cover or a back cover.
30. The photobook of claim 20, further comprising:
one or more decorations selected from the group consisting of a graphic design, a logo, a hologram, an emboss, a deboss, an emblem, an engraving, an imprint, a brand name, a trademark, a name, an event name, a wedding symbol, a holiday symbol, a special day symbol, a title, a number, a date, a decorative symbol, a foreign character, a word, a printed design, a painted decoration, a surface treatment, a ribbon, a metal ornament, a plastic ornament, and a leather strip.
31. The photobook of claim 20, further comprising:
one or more surface treatments selected from the group consisting of a dye, a stain, an opaque ink, a paint, a bleach, a tanning chemical, a wax, an oil, an acrylic spray, and a silicone.
32. The photobook of claim 20, further comprising:
end sheets on an inside of the front cover and an inside of a back cover.
33. The photobook of claim 20, further comprising:
one or more pages of photographic images between the front cover and a back cover.
34. A photobook, made by a process comprising the steps of:
printing a photo-strip having one or more images;
producing a front cover by wrapping a compressible cover material around a cover board material to make the front cover;
compressing the compressible cover material to form a channel in the compressible cover material, wherein the channel is of a size and a shape matching the photo-strip; and
setting the photo-strip permanently into the channel and flush with the front cover to create the photobook.
35. The photobook of claim 34, wherein the photo-strip set into the channel continues over a top edge and over a bottom edge of the front cover, and wherein the photo-strip has ends terminating on backsides of the front cover.
36. The photobook of claim 34, wherein the photo-strip is located on the front cover in a location selected from the group consisting of a center location, a right side location, a left side location, an upper location, and a lower location.
37. The photobook of claim 34, wherein the photo-strip displays a subset of the plurality of images comprising an image collage.
38. The photobook of claim 34, wherein the photo-strip displays a subset of the plurality of images comprising an image collage according to decisions of a user ordering the photobook.
39. The photobook of claim 34, further comprising:
a back cover;and
a spine between the front cover and the back cover,
wherein the front cover, the back cover, and the spine are composed of separate pieces of the cover board material.
40. The photobook of claim 39, further comprising:
a strip located along an inside part of the spine situated so that the inside part of the spine is flat.
41. The photobook of claim 34, wherein the front cover or a back cover of the photobook comprises a padding material.
42. The photobook of claim 34, wherein a connection between the front cover and a back cover comprises the compressible cover material.
43. The photobook of claim 34, further comprising:
one or more printed images as a decoration on a cover surface of the front cover or a back cover.
44. The photobook of claim 34, further comprising:
one or more decorations selected from the group consisting of a graphic design, a logo, a hologram, an emboss, a deboss, an emblem, an engraving, an imprint, a brand name, a trademark, a name, an event name, a wedding symbol, a holiday symbol, a special day symbol, a title, a number, a date, a decorative symbol, a foreign character, a word, a printed design, a painted decoration, a surface treatment, a ribbon, a metal ornament, a plastic ornament, and a leather strip.
45. The photobook of claim 34, further comprising:
one or more surface treatments selected from the group consisting of a dye, a stain, an opaque ink, a paint, a bleach, a tanning chemical, a wax, an oil, an acrylic spray, and a silicone.
46. The photobook of claim 34, further comprising:
end sheets on an inside of the front cover and an inside of a back cover.
47. The photobook of claim 34, further comprising:
one or more pages of photographic images between the front cover and a back cover.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US14/529,777 | 2014-10-31 | ||
| US14/529,777 US9077823B1 (en) | 2014-10-31 | 2014-10-31 | Systems and methods for automatically generating a photo-based project having a flush photo montage on the front cover |
| US201462079424P | 2014-11-13 | 2014-11-13 | |
| US62/079,424 | 2014-11-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2016069300A1 true WO2016069300A1 (en) | 2016-05-06 |
Family
ID=55858181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2015/056130 Ceased WO2016069300A1 (en) | 2014-10-31 | 2015-10-18 | Photobook and photoalbum having a flush photo montage on the front cover and methods for manufacre thereof |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2016069300A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115811582A (en) * | 2022-11-17 | 2023-03-17 | 中国电信股份有限公司 | Processing method and device based on video data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100047039A1 (en) * | 2006-12-01 | 2010-02-25 | Sean Kevin Anderson | Manufacturing system for personalized photo books |
| US20100164836A1 (en) * | 2008-03-11 | 2010-07-01 | Truview Digital, Inc. | Digital photo album, digital book, digital reader |
| US20140211065A1 (en) * | 2013-01-30 | 2014-07-31 | Samsung Electronics Co., Ltd. | Method and system for creating a context based camera collage |
-
2015
- 2015-10-18 WO PCT/US2015/056130 patent/WO2016069300A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100047039A1 (en) * | 2006-12-01 | 2010-02-25 | Sean Kevin Anderson | Manufacturing system for personalized photo books |
| US20100164836A1 (en) * | 2008-03-11 | 2010-07-01 | Truview Digital, Inc. | Digital photo album, digital book, digital reader |
| US20140211065A1 (en) * | 2013-01-30 | 2014-07-31 | Samsung Electronics Co., Ltd. | Method and system for creating a context based camera collage |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115811582A (en) * | 2022-11-17 | 2023-03-17 | 中国电信股份有限公司 | Processing method and device based on video data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9077823B1 (en) | Systems and methods for automatically generating a photo-based project having a flush photo montage on the front cover | |
| US8990672B1 (en) | Flexible design architecture for designing media-based projects in a network-based platform | |
| US8923551B1 (en) | Systems and methods for automatically creating a photo-based project based on photo analysis and image metadata | |
| US8958662B1 (en) | Methods and systems for automating insertion of content into media-based projects | |
| US9507506B2 (en) | Automatic target box in methods and systems for editing content-rich layouts in media-based projects | |
| US9329762B1 (en) | Methods and systems for reversing editing operations in media-rich projects | |
| US8935322B1 (en) | Methods and systems for improved uploading of media files for use in media-rich projects | |
| US8538986B2 (en) | System for coordinating user images in an artistic design | |
| US8422794B2 (en) | System for matching artistic attributes of secondary image and template to a primary image | |
| US8212834B2 (en) | Artistic digital template for image display | |
| US8274523B2 (en) | Processing digital templates for image display | |
| US8854395B2 (en) | Method for producing artistic image template designs | |
| US8849853B2 (en) | Method for matching artistic attributes of a template and secondary images to a primary image | |
| US8289340B2 (en) | Method of making an artistic digital template for image display | |
| US8345057B2 (en) | Context coordination for an artistic digital template for image display | |
| US20110029914A1 (en) | Apparatus for generating artistic image template designs | |
| US20110029860A1 (en) | Artistic digital template for image display | |
| US20070226653A1 (en) | Systems and methods for dynamically designing a product with digital content | |
| US20110029562A1 (en) | Coordinating user images in an artistic design | |
| JP2013225347A (en) | Automation of story sharing | |
| JP2010514056A (en) | Automated generation of multiple output products | |
| WO2011014233A1 (en) | Image capture device with artistic template design | |
| US8332427B2 (en) | Method of generating artistic template designs | |
| US20120179571A1 (en) | System and method for producing digital image photo-specialty products | |
| US20150077770A1 (en) | Method and system for custom designing nail wraps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15854923 Country of ref document: EP Kind code of ref document: A1 |
|
| DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 15854923 Country of ref document: EP Kind code of ref document: A1 |