WO2015087323A1

WO2015087323A1 - Emotion based 3d visual effects

Info

Publication number: WO2015087323A1
Application number: PCT/IL2014/051075
Authority: WO
Inventors: Ori BUBERMAN; Amihai LOVEN
Original assignee: MANTISVISION Ltd
Current assignee: MANTISVISION Ltd
Priority date: 2013-12-09
Filing date: 2014-12-09
Publication date: 2015-06-18
Anticipated expiration: 2016-06-09

Abstract

A technique for selecting a 3D visual effect to be applied on a 3D content is described. The selection is based on the emotional response the 3D content evoke in a viewer, or on a predicted emotional response to the 3D content. Different aspects of the technique may use an analysis of the 3D content, physical measurements of the viewer emotional response, and may learn to predict emotional response from previous events.

Description

EMOTION BASED 3D VISUAL EFFECTS

TECHNOLOGICAL FIELD

[001] Examples of the presently disclosed subject matter relate to the field of 3D media content processing and 3D visual effects.

BACKGROUND

[002] Media content can evoke an emotional effect in a person that is exposed to it. A person that experiences an emotional effect when exposed to media content can provide an emotion indicator which describes the emotional experience. For example, in some social networks, a user can upload media content with an emotion tag, also referred to sometimes as a mood tag, which in some cases relates to the emotional effect of the media content over the user. It is also possible for users to comment on some other user's uploaded media with an emotional tag which describes the emotional effect that is experienced by the commenting user when exposed to the media content. In this regard, an emotional affect can refer to the emotional reaction of any person exposed to the media content, including but not limited to, the person who created or uploaded the media content. In particular, for 3D media content, any person exposed to the 3D media content can experience an emotional affect. In the following description the terms emotional affect and emotional response are used interchangeably with the same meaning.

[003] Emotional affect can also be deduced in other ways. For example, based on

biofeedback, statistical data, or a combination of the two, an emotional affect in a person that is exposed to media content can be estimated, deduced or determined. Below there is provided a list of publications which describe various methods that can be used to determine an emotional affect on a person that is exposed to media content. It would be appreciated that the same methods, perhaps with some inherent modifications, can be used to determine an emotional affect in a person that is exposed to 3D media content.

[004] Soleymani, Mohammad. "Implicit and automated emotional tagging of videos." (2011). [005] Sander, David, Didier Grandjean, and Klaus R. Scherer. "A systems approach to appraisal mechanisms in emotion." Neural networks 18.4 (2005): 317-352.

[006] Soleymani, Mohammad, et al. "Continuous emotion detection in response to music videos." Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE

International Conference on. IEEE, 2011.

[007] Koelstra, Sander, et al. "Deap: A database for emotion analysis; using physiological signals." Affective Computing, IEEE Transactions on 3.1 (2012): 18-31.

[008] Vinciarelli, Alessandro, Nicolae Suditu, and Maja Pantic. "Implicit human-centered tagging." Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on. IEEE, 2009.

[009] Hanjalic, Alan, and Li-Qun Xu. "Affective video content representation and

modeling." Multimedia, IEEE Transactions on 7.1 (2005): 143-154.

[010] Soleymani, Mohammad, et al. "Affective characterization of movie scenes based on content analysis and physiological changes." International Journal of Semantic Computing 3.02 (2009): 235-254.

[011] Picard, Rosalind W., and Shaundra Bryant Daily. "Evaluating affective interactions:

Alternatives to asking what users feel." CHI Workshop on Evaluating Affective Interfaces:

Innovative Approaches. 2005.

[012] Scherer, Klaus R. "What are emotions? And how can they be measured?. "Social science information 44.4 (2005): 695-729.

[013] Soleymani, Mohammad, et al. "Affective ranking of movie scenes using physiological signals and content analysis." Proceedings of the 2nd ACM Workshop on Multimedia Semantics. ACM, 2008.

[014] Soleymani, Mohammad, Jeremy Davis, and Thierry Pun. "A collaborative personalized affective video retrieval system." 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. SUMMARY

[015] The present disclosure relates to a method of selecting a 3D visual effect to be applied on a 3D content. In an example of the presently disclosed subject matter, the 3D visual effect is selected based on the emotional response the 3D content evoked in a viewer, for example as determined by a physical measurements of the viewer response, or on the emotional response the 3D content is predicted to evoke in a viewer, for example, based on the substance of the 3D content, the identity of the viewer, and/or past experience.

BRIEF DESCRIPTION OF THE DRAWINGS

[016] In order to understand the invention and to see how it may be carried out in practice, a preferred embodiment will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:

[017] Figure 1 is a simplified block diagram of an example for one possible implementation of a mobile communication device with 3D capturing capabilities;

[018] Figure 2 is a simplified block diagram of an example for one possible implementation of a system that includes a mobile communication device with 3D capturing capabilities and a cloud platform;

[019] Figure 3 is a flow chart illustration of two possible implementations of a method of processing 3D content by a computing device, according to examples of the presently disclosed subject matter;

[020] Figure 4 is an illustration of a scenario in which a happy emotional response translates to an appropriate 3D visual effect that is applied to input 3D content, according to examples of the presently disclosed subject matter; and

[021] Figure 5 is an illustration of a scenario in which a sad emotional response translates appropriate 3D visual effect that is applied to input 3D content, according to examples of the presently disclosed subject matter. GENERAL DESCRIPTION

[022] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[023] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as "processing", "calculating", "computing", "determining", "generating", "applying", "obtaining", "accessing", "selecting", "fitting", "estimating", "analyzing", or the like, include action and/or processes of a computer that manipulate and/or transform data into other data, said data represented as physical quantities, e.g. such as electronic quantities, and/or said data representing the physical objects. The terms "computer", "processor", "controller", "processing unit", and "computing unit" should be expansively construed to cover any kind of electronic device, component or unit with data processing capabilities, including, by way of non-limiting example, a personal computer, a tablet, a smartphone, a server, a computing system, a communication device, a processor (for example, digital signal processor (DSP), and possibly with embedded memory), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), and so on) , a core within a processor, any other electronic computing device, and or any combination thereof.

[024] The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes or by a general purpose computer specially configured for the desired purpose by a computer program stored in a computer readable storage medium, e.g., in a non-transitory computer readable storage medium such a memory unit.

[025] As used herein, the phrase "for example," "such as", "for instance" and variants thereof describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases" or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment(s) is included in at least one embodiment of the presently disclosed subject matter. Thus the appearance of the phrase "one case", "some cases", "other cases" or variants thereof does not necessarily refer to the same embodiment(s).

[026] It is appreciated that certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.

[027] In embodiments of the presently disclosed subject matter one or more stages illustrated in the figures may be executed in a different order and/or one or more groups of stages may be executed simultaneously and vice versa. The figures illustrate a general schematic of the system architecture in accordance with an embodiment of the presently disclosed subject matter. A module, a unit a component or the like in the figures can be made up of any combination of software, hardware and/or firmware that performs the functions as defined and explained herein. The modules, units, components or the like in the figures may be centralized in one location or dispersed over more than one location.

[028] Provided below is a list of terms and definitions. As stated below, some of the terms define below have a conventional meaning in the field of image processing or in the field of video processing or in the field of 3D processing. For each of the terms below a short definition is provided. For terms which have a conventional meaning in the art, the following definitions are provided as a non-limiting example only for convenience purposes. Accordingly, the interpretation of the corresponding terms in the claims, unless stated otherwise, is not limited to the definitions below, and the terms used in the claims should be given their broadest reasonable interpretation.

[029] The term "egomotion" is recognized by those skilled in the art. In the following description and in the claims, the term egomotion is used to describe a motion of a 3D camera. In particular, egomotion is used to refer to the motion of the 3D camera with respect to a 3D scene, and/or to the estimation of such motion.

[030] Throughout the description and the claims, reference is made to the term "salient 3D object". The term salient 3D object as used herein relates to a 3D object included in 3D content, and which is determined to have a greatest emotional impact over a viewer (the viewer can either be actual or synthetic) that is exposed to (e.g., presented with) the input 3D content. In addition to adaption of known methods for determining a salient 2D object, which can be adapted to the 3D content context, the following examples can be used to determine a salient 3D object from input 3D content:

[031] a. Applying 3D segmentation to the input 3D content or to some portion thereof, possibly with subsequent identification of the 3D object which is closest to center of frame (or any other predefined point within the frame) and/or identification of the 3D object that most persistently appears in a visual component of the 3D content.

[032] b. Applying to a visual component of the input 3D content a 3D facial recognition process or any other 3D processing algorithms capable of identifying human bodies or any part of the human body and/or applying to a visual component of the input 3D content a facial expression recognition and analysis algorithm.

[033] c. Applying any one or a combination of the algorithms mentioned in example 'b' followed by selection of human subjects facing the camera for a longest span in the input 3D content, or in a representation of at least a visual component of the input 3D content.

[034] d. Determining which 3D object in the input 3D content is associated (e.g., has the best correlation score) with the emotional response. The association between a 3D object from the input 3D content and the emotional response can be determined based at least in part on a temporal relation between the appearance of an object in a representation of at least a visual component of the input 3D content or based on a relation between the location of a 3D object in a 3D scene that is generated based at least on a visual component of the input 3D content and an emotional response. In another example, the gaze direction of a viewer can be obtained. An orientation of reference for a certain instant during the representation of at least a visual component of the input 3D content can be determined based on the gaze direction of the viewer. The salient 3D object can be determined according to a relation among the emotional response, the orientation of reference at a certain instant, and a visual component of the input 3D content at the relevant instant.

[035] e. In another example, in case the orientation of reference corresponds to a location of a certain 3D object in the visual component of the 3D input for a majority of the duration (or is the greatest relative to other 3D object in the 3D scene, etc.) of the

representation of the 3D content, it is assumed to be the salient 3D object.

[036] f. In yet another example, a salient 3D object, can be determined based on a correlation between 3D objects identified in the input 3D content and a pre-stored 3D model (or a plurality of such pre-stored 3D models) of 3D objects. For example, in case the visual component of t input 3D content is a 3D scene in which many children appear, and only one of the children corresponds to a 3D model that is related to the viewer (for example, the viewer's son), the 3D object that corresponds the pre-stored 3D model of the viewer's son can be determined to be the salient 3D object.

[037] g. A combination of any one or more of examples a-f above.

[038] By way of further example, a salient 3D object can be determined to be comprised of two or more distinguishable 3D objects, for example, when the greatest emotional contribution to the emotional response is determined to be attributed to the combination of the two or more distinguishable 3D objects. 3D objects can be deemed "distinguishable" when a computer implemented 3D content processing algorithm identifies certain visual content as relating to two (or more) distinct 3D objects, and when such 3D objects are determined to have a greatest emotional contribution to the emotional effect, a salient 3D object is comprised of the two (or more) 3D objects.

[039] Throughout the description and the claims, reference is made to the term "emotional affect". The term emotional affect is known in the art of psychology, and also in the art of marking, analyzing, determining, etc. and emotional experience related to media content. The following definition is provided as a non-limiting example for convenience purposes only. Accordingly, the interpretation of the term emotional affect in the claims, unless explicitly stated or implicitly apparent otherwise, is not limited to the definitions below, and the term should be given its broadest reasonable interpretation.

[040] The term emotional affect as used herein relates to the experience of feeling or emotion. Furthermore, in some examples of the presently disclosed subject matter, the term emotional affect can mean the instinctual reaction to 3D content or at least to a visual component of 3D content. In other examples, the term emotional affect can mean the reaction which follows extensive perceptual and cognitive encoding. In yet further examples of the presently disclosed subject matter, the term emotional affect can relate to a computer generated conclusion that is inferred from a bio-emotional response(s), using known algorithms that are capable of determining an emotional affect from biofeedback data, of a user that is exposed to3D content or at least to a visual component of 3D content. The term emotional response is sometimes used herein with the same meaning as emotional affect.

[041] According to some examples of the presently disclosed subject matter, the emotional affect can be associated the entire input 3D content, and in other examples, the emotional affect can relate to some portion of the input 3D content. Still further by way of example, the emotional affect can relate to only a certain time-frame or within an input 3D content stream, for example, the emotional affect can relate to only part of the frames in an input 3D content stream that is comprised of a sequence of frames. Yet further by way of example, the emotional affect can relate to only part of a 3D scene that is represented by the input 3D content or some portion thereof. For example, the emotional affect can relate to a salient 3D object, which is a 3D object appearing in the input 3D content.

[042] There are various methods by which an emotion indicator can be added to media content, and in particular to 3D visual content, e.g., 3D content stream. In one example, the creator of the 3D visual content can simply add a metadata tag to the 3D visual content.

According to some examples of the presently disclosed subject matter, the metadata tag can indicate the emotion evoked in the creator when viewing the 3D content, or possible emotions which the creator anticipates that may be evoked in viewers. [043] In another example, the emotion metadata tag can be added to the 3D content by a person or a machine (e.g., computer, smartphone, etc.) other than the creator of the 3D content. Examples of other user users can include recipients of the 3D content, social network viewers of the 3D content, receipts of the 3D content with whom the creator chose to share it, etc. Still further by way of example, the emotion tag can be synthetically generated by a digital device, such as a computer or a smartphone, based on statistical models and application of content analysis algorithms known in the art, including but not limited to in the publications mentioned in the background section of this application.

[044] Furthermore, the emotion indicator need not necessarily be provided in the form of a metadata tag, and any other suitable form of digital data can be used to provide an emotion indicator, whether embedded in the 3D content, attached to the 3D content or otherwise associated with it.

[045] By way of example, an application or a component of an application can be provided, which can be adapted to identify and extract the emotion indicator from the input 3D content, or from any other input that is related to the input 3D content, and which is intended to provide an emotion indicator for a certain input 3D content (in case the emotion indicator is not included as part of the input 3D content).

[046] Still further by way of example, the computer program code that is capable of identifying and extracting the emotion indicator in connection with a certain input 3D content, can be implement as part of a 3D content decoder or as part of a 3D content viewer. Such 3D content decoder/viewer can be used to represent 3D content based on the input 3D content and can be adapted to identify and extract the emotion indicator from the input data.

[047] In yet a further example, a 3D content format or standard can be provided which supports inclusion of emotion indicators with the 3D content, and a data structure of the 3D content stream can include a field for emotion indicators. Such a field can be used in the 3D content processing method disclosed herein.

[048] Throughout the description and the claims, reference is made to the term "3D visual effect". The term 3D visual effect is known in the art of 3D content rendering. The following definition is provided as a non-limiting example for convenience purposes only. Accordingly, the interpretation of the term 3D visual effect in the claims, unless explicitly stated or implicitly apparent otherwise, is not limited to the definitions below, and the term should be given its broadest reasonable interpretation. The term 3D visual effect as used herein relates to various configurable process which can be applied to an input 3D content and which can transform the visual appearance of the 3D content in a predefined, yet configurable, manner. The

transformation can manipulate an existing 3D object that is included in the 3D content. In some examples of the presently disclosed subject matter, the 3D visual effect can integrate computer generated graphics or a second (and third, fourth, n-th) 3D frame or content layer or content stream with the input 3D content.

[049] According to some examples of the presently disclosed subject matter, at least one frame which corresponds to the input 3D content can include a plurality of different areas which are represented by a corresponding plurality of 3D particles in a 3D space. In such an example, and still by way of example, the 3D visual effect that is selected based at least on the emotion indicator can be effective for adapting a visual appearance of at least one 3D particle from the plurality of 3D particles. Still further by way of example, in case a salient 3D object is identified in the input 3D content, a visual appearance of the salient 3D object can be modified by adapting a visual appearance of at least one 3D particle from said plurality of 3D that represents a portion of the salient 3D object.

[050] Still further by way of example, the 3D visual effect is based on a law of physics. Further by way of example, a parameter value for a parameter that is used by and which influences the visual appearance of the 3D content when the visual effect is applied thereto is selected according to the emotion indicator and is applied to the law of physics to provide the 3D visual effect. A physics engine can be used to render the 3D visual effect according to the selected or computed parameter value.

[051] It should be noted that some examples of the presently disclosed subject matter are not limited in application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention can be capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.

[052] In this document, an element of a drawing that is not described within the scope of the drawing and is labeled with a numeral that has been described in a previous drawing has the same use and description as in the previous drawings. Similarly, an element that is identified in the text by a numeral that does not appear in the drawing described by the text, has the same use and description as in the previous drawings where it was described.

[053] The drawings in this document may not be to any scale. Different figures may use different scales and different scales can be used even within the same drawing, for example different scales for different views of the same object or different scales for the two adjacent objects.

[054] Figure 1 is a simplified block diagram of an example of one possible implementation of a mobile communication device with 3D capturing and processing capabilities. The mobile communication device 100 can includes a 3D camera 10 that is capable of providing 3D depth or range data. In the example of FIG. 1 there is shown a configuration of an active stereo 3D camera, but in further examples of the presently disclosed subject matter other known 3D cameras can be used. Those versed in the art can readily apply the teachings provided in the examples of the presently disclosed subject matter to other 3D camera configurations and to other 3D capture technologies, including for example, time of flight based depth information capture systems, passive stereo 3D capture, as well as other implementations of active triangulation methods, such as diffractive optical elements based 3D capture systems, etc.

[055] By way of example, the 3D camera 10 can include: a 3D capture sensor 12, a driver 14, a 3D capture processor 16 and a flash module 18. In this example, the flash module 18 is configured to project a structured light pattern and the 3D capture sensor 12 is configured to capture an image which corresponds to the reflected pattern, as reflected from the

environment onto which the pattern was projected. US Patent No. 8,090,194 to Gordon et. al. and US Patent No. 8,538,166 to Gordon et. al. describe examples of structured light patterns that can be used in a flash component of a 3D camera, as well as other aspects of active stereo 3D capture technology and is hereby incorporated into the present application in its entirety. International Application Publication No. WO2013/144952 describes an example of a possible flash design and is hereby incorporated by reference in its entirety.

[056] By way of example, the flash module 18 can include an IR light source, such that it is capable of projecting IR radiation or light, and the 3D capture sensor 12 can be and IR sensor, that is sensitive to radiation in the IR band, and such that it is capable of capturing the IR radiation that is returned from the scene. The flash module 18 and the 3D capture sensor 12 are calibrated. According to examples of the presently disclosed subject matter, the driver 14, the 3D capture processor 16 or any other suitable component of the mobile communication device 100 can be configured to implement auto-calibration for maintaining the calibration among the flash module 18 and the 3D capture sensor 12.

[057] The 3D capture processor 16 can be configured to perform various processing functions, and to run computer program code which is related to the operation of one or more components of the 3D camera. The 3D capture processor 16 can include memory 17 which is capable of storing the computer program instructions that are executed or which are to be executed by the processor 16.

[058] The driver 14 can be configured to implement a computer program which operates or controls certain functions, features or operations that the components of the 3D camera 10 are capable of carrying out.

[059] According to examples of the presently disclosed subject matter, the mobile

communication device 100 can also include hardware components in addition to the 3D camera 10, including for example, a power source 20, storage 30, a communication module 40, a device processor 50 and memory 60, one or more biometric sensors 90, device imaging hardware 110, a display unit 120, and other user interfaces 130. It should be noted that in some examples of the presently disclosed subject matter, one or more components of the mobile communication device 100 can be implemented as distributed components. In such examples, a certain component can include two or more units distributed across two or more interconnected nodes. Further by way of example, a computer program, possibly executed by the device processor 50, can be capable of controlling the distributed component and can be capable of operating the resources on each of the two or more interconnected nodes.

[060] It is known to use various types of power sources in a mobile communication device. The power source 20 can include one or more power source units, such as a battery, a short- term high current source (such as a capacitor), a trickle-charger, etc.

[061] The device processor 50 can include one or more processing modules which are capable of processing software programs. The processing module can each have one or more processors. In this description, the device processor 50 may include different types of processors which are implemented in the mobile communication device 100, such as a main processor, an application processor, etc. The device processor 50 or any of the processors which are generally referred to herein as being included in the device processor can have one or more cores, internal memory or a cache unit.

[062] The storage unit 30 can be configured to store computer program code that is necessary for carrying out the operations or functions of the mobile communication device 100 and any of its components. The storage unit 30 can also be configured to store one or more applications, including 3D applications 80, which can be executed on the mobile communication device 100. In a distributed configuration one or more 3D applications 80 can be stored on a remote computerized device, and can be consumed by the mobile communication device 100 as a service. In addition or as an alternative to application program code, the storage unit 30 can be configured to store data, including for example 3D data that is provided by the 3D camera 10.

[063] The communication module 40 can be configured to enable data communication to and from the mobile communication device. By way of example, examples of communication protocols which can be supported by the communication module 40 include, but are not limited to cellular communication (3G, 4G, etc.), wired communication protocols (such as Local Area Networking (LAN)), and wireless communication protocols, such as Wi-Fi, wireless personal area networking (PAN) such as Bluetooth, etc.

[064] The biometric sensors (90) can be configured to perform biometric measurements of the user. By way of example, biometric measurements may include physiological and/or behavioral characteristics of the users. For example, measurements related to the shape and/or other physical properties of the user body or of parts of the user body, including for example, fingerprints, palm-prints, mapping of the visible veins on the user's body or parts of the user's body, face images, facial measurements, identification of facial expressions, DNA, geometry of the user's body or parts of the user's body, iris images, iris measurements, retina images, retina measurements, body odor, user's pulse, user's blood pressure, user's voice, user's movements, properties of user's skin, conductivity of user's skin, resistance of user's skin, pH level of user's skin, moisture of user's skin, user's perspiration, and so forth.

[065] It should be noted that that according to some examples of the presently disclosed subject matter, some of the components of the 3D camera 10 can be implemented on the mobile communication hardware resources. For example, instead of having a dedicated 3D capture processor 16, the device processor 50 can be used. Still further by way of example, the mobile communication device 100 can include more than one processor and more than one type of processor, e.g., one or more digital signal processors (DSP), one or more graphical processing units (GPU), etc., and the 3D camera can be configured to use a specific one (or a specific set or type) processor(s) from the plurality of device 100 processors.

[066] The mobile communication device 100 can be configured to run an operating system 70. Examples of mobile device operating systems include but are not limited to: such as Windows MobileTM by Microsoft Corporation of Redmond, WA, and the Android operating system developed by Google Inc. of Mountain View, CA.

[067] The 3D application 80 can be any application which uses 3D data. Examples of 3D applications include a virtual tape measure, 3D video, 3D snapshot, 3D modeling, etc. It would be appreciated that different 3D applications can have different requirements and features. A 3D application 80 may be assigned to or can be associated with a 3D application group. In some examples, the device 100 can be capable of running a plurality of 3D applications 80 in parallel.

[068] Imaging hardware 110 can include any imaging sensor, in a particular example, an imaging sensor that is capable of capturing visible light images can be used. According to examples of the presently disclosed subject matter, the imaging hardware 110 can include a sensor, typically a sensor that is sensitive at least to visible light, and possibly also a light source (such as one or more LEDs) for enabling image capture in low visible light conditions. According to examples of the presently disclosed subject matter, the device imaging hardware 110 or some components thereof can be calibrated to the 3D camera 10, and in particular to the 3D capture sensor 12 and to the flash 18. It would be appreciated that such a calibration can enable texturing of the 3D image and various other co-processing operations as will be known to those versed in the art.

[069] In yet another example, the imaging hardware 110 can include a RGB-IR sensor that is used for capturing visible light images and for capturing IR images. Still further by way of example, the RGB-IR sensor can serve as the 3D capture sensor 12 and as the visible light camera. In this configuration, the driver 14 and the flash 18 of the 3D camera, and possibly other components of the device 100, are configured to operate in cooperation with the imaging hardware 110, and in the example given above, with the RGB-IR sensor, to provide the 3D depth or range data.

[070] The display unit 120 can be configured to provide images and graphical data, including a visual rendering of 3D data that was captured by the 3D camera 10, possibly after being processed using the 3D application 80. The user interfaces 130 can include various components which enable the user to interact with the mobile communication device 100, such as speakers, buttons, microphones, etc. The display unit 120 can be a touch sensitive display which also serves as a user interface.

[071] According to some examples of the presently disclosed subject matter, any processing unit, including the 3D capture processor 16 or the device processor 50 and/or any subcomponents or CPU cores, etc. of the 3D capture processor 16 and/or the device processor 50, can be configured to read 3D images and/or frames of 3D video clips stored in storage unit 30, and/or to receive 3D images and/or frames of 3D video clips from an external source, for example through communication module 40; produce 3D models out of said 3D images and/or frames. By way of example, the produced 3D models can be stored in storage unit 30, and/or sent to an external destination through communication module 40. According to further examples of the presently disclosed subject matter, any such processing unit can be configured to process 3D content.

[072] Figure 2 is a simplified block diagram of an example for one possible implementation of a system 200, that includes a mobile communication device with 3D capturing capabilities 100, and a could platform 210 which includes resources that allows the processing of 3D content.

[073] According to examples of the presently disclosed subject matter, the cloud platform 210 can include hardware components, including for example, one or more power sources 220, one or more storage units 230, one or more communication modules 240, one or more processors 250, optionally one or more memory units 260, and so forth.

[074] The storage unit 230 can be configured to store computer program code that is necessary for carrying out the operations or functions of the cloud platform 210 and any of its components. The storage unit 230 can also be configured to store one or more applications, including 3D applications, which can be executed on the cloud platform 210. In addition or as an alternative to application program code, the storage unit 230 can be configured to store data, including for example 3D data.

[075] The communication module 240 can be configured to enable data communication to and from the cloud platform. By way of example, examples of communication protocols which can be supported by the communication module 240 include, but are not limited to cellular communication (3G, 4G, etc.), wired communication protocols (such as Local Area Networking (LAN)), and wireless communication protocols, such as Wi-Fi, wireless personal area networking (PAN) such as Bluetooth, etc.

[076] The one or more processors 250 can include one or more processing modules which are capable of processing software programs. The processing module can each have one or more processing units. In this description, the device processor 250 may include different types of processors which are implemented in the cloud platform 210, such as general purpose processing units, graphic processing units, physics processing units, etc. The device processor 250 or any of the processors which are generally referred to herein can have one or more cores, internal memory or a cache unit. [077] According to examples of the presently disclosed subject matter, the one or more memory units 260 may include several memory units. Each unit may be accessible by all of the one or more processors 250, or only by a subset of the one or more processors 250.

[078] According to some examples of the presently disclosed subject matter, any processing unit, including the one or more processors 250 and/or any sub-components or CPU cores, etc. of the one or more processors 250, can be configured to read 3D images and/or frames of 3D video clips stored in storage unit 230, and/or to receive 3D images and/or frames of 3D video clips from an external source, for example through communication module 240, where, by a way of example, the communication module may be communicating with the mobile communication device 100, with another cloud platform, and so forth. By a way of example, the processing unit can be further configured to produce 3D models out of said 3D images and/or frames. Further by a way of example, the produced 3D models can be stored in storage unit 230, and/or sent to an external destination through communication module 240. According to further examples of the presently disclosed subject matter, any such processing unit can be configured to process 3D content.

[079] According to further examples of the presently disclosed subject matter, processing of 3D content can be executed in a distributed fashion by the mobile communication device 100 and the could platform 210 together, for example where some computational tasks are executed on the mobile communication device 100, while other computational tasks are executed on the could platform 210. Further by a way of example, communication for controlling and/or synchronizing and/or transferring information among the computational tasks can be performed through communication between the communication module 40 and the one or more communication modules 240.

[080] Figure 3 is a flow chart illustration of two possible implementations of a method of processing 3D content by a computing device, according to examples of the presently disclosed subject matter.

[081] Flow chart 301 includes four blocks: input 3D content is obtained (311) , emotion indicators are obtained (312), a visual effect is selected (313), and the selected visual effect is applied (314) on the input 3D content. Note that in other example implementations, 312 may precede 311.

[082] Flow chart 351 includes six blocks. According to examples of the presently disclosed subject matter, input 3D content is obtained (361), and a first part of the input 3D content is presented to a user (362). Emotion indicators are obtained (363) after the first part of the input 3D content was presented to the viewer (362). Further by a way of example, the obtained emotion indicators can then be used to select a 3D visual effect (364). The selected visual effect is applied (365) on a second part of the input 3D content, and the transformed second part of the input 3D content is presented to the user (366). Further by a way of example, the process can be repeated again and again using more and more parts of the input 3D content, therefore changing 3D visual effects several times during a continuous presentation of the input 3D content. Further by a way of example, the first part can be as short as is necessary to gain an initial emotion indication. Note that while a short first part may produce a faster adjustment of the 3D visual effects, it may possibly hamper the quality of the obtained emotion indicators due to lack of information, and therefore end up with less suitable 3D visual effects.

[083] According to examples of the presently disclosed subject matter, an assurance level with respect to an emotional affect can be computed and an indication in respect to the assurance level can be generated to provide an indication with regard to the reliability of a certain emotion indication. According to one example, a 3D visual effect which is associated with a certain emotional affect is applied to the 3D content only when the assurance level with regard to the emotional affect crosses a certain threshold. Further by way of example, the tracking of the emotional response can continue during the 3D content playback and the application of the 3D visual affect may be resumed so long as the assurance level with regard to the respective emotional affect is above a certain threshold (which can be the same threshold that was used to initiate the 3D visual effect, or it can be a different threshold). Various algorithms can be used to avoid transients in the emotional affect readings effecting the application of the 3D visual effect, including for example, smoothing a current sample using previous samples collected over a certain period of time, and so forth. The processing of the samples relating to the emotional response of a viewer of the 3D content and the operations required for the application of a suitable 3D visual effect can be carried out in real-time during playback or display of the 3D content. It would be appreciated that the term "real-time" also refers to the case where some leg exists between the emotional response of the viewer and the application of the 3D visual effect as a result of processing time, communication legs and similar delays caused by computer hardware and/or software which are involved in the capturing and processing of the emotional response, the selection of the 3D visual effect and its application.

[084] According to examples of the presently disclosed subject matter, the input 3D content can include visual 3D content, or can be presented visually as a visual 3D content. By a way of example, the visual 3D content may include one or more frames, for example, as a 3D video that includes one or more 3D objects where some of the 3D objects are in motion. Further by a way of example, emotion indicators can be embedded in the input 3D content or can be provided as a separate data stream that is operatively associated (for example, using some references) with the input 3D content. The input 3D content can also include or be associated with various other types of content including, for example, audio (stereo, surround or any other suitable format), text, 2D images, 2D video etc.

[085] According to examples of the presently disclosed subject matter, the input 3D content can be obtained (311 and 361) by: reading the information of the input 3D content from a storage device (for example using storage 30), receiving the information of the input 3D content from a remote source over a communication network (for example using

communication module 40, and further by a way of example the remote source can be a cloud platform 210), capturing the input 3D content (for example using a 3D capturing sensor 12), and so forth.

[086] According to examples of the presently disclosed subject matter, the input 3D content can be obtained (311 and 361) by: reading the information of the input 3D content from a storage device (for example using storage 30), receiving the information of the input 3D content from a remote source over a communication network (for example using

communication module 40, and further by a way of example the remote source can be a cloud platform 210), capturing the input 3D content (for example using a 3D capturing sensor 12), and so forth. [087] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312). According to some examples, the emotional indicators can be obtained by: reading the information regarding the emotional indicators from a storage device (for example using storage 30), receiving the information of the emotional indicators from a remote source over a communication network (for example, using communication module 40, and further by a way of example the remote source can be a cloud platform 210), and so forth. It would be appreciated that the indicators can also be sampled data that is received directly from a sensor.

[088] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312 and 363) by measuring the emotional response to the input 3D content or parts of the input 3D content by one or more viewers. Further by a way of example, measuring the emotional affect of a viewer may include capturing the viewer's voice, capturing images and/or videos of the viewer and/or viewer's face, and/or biofeedback indications (some of which are detailed below), and so forth. Further by a way of example, measuring the emotional affect of a viewer may be based on measuring properties associated with the emotional reaction of the viewer, for example, the viewer's pulse, the viewer's blood pressure, properties of the viewer's skin, the connectivity of the viewer's skin, the resistance of the viewer's skin, the pH level of the viewer's skin, the moisture of the viewer's skin, the viewer's perspiration, properties of the viewer's voice, the viewer's pupillary response, the viewer's brain wave activity and/or other properties extracted from images and/or videos of the viewer and/or the viewer's face, including identification of facial expressions, other biometric properties, and so forth. According to further examples of the presently disclosed subject matter, such measurements can use the biometric sensors (90), the 3D capture sensor (12), and so on.

[089] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312 and 363) by determining the emotional indicators based on the substance of the input 3D content. By a way of example, such determination can be performed using a set of 3D content exemplars, where each 3D content exemplar is associated with one or more associated emotion indicators. A subset of the 3D content exemplars is selected, for example based on measures of similarity that is calculated between the input 3D content and the 3D content exemplars, for example by selecting the most similar 3D content exemplars. Further by a way of example, the emotional indicators are determined to be a combination of the emotion indicators associated with the selected subset of 3D content exemplars.

[090] As an example of a 3D content exemplar, a 3D parametric model of a human face can be used with various expressions which are applied to the 3D parametric model. The model of the face, with the various expression applied thereto, can be used to provide a series of 3D content exemplars. An example of a 3D parametric model and some expressions applied to the 3D parametric model are described in US Patent No. 6,556,196 to Blanz et. al. Another example of a 3D content exemplar includes elements of a dataset of 3D video clips and/or of a set of 3D models and/or of a set of 2D images and/or of a set of 2D video clips and each one or each group of the above dataset elements can be used as a 3D content exemplar.

[091] In yet another example, the 3D content exemplar can be personalized to a given user or to a certain groups of users. For example, the 3D content exemplar, can a 3D model and some expressions of the specific person whose emotional affect is to be used for adapting the 3D content with the 3D visual effect. In another example, the exemplar can be "localized" the emotional affect is to be used for generating the 3D visual effect, the dataset that is to be used for determining the subject's emotional affect is selected based on a geolocation of the subject. Other factors which can be used for selecting the 3D content exemplars include, but are not limited to: the subject gender, the subject's age, the subject's viewing history, the subject's relationship with a salient object appearing in the 3D content, lighting conditions in a scene where the 3D content is captured, the perspective or egomotion of the camera that was used to capture the 3D content, and so forth.

[092] According to examples of the presently disclosed subject matter, a similarity measure can be used for evaluating an affinity between information (e.g., a sample) related to an emotional response by a viewer. For example, in case the information related to the emotional response by the viewer is the geographical location of the viewer, a similarity measure associated with a physical geographical distance between the current viewer and viewers associated with exemplars used for generating or selecting the 3D visual effect can be used to narrow down and make location specific the emotional response determination. Further by a way of example, the similarity measure can be used to rank exemplars, for example by ranking higher exemplars with higher similarity measure, and to use the ranking to control the combination, for example by giving priority in the combination of the emotion indicators and/or 3D visual effects to exemplars with higher rank. Further by a way of example, the similarity measure can be used to assign weights to exemplars, for example by assign a higher weight to exemplars with higher similarity measure, and to use the weights to control the combination, for example by sampling the emotion indicators and/or 3D visual effects with probability determined by the weights. According to further examples of the presently disclosed subject matter, the similarity measure can be used to determined emotion indicators and/or 3D visual effects and/or parameters of the emotion indicators and/or parameters of 3D visual effects by applying a regression algorithm that takes similarity measure as an input. Example of such algorithms includes support vector regression, kernel regression, non-parametric regression algorithms, regression tree algorithms, and so forth.

[093] Similarity measure between a viewer and exemplars of viewers can be used to select emotion indicators, similarity measure between 3D content and exemplars of 3D content can be used to select emotion indicators, similarity measure between a pair of viewers and 3D content and between exemplars of pairs of viewers and 3D content can be used to select emotion indicators, and similarity measure between emotional indicator and exemplars of emotional indicators can be used to select 3D visual effects, and so forth. Further by a way of example, the similarity measure can be used to select a subset of exemplars, for example using a k-nearest-neighbor algorithm, or for example to by using a directional similarity measure and to exclude exemplars that has a similar similarity measure, and the emotion indicators and/or 3D visual effects can be set to a combination of the emotion indicators and/or 3D visual effects associated with the selected exemplars.

[094] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312 and 363) by predicting the emotional affect on a viewer or on an anticipated viewer that is expected to view the input 3D content. By a way of example, such prediction can be performed using a set of viewer exemplars, where each viewer exemplar is associated with one or more associated emotion indicators. A subset of the viewer exemplars is selected, for example based on measures of similarity that is calculated between the viewer or anticipated viewer and the viewer exemplars, for example by selecting the most similar viewer exemplars. Further by a way of example, the emotional indicators are determined to be a combination of the emotion indicators associated with the selected subset of viewer exemplars.

[095] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312 and 363) by predicting the emotional effect on a viewer or on an anticipated viewer that is expected to view the input 3D content based on the identity of the viewer together with the substance of the input 3D content. By a way of example, such prediction can be performed using a set of viewer exemplars paired with 3D content exemplars, where each pair of exemplars is associated with one or more associated emotion indicators. A subset of the pairs of exemplars is selected, for example based on measures of similarity that is calculated between the viewer or anticipated viewer together with the input 3D content and the pairs of exemplars, for example by selecting the most similar pairs of exemplars. Further by a way of example, the emotional indicators are determined to be a combination of the emotion indicators associated with the selected subset of pairs of exemplars.

[096] According to some examples of the presently disclosed subject matter, the method can further include processing the input 3D content to identify a salient 3D object, wherein a salient 3D object is a 3D object in the input 3D content that is determined to have a greatest emotional contribution to the emotional effect. Further by a way of example, the emotional indicators can be determined based on the identified salient 3D objects. Further by a way of example, the application of the 3D visual effect can be based on the identified salient 3D objects. In yet another example, the 3D visual effect can provide a scene or a background, possibly a 3D scene or a 3D background. Still further by way of example, a salient 3D object can be identified in the input 3D content. The salient 3D object can be extracted from the input 3D content and a 3D scene into which the extract salient 3D object is to be injected can be selected based at least on the emotion indication. Next, the salient 3D object can be injected to the selected 3D scene to provide the transformed 3D content. Still further by way of example, injecting the salient 3D object can involve processing and adapting at least one 3D characteristic of the salient 3D object according to a 3D characteristic of the selected 3D scene. In yet another example, injecting the salient 3D object can involve processing and adapting at least one 3D

characteristic of the 3D scene according to a 3D characteristic of the salient 3D object.

[097] According to examples of the presently disclosed subject matter, the emotional indicators can be obtained (312 and 363) by analyzing the input 3D content. By a way of example, analyzing the input 3D content can detect one or more 3D objects in the input 3D content, and the emotional indicators can be determined based on the detected 3D objects. According to further examples of the presently disclosed subject matter, a subset of the detected 3D objects can be selected, for example by selecting the 3D object most likely to evoke an emotional response, and the emotional indicators can be determined based on the selected subset of the detected 3D objects, ignoring the 3D objects not selected. According to further examples of the presently disclosed subject matter, the detected 3D objects can be used to access a database of 3D objects and/or a database of 3D models, and the emotional indicators can be determined based on the accessed entries in the database.

[098] According to examples of the presently disclosed subject matter, one or more 3D visual effects can be selected based on the emotional indicators (313 and 364). For example, a set of predefined rules can match each emotional indicator or each combination of emotional indicators to one or more 3D visual effects. As another example, a database of emotional indicators can be used, where each entry in the database can indicate one or more 3D visual effects. The database can be accessed using the obtained emotional indicators. A list of one or more 3D visual effects is compiled from the 3D visual effects indicated by all the database entries that were accessed.

[099] According to examples of the presently disclosed subject matter, one or more 3D visual effects can be selected based on the emotional indicators (313 and 364) by using a set of emotional indicator exemplars, where each emotional indicator exemplar is associated with one or more 3D visual effects. A subset of the emotional indicator exemplars is selected, for example based on measures of similarity that is calculated between the emotional indicators and the emotional indicator exemplars, for example by selecting the most similar emotional indicator exemplars. Further by a way of example, the 3D visual effects are determined according to the 3D visual effects associated with the selected subset of emotional indicator exemplars.

[0100] According to examples of the presently disclosed subject matter, examples of 3D visual effects include: blur effects that blur the entire 3D scene or parts of the 3D scene; distortion effect that distort the entire 3D scene, parts of the 3D scene, or distort the 2D projection of a 3D scene; brightness effect that change the brightness of the entire 3D scene or parts of the 3D scene; smoke effect that add smoke in the 3D scene at predefined locations; transition effect that move the entire 3D scene or parts of the 3D scene; any 2D visual effect that is applied to one or more 2D projections of the 3D scene; changing the egomotion; changing the intensity of one or more light sources; changing the color of one or more light sources; changing the position of one or more light sources; changing the directionality of one or more light sources; removing one or more light sources; adding one or more light sources.

[0101] According to further examples of the presently disclosed subject matter, given more than one 3D visual effects, one 3D visual effect can be selected, for example randomly or according to some predefined rule. As another example, given more than one 3D visual effects, two or more 3D visual effects can be combined to a single 3D visual effect, for example by applying the two or more 3D visual effects concurrently.

[0102] According to examples of the presently disclosed subject matter, applying the 3D visual effect (314 and 365) may involve an analysis of the input 3D content. For example, applying the 3D visual effect may involve obtaining the egomotion, for example in order to change the egomotion. As another example, applying the 3D visual effect may involve detecting one or more light sources in the 3D scene, for example in order to: changing the intensity of at least one of the identified one or more light sources; changing the color of at least one of the identified one or more light sources; changing the position of at least one of the identified one or more light sources; changing the directionality of at least one of the identified one or more light sources; removing at least one of the identified one or more light sources; adding one or more light sources. [0103] According to further examples of the presently disclosed subject matter, analysis of the input 3D content may include detection and/or identification of 3D objects in the 3D scene, and applying the 3D visual effect (314 and 365) may be based on the detected and/or identified 3D objects. Further by a way of example, applying the 3D visual effect may consist of: removing at least one of the identified 3D objects; changing at least one of the identified 3D objects;

duplicating at least one of the identified 3D objects; adding at least one 3D object; matting at least one of the identified 3D objects over a different 3D scene; changing the camera position and/or lighting position with respect to at least one of the identified 3D objects; changing the motion of at least one of the identified 3D objects, and so forth. According to further examples of the presently disclosed subject matter, one or more of the 3D objects may be analyzed to identify and/or measure one or more of the 3D object's properties. Further by a way of example, applying the 3D visual effect may be based on the identity or value of the 3D object's properties. Example of such 3D object's properties includes colors, shape, size, position, motion, and so forth. Further by a way of example, applying the 3D visual effect may consist of changing one or more of the 3D object's properties. According to further examples of the presently disclosed subject matter, the detected and/or identified 3D objects can be used to access a database of 3D objects or 3D models. Further by a way of example, applying the 3D visual effect may be based on the matches found in the database.

[0104] According to examples of the presently disclosed subject matter, applying the 3D visual effect (314 and 365) may involve fitting a mechanical mathematical model to the input 3D content, followed by changing the value of one or more parameters of the mechanical mathematical model, and re-render the 3D scene and generate the transformed 3D content using the mechanical mathematical model with the changed value of the parameters.

[0105] According to examples of the presently disclosed subject matter, the generated transformed input 3D content is outputted. For example, the information of the transformed input 3D content can be: stored on a storage device (for example using storage 30), transmitted to a remote destination over a communication network (for example using communication module 40, and further by a way of example the remote destination can be a cloud platform 210), visually presented, and so forth. [0106] According to further examples of the presently disclosed subject matter, 3D content or parts of 3D content can be visually presented: by presenting a 2D view of the 3D content using a 2D display device, by presenting a visual representation or a visual content of the 3D content using a 3D display device, and so forth.

[0107] Figure 4 is an illustration of a scenario (401) in which a happy emotional response translates an appropriate 3D visual effect that is applied to input 3D content. This example scenario includes a device (412) capable of displaying 3D content and of estimating an emotional response of the viewer (411). For example, device 100 can be used as a possible implementation of device 412. For example, the device 412 displays a 3D visual content, which results in a happy response in the viewer 411. Further by a way of example, the happy response is visually detected, for example by analyzing the facial expression in a captured image of viewer's 411 face. Example of a captured image of a face with a happy facial expression is given in 421. Further by a way of example, an appropriate 3D visual effect is selected and applied by device 412. For example, a 3D visual effect of colorful butterflies flying around in the 3D scene and in this particular around a salient object (the girl in the image) is demonstrated in 422. Here, the butterflies appear to fly around the head of the girl and behave as and interact with 3D objecta in space (the scene in this case). In addition, the salient object in the 3D scene 422 appears in full color.

[0108] Figure 5 is an illustration of a scenario (501) in which a sad emotional response translates to the usage of an appropriate 3D visual effect that is applied to input 3D content. This example scenario includes a device (512) capable of displaying 3D content and of estimating the emotional response of the viewer (511). For example, device 100 can be used as a possible implementation of device 512. For example, the device 512 displays a 3D visual content, which results in a sad response in the viewer 511. Further by a way of example, the sad response is visually detected, for example by analyzing the facial expression in a captured image of viewer's 511 face. Example of a captured image of a face with a sad facial expression is given in 521. Further by a way of example, an appropriate 3D visual effect is selected and applied by device 512. For example, a 3D visual effect of brown foliage leaves falling around a salient object in the 3D scene is demonstrated in 522. The leaves too appear to fall around the head of the girl and behave as and interact with 3D objects in space (the scene in this case). In addition, the salient object in the 3D scene 522 appears in black and white.

[0109] It will also be understood that the system according to the invention may be a suitably programmed computer. Likewise, the invention contemplates a computer program being readable by a computer for executing the method of the invention. The invention further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing the method of the invention.

[0110] The functions or algorithms described herein may be implemented in software or a combination of software and hardware in one embodiment. The software may consist of computer executable instructions stored on computer readable media, including tangible and non-transient computer readable media such as memory or other type of storage devices. Further, such functions correspond to modules, which are software, hardware, firmware or any combination thereof. Multiple functions may be performed in one or more modules as desired, and the embodiments described are merely examples. The software may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a computer system, such as a personal computer, server or other computer system.

Claims

A method of processing 3D content by a computing device, comprising:

obtaining input 3D content;

obtaining one or more emotion indicators related to emotional effects that are evoked by the input 3D content;

selecting by a computing device one or more 3D visual effects, wherein the selection is based on the one or more emotion indications;

applying the one or more 3D visual effects to the input 3D content to thereby generate a transformed 3D content.

The method of claim 1, wherein obtaining the input 3D content comprising: reading the input 3D content information from a storage device.

The method of claim 1, wherein obtaining the input 3D content comprising: receiving the input 3D content information from a communication network.

The method of claim 1, wherein obtaining the input 3D content comprising: capturing the input 3D content information using a 3D capturing device.

The method of claim 1, further comprising: outputting the transformed 3D content.

The method of claim 1, further comprising: rendering the transformed 3D content on an output device.

The method of claim 1, further comprising: storing the transformed 3D content on a storage device.

The method of claim 1, further comprising: transmitting the transformed 3D content over a communication network.

The method of claim 1, wherein selecting the one or more 3D visual effects is based on one or more predefined rules for that associates 3D visual effect to emotion indicators.

The method of claim 1, wherein selecting the one or more 3D visual effects comprising: accessing a database of 3D visual effects based on one or more emotion indicators to obtain one or more 3D visual effects.

11. The method of claim 1, wherein selecting the one or more 3D visual effects comprising: accessing a set of emotion indicator exemplars, wherein each emotion indicator exemplar is associated with one or more associated 3D visual effects;

calculating one or more measures of similarity between the one or more emotion

indicators to at least one of the emotion indicator exemplars;

selecting a subset of emotion indicator exemplars based on the one or more measures of similarity;

selecting one or more 3D visual effects based on the one or more associated 3D visual effects associated with the subset of emotion indicator exemplars.

12. The method of claim 1, wherein the one or more 3D visual effects are selected from a list of 3D visual effects that includes at least one of: blur effect, distortion effect, brightness effect, smoke effect, transition effect.

13. The method of claim 1, further comprising analyzing the input 3D content to obtain the egomotion, and wherein applying the one or more 3D visual effects is based on the obtained egomotion.

14. The method of claim 13, wherein the one or more 3D visual effects include changing the egomotion.

15. The method of claim 1, further comprising analyzing the input 3D content to identify one or more light sources, and wherein applying the one or more 3D visual effects is based on the identified one or more light sources.

16. The method of claim 15, wherein the one or more 3D visual effects includes at least one of: changing the intensity of at least one of the identified one or more light sources; changing the color of at least one of the identified one or more light sources; changing the position of at least one of the identified one or more light sources; changing the directionality of at least one of the identified one or more light sources; removing at least one of the identified one or more light sources; adding at least one light source.

17. The method of claim 1, further comprising analyzing the input 3D content to identify one or more 3D objects in the input 3D content, and wherein applying the one or more 3D visual effects is based on the identified one or more 3D objects.

18. The method of claim 17, wherein the identified one or more 3D objects are salient 3D

objects.

19. The method of claim 17, wherein the one or more 3D visual effects includes at least one of: removing at least one of the identified one or more 3D objects; changing at least one of the identified one or more 3D objects; duplicating at least one of the identified one or more 3D objects; adding at least one 3D object; matting at least one of the identified one or more 3D objects over a different 3D scene; changing the camera position and/or lighting position with respect to at least one of the identified one or more 3D objects; changing the motion of at least one of the identified one or more 3D objects.

20. The method of claim 17, further comprising analyzing the identified one or more 3D objects to obtain one or more properties of at least one of the identified one or more 3D objects, and wherein applying the one or more 3D visual effects is based on the one or more properties.

21. The method of claim 20, wherein the one or more properties includes at least one of: colors, shape, size, position, motion.

22. The method of claim 21, wherein applying the one or more 3D visual effects is based

changing the one or more properties.

23. The method of claim 17, further comprising selecting one or more matches to the identified one or more 3D objects in a database of 3D models; and wherein applying the one or more 3D visual effects is based the one or more matches.

24. The method of claim 1, wherein applying the one or more 3D visual effects comprising: fitting a mechanical mathematical model to the input 3D content;

changing the value of one or more parameters of the mechanical mathematical model; generating the transformed 3D content using the mechanical mathematical model with the changed value of the one or more parameters.

25. The method of claim 1, wherein obtaining the one or more emotion indicators comprises reading the one or more emotion indicators information from a storage device.

26. The method of claim 1, wherein obtaining the one or more emotion indicators comprises receiving the one or more emotion indicators information from a communication network.

27. The method of claim 1, wherein obtaining the one or more emotion indicators comprises measuring the emotional effect the input 3D content on one or more viewers.

28. The method of claim 27, wherein measuring the emotional effect includes at least one of: measuring viewer pulse, measuring viewer blood pressure, measuring properties of viewer skin, measuring conductivity of viewer skin, measuring resistance of viewer skin, measuring pH level of viewer skin, measuring moisture of viewer skin, measuring viewer perspiration, capturing and analyzing viewer voice, capturing and analyzing viewer images, capturing and analyzing images of viewer face, capturing and analyzing images of viewer face to identify facial expressions.

29. The method of claim 1, wherein obtaining the one or more emotion indicators comprising: accessing a set of 3D content exemplars, wherein each 3D content exemplar is associated with one or more associated emotion indicators;

calculating one or more measures of similarity between the input 3D content and at least one of the 3D content exemplars;

selecting a subset of 3D content exemplars based on the one or more measures of

similarity;

determining one or more emotion indicators based on the one or more associated

emotion indicators associated with the subset of 3D content exemplars.

30. The method of claim 1, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

estimating the emotional effects of the input 3D content on the anticipated viewer; determining the emotion indicators based on the estimated emotional effects.

31. The method of claim 1, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

accessing a set of viewer exemplars, wherein each viewer exemplar is associated with one or more associated emotion indicators;

calculating one or more measures of similarity between the anticipated viewer and at least one of the viewer exemplars;

selecting a subset of viewer exemplars based on the one or more measures of similarity; determining one or more emotion indicators based on the one or more associated

emotion indicators associated with the subset of viewer exemplars.

32. The method of claim 1, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

accessing a set of viewer exemplars paired with 3D content exemplars, wherein each pair is associated with one or more associated emotion indicators;

calculating one or more measures of similarity between the anticipated viewer and the input 3D content on the one hand, to at least one of the pairs on the other hand; selecting a subset of pairs based on the one or more measures of similarity;

determining one or more emotion indicators based on the one or more associated

emotion indicators associated with the subset of pairs.

33. The method of claim 1, wherein obtaining the one or more emotion indicators comprising: analyzing the input 3D content to identify one or more 3D objects in the input 3D content; determining one or more emotion indicators based on one or more identified 3D objects.

34. The method of claim 33, wherein the identified one or more 3D objects are salient 3D

objects.

35. The method of claim 33, wherein determining one or more emotion indicators based on one or more identified 3D objects comprising: selecting of subset of the one or more identified 3D objects;

determining one or more emotion indicators based on the selected subset of the one or more identified 3D objects, ignoring all other identified 3D objects.

36. The method of claim 33, wherein determining one or more emotion indicators based on one or more identified 3D objects comprising: selecting one or more matches to the one or more identified 3D objects in a database of 3D models;

determining one or more emotion indicators based on the one or more matches.

37. A method of processing 3D content by a computing device, comprising:

obtaining input 3D content;

presenting a first part of the input 3D content to a viewer;

obtaining one or more emotion indicators related to emotional effects that are evoked in the viewer by the first part of the input 3D content;

applying the one or more 3D visual effects to a second part of the input 3D content to thereby generate a transformed 3D content;

presenting the transformed 3D content to the viewer.

38. The method of claim 37, wherein the duration of presenting the first part of the input 3D content in shorter than one minute.

39. The method of claim 37, wherein the duration of presenting the first part of the input 3D content in shorter than ten seconds.

40. The method of claim 37, wherein the duration of presenting the first part of the input 3D content in shorter than two seconds.

41. The method of claim 37, wherein obtaining the input 3D content comprising: reading the input 3D content information from a storage device.

42. The method of claim 37, wherein obtaining the input 3D content comprising: receiving the input 3D content information from a communication network.

43. The method of claim 37, wherein obtaining the input 3D content comprising: capturing the input 3D content information using a 3D capturing device.

44. The method of claim 37, wherein selecting the one or more 3D visual effects is based on one or more predefined rules for that associates 3D visual effect to emotion indicators.

45. The method of claim 37, wherein selecting the one or more 3D visual effects comprising: accessing a database of 3D visual effects based on one or more emotion indicators to obtain one or more 3D visual effects.

46. The method of claim 37, wherein selecting the one or more 3D visual effects comprising: accessing a set of emotion indicator exemplars, wherein each emotion indicator exemplar is associated with one or more associated 3D visual effects;

calculating one or more measures of similarity between the one or more emotion

indicators to at least one of the emotion indicator exemplars;

47. The method of claim 37, wherein the one or more 3D visual effects are selected from a list of 3D visual effects that includes at least one of: blur effect, distortion effect, brightness effect, smoke effect, transition effect.

48. The method of claim 37, further comprising analyzing the input 3D content to obtain the egomotion, and wherein applying the one or more 3D visual effects is based on the obtained egomotion.

49. The method of claim 48, wherein the one or more 3D visual effects include changing the egomotion.

50. The method of claim 37, further comprising analyzing the input 3D content to identify one or more light sources, and wherein applying the one or more 3D visual effects is based on the identified one or more light sources.

51. The method of claim 50, wherein the one or more 3D visual effects includes at least one of: changing the intensity of at least one of the identified one or more light sources; changing the color of at least one of the identified one or more light sources; changing the position of at least one of the identified one or more light sources; changing the directionality of at least one of the identified one or more light sources; removing at least one of the identified one or more light sources; adding at least one light source.

52. The method of claim 37, further comprising analyzing the input 3D content to identify one or more 3D objects in the input 3D content, and wherein applying the one or more 3D visual effects is based on the identified one or more 3D objects.

53. The method of claim 52, wherein the identified one or more 3D objects are salient 3D

objects.

54. The method of claim 52, wherein the one or more 3D visual effects includes at least one of: removing at least one of the identified one or more 3D objects; changing at least one of the identified one or more 3D objects; duplicating at least one of the identified one or more 3D objects; adding at least one 3D object; matting at least one of the identified one or more 3D objects over a different 3D scene; changing the camera position and/or lighting position with respect to at least one of the identified one or more 3D objects; changing the motion of at least one of the identified one or more 3D objects.

55. The method of claim 52, further comprising analyzing the identified one or more 3D objects to obtain one or more properties of at least one of the identified one or more 3D objects, and wherein applying the one or more 3D visual effects is based on the one or more properties.

56. The method of claim 55, wherein the one or more properties includes at least one of:

colors, shape, size, position, motion.

57. The method of claim 56, wherein applying the one or more 3D visual effects is based

changing the one or more properties.

58. The method of claim 52, further comprising selecting one or more matches to the identified one or more 3D objects in a database of 3D models; and wherein applying the one or more 3D visual effects is based the one or more matches.

59. The method of claim 37, wherein applying the one or more 3D visual effects comprising: fitting a mechanical mathematical model to the input 3D content;

60. The method of claim 37, wherein obtaining the one or more emotion indicators comprises reading the one or more emotion indicators information from a storage device.

61. The method of claim 37, wherein obtaining the one or more emotion indicators comprises receiving the one or more emotion indicators information from a communication network.

62. The method of claim 37, wherein obtaining the one or more emotion indicators comprises measuring the emotional effect the input 3D content on one or more viewers.

63. The method of claim 62, wherein measuring the emotional effect includes at least one of: measuring viewer pulse, measuring viewer blood pressure, measuring properties of viewer skin, measuring conductivity of viewer skin, measuring resistance of viewer skin, measuring pH level of viewer skin, measuring moisture of viewer skin, measuring viewer perspiration, capturing and analyzing viewer voice, capturing and analyzing viewer images, capturing and analyzing images of viewer face, capturing and analyzing images of viewer face to identify facial expressions.

64. The method of claim 37, wherein obtaining the one or more emotion indicators comprising: accessing a set of 3D content exemplars, wherein each 3D content exemplar is associated with one or more associated emotion indicators;

selecting a subset of 3D content exemplars based on the one or more measures of

similarity;

determining one or more emotion indicators based on the one or more associated

emotion indicators associated with the subset of 3D content exemplars.

65. The method of claim 37, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

estimating the emotional effects of the input 3D content on the anticipated viewer;

determining the emotion indicators based on the estimated emotional effects.

66. The method of claim 37, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

emotion indicators associated with the subset of viewer exemplars.

67. The method of claim 37, wherein obtaining the one or more emotion indicators comprising: obtaining information about an anticipated viewer;

determining one or more emotion indicators based on the one or more associated

emotion indicators associated with the subset of pairs.

68. The method of claim 37, wherein obtaining the one or more emotion indicators comprising: analyzing the input 3D content to identify one or more 3D objects in the input 3D content; determining one or more emotion indicators based on one or more identified 3D objects.

69. The method of claim 68, wherein the identified one or more 3D objects are salient 3D

objects.

70. The method of claim 68, wherein determining one or more emotion indicators based on one or more identified 3D objects comprising: selecting of subset of the one or more identified 3D objects;

71. The method of claim 68, wherein determining one or more emotion indicators based on one or more identified 3D objects comprising: selecting one or more matches to the one or more identified 3D objects in a database of 3D models;

determining one or more emotion indicators based on the one or more matches.

72. An apparatus , comprising:

at least one memory units;

at least one processing devices configured to:

obtain input 3D content;

obtain one or more emotion indicators related to emotional effects that are evoked by the input 3D content;

select by a computing device one or more 3D visual effects, wherein the

selection is based on the one or more emotion indications;

apply the one or more 3D visual effects to the input 3D content to thereby

generate a transformed 3D content.

73. The apparatus of claim 72, further comprising of a storage device, wherein the at least one processing devices is further configured to read the input 3D content information from the storage device.

74. The apparatus of claim 72, further comprising of a communication device, wherein the at least one processing devices is further configured to receive the input 3D content information from the communication device.

75. The apparatus of claim 72, further comprising of a 3D capturing device, wherein the at least one processing devices is further configured to capture the input 3D content information using the 3D capturing device.

76. The apparatus of claim 72, further comprising of at least one visual output device, wherein the at least one processing devices is further configured to render the transformed 3D content on the at least one visual output device.

77. The apparatus of claim 76, wherein the at least one visual output device is at least one visual 3D output device.

78. The apparatus of claim 72, further comprising of a storage device, wherein the at least one processing devices is further configured to store the transformed 3D content on the storage device.

79. The apparatus of claim 72, further comprising of a communication device, wherein the at least one processing devices is further configured to transmit the transformed 3D content through the communication device.

80. The apparatus of claim 72, further comprising of a storage device, wherein the at least one processing devices is further configured to read the one or more emotion indicators information from the storage device.

81. The apparatus of claim 72, further comprising of a communication device, wherein the at least one processing devices is further configured to receive the one or more emotion indicators information through the communication device.

82. The apparatus of claim 72, further comprising of one or more biometric sensors, wherein the at least one processing devices is further configured to obtain biometric measurements from the one or more biometric sensors, and to estimates the emotional effect based on the biometric measurements.

83. The apparatus of claim 82, wherein the biometric measurements includes at least one of: viewer pulse, viewer blood pressure, properties of viewer skin, conductivity of viewer skin, resistance of viewer skin, pH level of viewer skin, moisture of viewer skin, viewer perspiration.

84. An apparatus , comprising:

at least one memory units; at least one visual output devices;

at least one processing devices configured to:

obtain input 3D content;

present a first part of the input 3D content to a viewer;

obtain one or more emotion indicators related to emotional effects that are evoked in the viewer by the first part of the input 3D content;

select by a computing device one or more 3D visual effects, wherein the

selection is based on the one or more emotion indications;

apply the one or more 3D visual effects to a second part of the input 3D content to thereby generate a transformed 3D content;

present the transformed 3D content on the at least one visual output devices.

85. The apparatus of claim 84, wherein the at least one visual output devices is at least one visual 3D output devices.

86. The apparatus of claim 84, further comprising of a storage device, wherein the at least one processing devices is further configured to read the input 3D content information from the storage device.

87. The apparatus of claim 84, further comprising of a communication device, wherein the at least one processing devices is further configured to receive the input 3D content information from the communication device.

88. The apparatus of claim 84, further comprising of a 3D capturing device, wherein the at least one processing devices is further configured to capture the input 3D content information using the 3D capturing device.

89. The apparatus of claim 84, further comprising of a storage device, wherein the at least one processing devices is further configured to read the one or more emotion indicators information from the storage device.

90. The apparatus of claim 84, further comprising of a communication device, wherein the at least one processing devices is further configured to receive the one or more emotion indicators information through the communication device.

91. The apparatus of claim 84, further comprising of one or more biometric sensors, wherein the at least one processing devices is further configured to obtain biometric measurements from the one or more biometric sensors, and to estimates the emotional effect based on the biometric measurements.

92. The apparatus of claim 91, wherein the biometric measurements includes at least one of: viewer pulse, viewer blood pressure, properties of viewer skin, conductivity of viewer skin, resistance of viewer skin, pH level of viewer skin, moisture of viewer skin, viewer perspiration.

93. A software product stored on a non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out the method of claim 1.

94. A software product stored on a non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out the method of claim 37.