WO2014002790A1 - Image processing device, image processing method, and program - Google Patents

Abstract

The present invention pertains to an image processing device, an image processing method and a program that enable icons and the like, which are intended to be handled as 3D icons, to be superimposed more quickly. A primary processing unit superimposes first superimposition data among a plurality of superimposition data retained in advance on a first viewpoint image acquired from a first viewpoint. A secondary processing unit superimposes the first superimposition data among the plurality of superimposition data retained in advance on a second viewpoint image obtained from a second viewpoint, which differs from the first viewpoint, in synchronization with the primary processing unit. A 3D image generating unit generates a 3D image composed of the superimposed first viewpoint image and the superimposed second viewpoint image. The present invention can be used in an image processing device that generates 3D images, for example.

Description

Image processing apparatus, image processing method, and program

The present disclosure relates to an image processing device, an image processing method, and a program, and more particularly, to an image processing device, an image processing method, and a program that can more quickly superimpose icons and the like for 3D images. .

Conventionally, for example, there is an OSD (On Screen Display) superposition technique for superimposing an icon or the like on a 3D image using one LSI (Large Scale Integration) (see, for example, Patent Documents 1 and 2).

Here, the 3D image is an image that is recognized by the viewer as a three-dimensional image, and is, for example, a two-dimensional image for the right eye that is visually recognized by the viewer's right eye, and is visually recognized by the viewer's left eye. It is comprised from the two-dimensional image for left eyes for doing.

For this reason, the superimposition of an icon or the like on the 3D image is realized by superimposing the icon or the like on the right-eye two-dimensional image and the left-eye two-dimensional image constituting the 3D image.

JP 2011-55148 A JP 2011-135252 A

By the way, according to the conventional OSD superimposing technology, since the superimposition of the icon or the like is performed by one LSI, the superimposition of the icon or the like cannot be performed relatively quickly.

The present disclosure has been made in view of such a situation, and is intended to more quickly superimpose icons and the like for 3D images.

An image processing apparatus according to an aspect of the present disclosure includes a main processing unit that superimposes first superimposition data on a first viewpoint image obtained from a first viewpoint among a plurality of superposition data held in advance. A second viewpoint image obtained from a second viewpoint different from the first viewpoint from among the plurality of superimposing data held in advance in synchronization with the main processing unit. The image processing apparatus includes a slave processing unit that superimposes the image, a first viewpoint image after superimposition, and a 3D image generation unit that generates a 3D image composed of the second viewpoint image after superimposition.

A first output unit that outputs the first viewpoint image to the main processing unit and a second output unit that outputs the second viewpoint image to the sub-processing unit may be further provided.

The main processing unit communicates a synchronization signal for performing processing in synchronization with the main processing unit to the sub processing unit via a dedicated communication path, and the sub processing unit The first superimposing data can be superimposed on the second viewpoint image in synchronization with the synchronization signal communicated from the main processing unit via a communication path.

The main processing unit includes a first holding unit that holds the plurality of superimposing data in advance, a first changing unit that changes an internal state of the main processing unit based on a user input operation, and the first Of the plurality of superposition data held in one holding unit, the first superposition data indicating the internal state of the main processing unit changed by the first change unit is used as the first superposition data. A first superimposing unit that superimposes on the viewpoint image.

The secondary processing unit is configured to change the internal state of the secondary processing unit in synchronization with a change in the internal state of the second processing unit that holds the plurality of superimposing data in advance and the main processing unit. A second changing unit that changes the internal state to the same, and among the plurality of superimposition data held in the second holding unit, the second processing unit changed by the second changing unit A second superimposing unit that superimposes the first superimposing data indicating an internal state on the second viewpoint image.

The image processing apparatus may be provided with a plurality of sub processing units, and the 3D image generation unit may be obtained from the first viewpoint image after superimposition and the plurality of sub processing units, respectively. A 3D image composed of the second viewpoint image can be generated.

An image processing method according to an aspect of the present disclosure is an image processing method of an image processing device that generates a 3D image, and the image processing device includes a main processing unit, a sub processing unit, and a 3D image generation unit, The main processing unit superimposes first superposition data among a plurality of superposition data held in advance on a first viewpoint image obtained from a first viewpoint, and the sub-processing unit includes the main processing. The first superimposition data is superimposed on a second viewpoint image obtained from a second viewpoint different from the first viewpoint among the plurality of superimposition data held in advance in synchronization with the unit. The image processing method includes a step in which the 3D image generation unit generates a 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition.

A program according to an aspect of the present disclosure includes a main processing unit that superimposes a first superposition data among a plurality of superposition data held in advance on a first viewpoint image obtained from a first viewpoint. The second viewpoint obtained from the second viewpoint different from the first viewpoint among the plurality of overlapping data held in advance in synchronization with the main processing unit. A program for functioning as a 3D image generation unit that generates a 3D image composed of a slave processing unit to be superimposed on an image, the first viewpoint image after being superimposed, and the second viewpoint image after being superimposed. .

According to the present disclosure, among the plurality of superimposing data held in advance by the main processing unit, the first superimposing data is superimposed on the first viewpoint image obtained from the first viewpoint, and the main processing unit Synchronously, among the plurality of superimposing data held in advance by the slave processing unit, the first superimposing data is converted into a second viewpoint image obtained from a second viewpoint different from the first viewpoint. A 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition is generated.

According to the present disclosure, it is possible to more quickly superimpose icons and the like for 3D images.

It is a block diagram which shows the structural example of the image processing apparatus to which this technique is applied. It is a block diagram which shows the detailed structural example of a main process part, a sub process part, and a production | generation display part. It is a flowchart for demonstrating the display process which an image processing apparatus performs. It is a block diagram which shows the structural example of a computer.

Hereinafter, an embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described. The description will be given in the following order.
1. This embodiment (an example when the main processing unit and the sub-processing unit each hold OSD data in advance)
2. Modified example

<1. Embodiment>
[Configuration Example of Image Processing Apparatus 1]
FIG. 1 shows a configuration example of an image processing apparatus 1 according to the present embodiment.

The image processing apparatus 1 is, for example, a digital camera that captures a 3D image, and includes lenses 21 ₁ and 21 ₂ , imaging elements 22 ₁ and 22 ₂ , a main processing unit 23 ₁ , a sub processing unit 23 ₂ , and an operation unit 24. , A data storage unit 25, relay units 26 and 27, and a generation display unit 28.

In the present embodiment, the image processing apparatus 1 is described as a digital camera that captures a 3D image. However, the image processing apparatus 1 is not limited to a digital camera as long as it is a device that performs a process of superimposing an icon or the like on a 3D image. The present technology can also be applied to an apparatus.

The image processing apparatus 1 suppresses the communication amount of data communicated between the main processing unit 23 ₁ and the sub-processing unit 23 ₂ and quickly superimposes OSD data on the 3D image.

Here, the 3D image is an image recognized by the user (viewer) as a three-dimensional image. For example, a 2D image for the right eye for allowing the user's right eye to visually recognize the image and a left eye of the user. It is composed of a left-eye two-dimensional image for visual recognition.

The 3D image may be composed of two two-dimensional images (two-dimensional image for right eye and two-dimensional image for left eye) and three or more two-dimensional images obtained from different viewpoints. it can.

However, in the present embodiment, in order to simplify the description, the 3D image is assumed to be composed of two two-dimensional images, that is, a right-eye two-dimensional image and a left-eye two-dimensional image. To do.

That is, in the image processing apparatus 1, the main processing unit 23 ₁ and the sub processing unit 23 ₂ hold a plurality of OSD data in advance, so that the OSD is connected between the main processing unit 23 ₁ and the sub processing unit 23 _2. Data is not exchanged.

Further, the imaging element 22 ₁ outputs the right-eye two-dimensional image to the main processing unit 23 ₁ , and the imaging element 22 ₂ outputs the left-eye two-dimensional image to the sub-processing unit 23 ₂ , thereby performing main processing. The two-dimensional image for the right eye and the two-dimensional image for the left eye are not exchanged between the unit 23 ₁ and the sub-processing unit 23 ₂ .

Thereby, the image processing apparatus 1 suppresses the communication amount of data communicated between the main processing unit 23 ₁ and the sub-processing unit 23 ₂ and reduces the time required for the communication, thereby superimposing the OSD data. To do it quickly.

In the image processing apparatus 1 of FIG. 1, light from the outside (for example, reflected light from a subject) is incident on the image sensor 22 ₁ via an optical system such as a lens 21 ₁ . Further, external light is incident on the image sensor 22 ₂ via an optical system such as the lens 21 ₂ .

The imaging element 22 ₁ performs photoelectric conversion for converting the light from the lens 21 ₁ into an image signal, and supplies a right-eye two-dimensional image as an image signal obtained as a result to the main processing unit 23 ₁ .

In addition, the imaging element 22 ₂ performs photoelectric conversion for converting light from the lens 21 ₂ into an image signal, and supplies a two-dimensional image for the left eye as an image signal obtained as a result to the slave processing unit 23 ₂ .

Here, as the image sensors 22 ₁ and 22 ₂ , for example, image sensors such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor) are employed.

The lens 21 ₁ and the lens 21 ₂ are provided at positions separated by a predetermined distance (for example, a distance corresponding to the distance between the human right eye and the left eye).

For this reason, the image sensor 22 ₁ outputs a two-dimensional image for the right eye obtained by imaging the subject from the first viewpoint to the main processing unit 23 ₁ .

In addition, the imaging element 22 ₂ outputs a left-eye two-dimensional image obtained by capturing an image of a subject from a second viewpoint different from the first viewpoint to the slave processing unit 23 ₂ .

The main processing unit 23 _1, for example, among the plurality of OSD data held in advance, the OSD data representing the internal state of the main processor 23 _1, superimposed on the right eye two-dimensional images from the imaging device 22 _1, the The right-eye OSD superimposed image obtained by superposition is output to the generation display unit 28.

Here, the internal state of the main processing unit 23 _1, for example, and the internal state of the main processor 23 ₁ performs the image stabilization processing for the two-dimensional image for the right eye, the main processing unit 23 ₁ is the right-eye 2 An internal state in which a dimensional image is recorded in the data storage unit 25 can be considered. The same applies to the internal state of the sub processor 23 ₂ .

For example, the main processing unit 23 ₁ changes the internal state of the main processing unit 23 ₁ based on an operation signal from the operation unit 24.

Further, for example, the main processing unit 23 ₁ controls the sub processing unit 23 ₂ to perform the same processing as the main processing unit 23 ₁ . Note that the processing of the main processing unit 23 ₁ and the sub processing unit 23 ₂ is performed in synchronization with the synchronization signal generated by the main processing unit 23 ₁ .

That is, for example, the main processing unit 23 ₁ generates a synchronization signal for synchronizing with the main processing unit 23 _1, and transmits the generated synchronization signal to the slave processing unit 23 ₂ via the relay unit 26.

In addition, for example, the main processing unit 23 ₁ generates instruction data for instructing the sub-processing unit 23 ₂ to change the internal state, output a left-eye OSD superimposed image to be described later, and the like. The instruction data is transmitted to the slave processing unit 23 ₂ via the relay unit 27.

Here, in the following description, the instruction data for instructing the change of the internal state is referred to as first instruction data. Also, the instruction data that instructs the output of the left-eye OSD superimposed image is referred to as second instruction data. Note that the instruction data is not limited to the first instruction data and the second instruction data.

The sub processor 23 ₂ receives the synchronization signal transmitted from the main processor 23 ₁ via the relay unit 26 and the instruction data transmitted from the main processor 23 ₁ via the relay unit 27.

Then, the sub processor 23 ₂ performs the same processing as the main processor 23 ₁ in synchronization with the synchronization signal from the relay unit 26 based on the instruction data from the relay unit 27.

That is, for example, the slave processing unit 23 ₂ changes the internal state of the slave processing unit 23 ₂ in synchronization with the synchronization signal from the relay unit 26 based on the first instruction data from the relay unit 27. Change to the same internal state as ₁ .

In this case, the change of the internal state of the sub processor 23 ₂ is performed at (almost) the same timing as the change of the internal state of the main processor 23 ₁ .

Further, for example, the slave processing unit 23 ₂ synchronizes the left eye from the image sensor 22 ₂ with OSD data representing the internal state of the slave processing unit 23 ₂ among a plurality of OSD data held in advance in synchronization with the synchronization signal. Is superimposed on the two-dimensional image. It is assumed that the sub processor 23 ₂ is instructed to superimpose OSD data by instruction data from the relay unit 27.

In this case, the superimposition of OSD data in the slave processing unit 23 ₂ is performed at the same timing as the superimposition timing of the OSD data in the main processing unit 23 ₁ .

Then, the _secondary processing unit 23 ₂ generates the left-eye OSD superimposed image obtained by the superimposition based on the second instruction data from the relay unit 27 in synchronization with the synchronization signal from the relay unit 26, To 28.

In this case, the output of the left-eye OSD superimposed image by the sub-processing unit 23 ₂ is performed at the same timing as the output timing of the right-eye OSD superimposed image from the main processing unit 23 ₁ .

Operation unit 24 is composed of operation buttons to be operated by the user, in response to being operated by the user, an operation signal corresponding to a user operation, and supplies to the main processor 23 _1.

The user can change the internal state of the image processing apparatus 1, that is, the internal state of the main processing unit 23 ₁ and the _secondary processing unit 23 ₂ by operating the operation unit 24.

The data storage unit 25 is, for example, a flash memory or the like, and stores the right-eye two-dimensional image from the main processing unit 23 ₁ and the left-eye two-dimensional image from the sub-processing unit 23 ₂ . The data storage unit 25 stores in advance a program executed by the main processing unit 23 ₁ and the sub-processing unit 23 ₂ .

The relay unit 26, between the main processing unit 23 ₁ and the sub processing unit 23 ₂ , a synchronization signal output from the main processing unit 23 ₁ and a response output from the sub processing unit 23 ₂ (from the main processing unit 23 ₁ Response to the data).

The relay unit 27, between the main processor 23 ₁ and the slave processing unit 23 _2, instruction data and output from the main processing unit 23 _1, output from the slave processing unit 23 _2, the response to the instruction data Relay etc.

In the image processing apparatus 1, the synchronization signal is communicated via a dedicated communication path (communication path via the relay unit 26) different from the communication path for communicating instruction data and the like.

As a result, a situation in which the synchronization signal supplied from the main processing unit 23 ₁ to the sub processing unit 23 ₂ is delayed is prevented, and the main processing unit 23 ₁ and the sub processing unit 23 ₂ are accurately synchronized. I am doing so.

The generation display unit 28, based on the OSD superimposed image for the right eye from the main processing unit 23 ₁ and the OSD superimposed image for the left eye from the sub-processing unit 23 ₂ , the OSD superimposed image for the right eye and the OSD superimposed for the left eye. A 3D image composed of images is generated and displayed on a built-in display unit (not shown).

[Detailed Configuration Example of Main Processing Unit 23 ₁ , Sub-Processing Unit 23 ₂ , and Generation Display Unit 28]
Next, FIG. 2 shows a detailed configuration example of the main processing unit 23 ₁ , the sub-processing unit 23 ₂ , and the generation display unit 28.

The main processing unit 23 ₁ includes an image processing unit 41 ₁ , an AD (analog to digital) conversion unit 42 ₁ , a CPU (central processing unit) 43 ₁ , a holding unit 44 ₁ , a RAM (random access memory) 45 ₁ , and a superposition unit 46. ₁ and a communication unit 47 ₁ .

A right-eye two-dimensional image is supplied from the image sensor 22 ₁ to the image processing unit 41 ₁ . The image processing unit 41 ₁ performs image processing such as gamma correction processing, white balance processing, and camera shake correction processing on the two-dimensional image for the right eye from the image sensor 22 _1, and performs two-dimensional processing for the right eye after the image processing. The image is supplied to the AD conversion unit 42 ₁ .

The AD conversion unit 42 ₁ performs AD conversion for converting the two-dimensional image for the right eye as an analog signal from the image processing unit 41 ₁ into the two-dimensional image for the right eye as a digital signal, and performs the right conversion as a digital signal. a two-dimensional image for the eye, and supplies the CPU 43 _1.

For example, the CPU 43 ₁ controls the image processing unit 41 ₁ , the AD conversion unit 42 ₁ , the superimposition unit 46 ₁ , and the communication unit 47 ₁ .

Further, for example, CPU 43 ₁ generates the synchronization signal in synchronization with the generated synchronization signal, performs a predetermined process.

That is, for example, the CPU 43 ₁ changes the internal state of the main processing unit 23 ₁ based on the operation signal from the operation signal 25. State information indicating the internal state of the main processor 23 _1, for example, is held in the built-in memory or the like (not shown) of the CPU 43 _1.

Further, for example, the CPU 43 ₁ uses the first instruction data for instructing to change the internal state of the sub processor 23 _{2 to} the same internal state as the main processor 23 ₁ , and the left eye from the sub processor 23 _2. Second instruction data or the like for instructing to output the OSD superimposed image is generated. Then, the CPU 43 ₁ supplies the generated first instruction data, second instruction data, and the like together with the synchronization signal to the communication unit 47 ₁ .

Further, for example, the CPU 43 ₁ controls the superimposing unit 46 ₁ to perform processing for superimposing OSD data representing the internal state of the main processing unit 23 _{1 on} the right-eye two-dimensional image.

That is, for example, the CPU 43 ₁ supplies the right-eye two-dimensional image from the AD conversion unit 42 ₁ to the RAM 45 ₁ and temporarily holds it.

For example, the CPU 43 ₁ identifies the internal state of the main processing unit 23 ₁ based on the state information held in a built-in memory (not shown). Furthermore, for example, CPU 43 _1, among the plurality of OSD data previously held in the holding unit 44 _1, the OSD data representing the internal state of the main processor 23 ₁ were identified, read from the holding unit 44 _1, RAM 45 ₁ To temporarily hold.

Then, CPU 43 ₁ from RAM 45 _1, reads out the retained of the right-eye 2-D image and OSD data, and supplies the superposed portions 46 ₁ to perform the overlap of the OSD data to the two-dimensional image for the right eye.

The holding unit 44 ₁ holds in advance a plurality of OSD data each representing the internal state of the main processing unit 23 ₁ .

RAM 45 ₁ is adapted to temporarily hold the data writing has been instructed from the CPU 43 _1, the data read is instructed from CPU 43 _1, and supplies the CPU 43 _1.

Superimposing unit 46 _1, to the right eye two-dimensional images from the CPU 43 _1, likewise superimposes the OSD data from the CPU 43 _1, supplies the right-eye OSD superimposed image obtained by the superposition, on the generation and display unit 28 To do.

The communication unit 47 ₁ supplies the synchronization signal from the main processing unit 23 ₁ to the communication unit 47 ₂ via the relay unit 26. Further, the communication unit 47 ₁ supplies the first instruction data, the second instruction data, and the like from the main processing unit 23 ₁ to the communication unit 47 ₂ via the relay unit 27.

In addition, for example, the communication unit 47 ₁ supplies data such as a response returned from the slave processing unit 23 ₂ via the relay unit 26 or the relay unit 27 to the CPU 43 ₁ .

The slave processing unit 23 ₂ includes an image processing unit 41 ₂ , an AD conversion unit 42 ₂ , a CPU 43 ₂ , a holding unit 44 ₂ , a RAM 45 ₂ , a superimposing unit 46 ₂ , and a communication unit 47 ₂ .

A left-eye two-dimensional image is supplied from the image sensor 22 ₂ to the image processing unit 41 ₂ . The image processing unit 41 ₂ performs image processing such as gamma correction processing, white balance processing, and camera shake correction processing on the two-dimensional image for the left eye from the image sensor 22 _2, and performs two-dimensional processing for the left eye after image processing. images, and supplies to the AD conversion unit 42 _2.

The AD conversion unit 42 ₂ performs AD conversion for converting the two-dimensional image for the left eye as an analog signal from the image processing unit 41 ₁ into the two-dimensional image for the left eye as a digital signal, and performs left conversion as a digital signal. a two-dimensional image for the eye, and supplies the CPU 43 _2.

The CPU 43 ₂ receives a synchronization signal transmitted from the CPU 43 ₁ of the main processing unit 23 ₁ via the communication unit 47 ₁ , the relay unit 26, and the communication unit 47 ₂ , and synchronizes with the received synchronization signal, The image processing unit 41 ₂ , AD conversion unit 42 ₂ , superimposing unit 46 ₂ , and communication unit 47 ₂ are controlled.

For example, the CPU 43 ₂ uses the first instruction data and the second instruction data transmitted from the CPU 43 ₁ of the main processing unit 23 ₁ via the communication unit 47 ₁ , the relay unit 27, and the communication unit 47 _2. Etc.

Then, CPU 43 ₂ in synchronization with the received synchronization signal, also performs the processing instructed by the first instruction data and the second instruction data received.

That is, for example, when the CPU 43 ₂ receives the first instruction data instructing the change of the internal state, the timing at which the change of the internal state of the main processing unit 23 ₁ is performed in synchronization with the synchronization signal (almost). At the same timing, the internal state of the sub processor 23 ₂ is changed.

Note that the state information indicating the internal state of the sub processor 23 ₂ is held in, for example, a built-in memory (not shown) of the CPU 43 ₂ .

Further, for example, CPU 43 _2, when receiving the second instruction data for instructing the output of the left-eye OSD superimposed image, in synchronization with the synchronizing signal, for the right eye from the main processing unit 23 ₁ of the overlapping portion 46 ₂ timing the same timing OSD superimposed image is output, to output the OSD superimposed image for the left eye superimposing unit 46 _1.

Further, for example, the CPU 43 ₂ controls the superimposing unit 46 ₂ to perform a process of superimposing OSD data representing the internal state of the slave processing unit 23 _{2 on} the left-eye two-dimensional image. Assume that the CPU 43 ₂ is instructed to superimpose OSD data by instruction data output from the CPU 43 ₁ and supplied.

That is, for example, the CPU 43 ₂ supplies the left-eye two-dimensional image from the AD conversion unit 42 ₂ to the RAM 45 ₂ and temporarily holds it.

Further, for example, the CPU 43 ₂ identifies the internal state of the sub processor 23 ₂ based on the state information held in a built-in memory (not shown). Furthermore, for example, CPU 43 _2, of the plurality of OSD data previously held in the holding unit 44 _2, the OSD data representing the internal state of the slave processor 23 ₂ identified, read from the storage portion 44 _2, RAM 45 ₂ To temporarily hold.

Then, CPU 43 ₂ from RAM 45 _2, reads out the retained two-dimensional image and OSD data for the left eye, and supplies to the superimposing unit 46 _2, and superimposes the OSD data into two-dimensional image for the left eye.

Holding portion 44 ₂ previously holds a plurality of OSD data respectively representing the internal state of the slave processing unit 23 _2. Note that the holding portion 44 _2, the same OSD data and a plurality of OSD data holding unit 44 ₁ is held is previously held.

RAM 45 ₂ is adapted to temporarily hold the data writing has been instructed from the CPU 43 _2, the data read is instructed from the CPU 43 _2, and supplies the CPU 43 _2.

The superimposing unit 46 ₂ superimposes the OSD data from the CPU 43 _{2 on the} two-dimensional image for the left eye from the CPU 43 ₂ , and supplies the OSD superimposed image for the left eye obtained by the superimposition to the generation display unit 28. To do.

The communication unit 47 ₂ supplies the synchronization signal from the relay unit 26 to the CPU 43 _2, and supplies the first instruction data, the second instruction data, and the like from the relay unit 27 to the CPU 43 ₂ .

For example, the communication unit 47 ₂ transmits a response from the relay unit 26 to the communication unit 47 ₁ via the relay unit 26. Further, the communication unit 47 ₂ transmits a response from the relay unit 27 to the communication unit 47 ₁ via the relay unit 27.

The generation display unit 28 includes a 3D image generation unit 61, a display control unit 62, and a display unit 63.

3D image generation unit 61, a right-eye OSD overlapping image supplied from the superimposing unit ₄₆₁ of the main processing unit 23 _1, the left-eye OSD overlapping image supplied from the superimposing unit 46 ₂ of the slave processing unit 23 ₂ Based on this, a 3D image composed of the OSD superimposed image for the right eye and the OSD superimposed image for the left eye is generated and supplied to the display control unit 62.

The display control unit 62 supplies the 3D image from the 3D image generation unit 61 to the display unit 63 for display.

The display unit 63 is, for example, an LCD (liquid crystal display) or the like, and displays a 3D image according to control from the display control unit 62.

That is, for example, the display unit 63 is provided with a parallax barrier, a lenticular lens, or the like that optically separates the right-eye two-dimensional image and the left-eye two-dimensional image included in the 3D image.

Therefore, the display unit 63 displays the OSD superimposed image for the left eye included in the 3D image so as to be visually recognized by the user's left eye using a parallax barrier, a lenticular lens, or the like, and for the right eye included in the 3D image. The OSD superimposed image is displayed so as to be visually recognized by the user's right eye.

The display unit 63 may cause the user to visually recognize the 3D image by alternately displaying the two-dimensional image for the right eye and the two-dimensional image for the left eye. In this case, the user needs to wear 3D glasses or the like that alternately block the field of view of the right eye and the left eye.

[Description of Operation of Image Processing Apparatus 1]
Next, display processing performed by the image processing apparatus 1 will be described with reference to the flowchart of FIG.

This display process is started, for example, when the image processing apparatus 1 is powered on.

In step S21, the processing branches depending on whether the main subject of processing is the main processing unit 23 ₁ or the sub processing unit 23 ₂ .

That is, in step S21, when the processing subject is the main processing unit 23 ₁ , the main processing unit 23 ₁ advances the processing to step S23, and when the processing subject is the sub processing unit 23 ₂ , the sub processing unit 23 _In step ₂ , the process proceeds to step S22.

In step S22, CPU 43 ₂ of the slave processing unit 23 _2, the CPU 43 ₁ of the main processing unit 23 _1, the communication unit 47 _1, via the relay unit 27 and the communication unit 47 _2, the internal state of the slave processor 23 ₂ It is determined whether or not first instruction data for instructing change has been received.

The CPU 43 ₂ of the slave processing unit 23 ₂ waits to determine that the first instruction data has been received, and then proceeds to step S25.

In step S25, the CPU 43 ₂ of the slave processing unit 23 ₂ synchronizes with the synchronization signal output and supplied from the CPU 43 ₁ of the main processing unit 23 ₁ based on the received first instruction data. The same processing as 23 ₁ is performed.

A synchronization signal is supplied to the CPU 43 ₂ of the slave processing unit 23 _{2 from} the CPU 43 ₁ of the main processing unit 23 ₁ via the communication unit 47 ₁ , the relay unit 26, and the communication unit 47 ₂ .

The processing performed by the _secondary processing unit 23 ₂ will be described together with the processing performed by the main processing unit 23 ₁ in steps S25 to S30.

In step S23, the CPU 43 ₁ of the main processing unit 23 ₁ receives an operation signal from the operation unit 24. If the CPU 43 ₁ of the main processing unit 23 ₁ does not receive an operation signal from the operation unit 24, the process proceeds to step S26, and the subsequent processing is performed.

In step S24, the CPU 43 ₁ generates first instruction data for instructing the change of the internal state of the slave processing unit 23 ₂ based on the operation signal from the operation unit 24, the communication unit 47 ₁ , the relay unit 27, The data is supplied to the CPU 43 ₂ of the slave processing unit 23 ₂ via the communication unit 47 ₂ .

In step S25, the CPU 43 ₁ changes the internal state of the main processing unit 23 ₁ based on the operation signal from the operation unit 24.

That is, for example, when the user performs a recording operation for causing the image processing apparatus 1 to perform recording using the operation unit 24, the operation unit 24 sends an operation signal corresponding to the user's recording operation to the CPU 43 _1. Output to.

In this case, based on the operation signal from the operation unit 24, the CPU 43 ₁ changes the internal state of the main processing unit 23 _{1 to} an internal state in which recording processing is performed.

Further, the CPU 43 ₂ of the slave processing unit 23 ₂ synchronizes with the synchronization signal based on the first instruction data received in step S22, at (almost) the same timing as the change of the internal state of the main processing unit 23 _1. Then, the internal state of the sub processor 23 ₂ is changed to the same internal state as that of the sub processor 23 ₁ .

In step S26, the image pickup element ₂₂₁ is photoelectrically converted transmitted light from the lens 21 _1, and outputs the result to the right eye two-dimensional images obtained, the image processing unit 41 of _the main processing unit 23 _1.

The imaging element 22 _2, lens photoelectrically converting the transmitted light from 21 _2, and outputs the result to the two-dimensional image for the left eye obtained, the image processing unit 41 ₂ of the slave processing unit 23 _2.

In step S27, the image processing unit 41 ₁ performs image processing on the right eye two-dimensional images from the imaging device 22 _1, the two-dimensional image for the right eye after the image processing, and supplies to the AD converter section 42 _1.

The image processing unit 41 ₂ performs image processing on the left-eye 2-D image from the image pickup element 22 _2, the two-dimensional image for the left eye after the image processing, and supplies to the AD conversion unit 42 _2.

In step S28, the AD converter section 42 _1, a right two-dimensional image for the eye from the image processing unit 41 ₁ to AD conversion, and supplies the CPU 43 _1. Then, the CPU 43 ₁ supplies the right-eye two-dimensional image from the AD conversion unit 42 ₁ to the RAM 45 ₁ and temporarily holds it.

Further, the AD conversion unit 42 _2, the left two-dimensional image for the eye from the image processing unit 41 ₂ performs AD conversion and supplies it to the CPU 43 _2. Then, the CPU 43 ₂ supplies the left-eye two-dimensional image from the AD conversion unit 42 ₂ to the RAM 45 ₂ and temporarily holds it.

In step S29, the main processing unit 23 ₁ of the CPU 43 _1, based on the status information held in the built-in memory (not shown) to identify the internal state of the main processor 23 _1.

Then, CPU 43 _1, among the plurality of OSD data stored in advance in storage portion 44 _1, the OSD data representing the internal state of the main processor 23 ₁ were identified, read from the holding unit 44 _1, supplied to the RAM 45 ₁ And hold it temporarily.

Further, CPU 43 ₂ of the slave processing unit 23 _2, based on the state information held in the built-in memory (not shown) to identify the internal state of the slave processing unit 23 _2.

Then, CPU 43 _2, of the plurality of OSD data stored in advance in storage portion 44 _2, the OSD data representing the internal state of the slave processor 23 ₂ identified, read from the storage portion 44 _2, supplied to the RAM 45 ₂ And hold it temporarily.

In step S30, CPU 43 _1, the two-dimensional image and OSD data for the right eye is held in RAM 45 _1, is read from RAM 45 _1, and supplies the superimposed portion 46 _1. The superimposing unit 46 _1, the OSD data supplied from the CPU 43 _1, likewise by superimposing the right eye two-dimensional images supplied from the CPU 43 _1, to produce an OSD superimposed image for the right eye.

Further, CPU 43 ₂ is the 2-dimensional image and OSD data for the left eye is held in RAM 45 _2, it is read from RAM 45 _2, and supplies the superimposed portion 46 _2. The superimposing unit 46 _2, CPU 43 the OSD data supplied from the ₂ by superimposing a two-dimensional image for the left eye supplied likewise from the CPU 43 _2, and generates an OSD superimposed image for the left eye.

In step S31, when the subject of processing is the main processing unit 23 ₁ , the main processing unit 23 ₁ advances the processing to step S33, and when the subject of processing is the sub processing unit 23 ₂ , the sub processing unit 23 ₂ Then, the process proceeds to step S32.

In step S32, CPU 43 ₂ of the slave processing unit 23 _2, the CPU 43 ₁ of the main processing unit 23 _1, the communication unit 47 _1, via the relay unit 27, and a communication unit 47 _2, the OSD superimposing image for the left eye output It is determined whether or not the second instruction data for instructing is received.

The CPU 43 ₂ of the slave processing unit 23 ₂ waits to determine that the second instruction data has been received, and advances the process to step S34.

In step S33, the CPU 43 ₁ of the main processing unit 23 ₁ generates second instruction data, and sends it to the CPU 43 ₂ of the slave processing unit 23 ₂ via the communication unit 47 ₁ , the relay unit 27, and the communication unit 47 _2. Send.

In step S34, the superimposing unit 46 ₁ of the main processing unit 23 ₁ outputs the right-eye OSD superimposed image generated in the immediately preceding step S30 to the 3D image generating unit 61 of the generation display unit 28.

Further, superimposing unit 46 ₂ of the slave processing unit 23 _2, based on the second instruction data received from CPU 43 ₁ of the main processing unit 23 _1, already generated OSD superimposed image for the left eye in step S30 of immediately preceding generation The data is output to the 3D image generation unit 61 of the display unit 28. Thus, the left eye OSD superimposed image is output from the superimposing unit 46 ₂ at the same timing as the right eye OSD superimposed image is output from the superimposing unit 46 ₁ .

In step S35, 3D image generation unit 61 of generating the display unit 28, and a right-eye OSD overlapping images from the superimposing unit _461, on the basis of the left-eye OSD overlapping images from the superimposing unit 46 _2, OSD for the right eye A 3D image including the superimposed image and the left-eye OSD superimposed image is generated and supplied to the display control unit 62.

In step S36, the display control unit 62 supplies the 3D image from the 3D image generation unit 61 to the display unit 63 for display. Thereby, the display unit 63 displays the OSD superimposed image for the left eye included in the 3D image so that the left eye of the user can visually recognize the OSD superimposed image for the right eye included in the 3D image. It is displayed so that it can be visually recognized.

In step S37, for example, the CPU 43 ₁ of the main processing unit 23 ₁ determines whether or not to continue the display process based on an operation signal from the operation unit 24. Returning to step S21, the same processing is performed thereafter.

In step S37, when the main processing unit 23 ₁ determines not to continue the display process, the display process ends.

As described above, according to the display processing, the holding portion 44 of _the main processing unit 23 _1, the holding portion 44 ₂ of the slave processing unit 23 _2, and so as to advance hold a plurality of OSD data.

For this reason, for example, the main processing unit 23 ₁ does not need to communicate the OSD data for the left-eye two-dimensional image to the sub-processing unit 23 ₂ , so that the time required for such communication can be omitted.

Further, for example, the image sensor 22 ₁ outputs a right-eye two-dimensional image obtained by photoelectric conversion of the light from the lens 21 ₁ to the main processing unit 23 ₁ , and the image sensor 22 ₂ has the lens 21 _2. The two-dimensional image for the left eye obtained by photoelectric conversion of the light from is output to the sub processor 23 ₂ .

Thus, for example, the main processing unit 23 _1, the image pickup element 22 ₂ to obtain a two-dimensional image for the left eye, as compared with the case of the configuration to communicate to the slave processor 23 _2, such communication Time required can be omitted.

Therefore, according to the display processing, it is possible to quickly superimpose OSD data on the 3D image composed of the two-dimensional image for the right eye and the two-dimensional image for the left eye.

Furthermore, for example, since the communication unit 47 ₁ is provided with the relay unit 26 for communicating the synchronization signal and the relay unit 27 for communicating the instruction data and the like separately, the delivery of the synchronization signal is delayed. Thus, it is possible to prevent a situation in which synchronization cannot be established between the main processing unit 23 ₁ and the sub-processing unit 23 ₂ .

For example, since the main processing unit 23 ₁ and the sub processing unit 23 ₂ are configured in the same manner, the main processing unit 23 ₁ and the sub processing unit 23 ₂ can be manufactured in the same manufacturing process. For this reason, the manufacturing cost of the image processing apparatus 1 can be reduced.

Further, for example, if the main processing unit 23 ₁ and the sub-processing unit 23 ₂ are manufactured using parts common to a conventional imaging device that images a subject, it is not necessary to develop an additional electronic circuit or software, Costs for development can be reduced.

<2. Modification>
In the present embodiment, the main processing unit 23 ₁ superimposes the OSD data on the two-dimensional image for the right eye from the image sensor 22 ₁ , and the slave processing unit 23 ₂ for the left eye from the image sensor 22 ₂ . The OSD data was superimposed on the 3D image by superimposing the OSD data on the two-dimensional image.

However, for example, when a 3D image is stored in the data storage unit 25, the OSD data may be superimposed on the 3D image stored in the data storage unit 25.

That is, the main processing unit 23 ₁ reads the right-eye two-dimensional image included in the 3D image from the data storage unit 25, and superimposes the OSD data on the read right-eye two-dimensional image. Further, the sub processor 23 ₂ reads the left-eye two-dimensional image included in the 3D image from the data storage unit 25, and superimposes the OSD data on the read left-eye two-dimensional image.

Further, for example, in the image processing apparatus 1, as shown in FIG. 1, one data storage unit 25 is provided. However, for example, the main processing unit 23 ₁ may include the data storage unit 25 ₁ and the sub-processing unit 23 ₂ may include the data storage unit 25 ₂ . It is assumed that the same 3D image is stored in the data storage unit 25 ₁ and the data storage unit 25 ₂ .

In this case, the main processing unit 23 ₁ reads the right-eye two-dimensional image included in the 3D image from the built-in data storage unit 25 ₁ and superimposes the OSD data on the read right-eye two-dimensional image. Further, the sub processor 23 ₂ reads the left-eye two-dimensional image included in the 3D image from the built-in data storage unit 25 ₂ and superimposes the OSD data on the read left-eye two-dimensional image.

Furthermore, in this embodiment, the image processing apparatus 1 has been described as including one slave processing unit 23 _2, can be configured to include two slave processing unit 23 ₂ and 23 _3.

In this case, for example, the image processing apparatus 1, is also possible as will the included lens 21 ₃ and the image pickup element 22 _3, the image pickup element 22 _3, the light from the lens 21 ₃ and photoelectric conversion, the image pickup device 22 ₁ and 22 It acquires two-dimensional images of different viewpoints from the image obtained from _2, and outputs to the slave processing unit 23 _3.

Then, the sub processor 23 ₃ outputs, from the image sensor 22 ₃ , OSD data indicating the internal state of the sub processor 23 ₃ changed by the control from the main processor 23 ₁ among the plurality of OSD data held in advance. The OSD superimposed image obtained by superimposing on the two-dimensional image is output to the 3D image generating unit 61.

The 3D image generation unit 61 generates a 3D image including the OSD superimposed image from the main processing unit 23 ₁ , the OSD superimposed image from the sub processing unit 23 _2, and the OSD superposed image from the sub processing unit 23 ₃ , and performs display control. To the unit 62.

The display control unit 62 supplies the 3D image from the 3D image generation unit 61 to the display unit 63 for display.

In the image processing apparatus 1, in addition to the two sub-processing units 23 ₂ and 23 ₃ , N (N is a natural number of 3 or more) sub-processing units 23 _{2 to} 23 _{N + 1} may be provided. it can.

By the way, this technique can take the following structures.
(1) A main processing unit that superimposes first superposition data among a plurality of superposition data held in advance on a first viewpoint image obtained from a first viewpoint, and in synchronization with the main processing unit A slave processing unit that superimposes the first superimposition data on a second viewpoint image obtained from a second viewpoint different from the first viewpoint among the plurality of superimposition data held in advance; An image processing apparatus comprising: the first viewpoint image after superimposition; and a 3D image generation unit that generates a 3D image composed of the second viewpoint image after superimposition.
(2) further including a first output unit that outputs the first viewpoint image to the main processing unit, and a second output unit that outputs the second viewpoint image to the sub-processing unit. The image processing apparatus according to (1).
(3) The main processing unit communicates a synchronization signal for performing processing in synchronization with the main processing unit to the sub processing unit via a dedicated communication path. (1) or (2) superimposing the first superimposition data on the second viewpoint image in synchronization with the synchronization signal communicated from the main processing unit via the dedicated communication path ).
(4) The main processing unit includes a first holding unit that holds the plurality of superimposing data in advance, and a first changing unit that changes an internal state of the main processing unit based on a user input operation. The first superimposing data indicating the internal state of the main processing unit changed by the first changing unit among the plurality of superimposing data held in the first holding unit, The image processing apparatus according to (1) or (2), further including a first superimposing unit that superimposes on the first viewpoint image.
(5) The sub processing unit synchronizes with the change of the internal state of the second processing unit that stores the plurality of superimposing data in advance and the internal state of the main processing unit. Of the plurality of superposition data held in the second holding unit, the second changing unit that changes to the same internal state as the main processing unit, and the slave changed by the second changing unit. The image processing apparatus according to (4), further including: a second superimposing unit that superimposes the first superimposing data indicating an internal state of the processing unit on the second viewpoint image.
(6) The image processing apparatus includes a plurality of sub processing units, and the 3D image generation unit is obtained from the first viewpoint image after superimposition and the plurality of sub processing units, respectively. The image processing device according to any one of (1) to (5), wherein a 3D image configured by the second viewpoint image is generated.
(7) In the image processing method of the image processing apparatus that generates a 3D image, the image processing apparatus includes a main processing unit, a sub processing unit, and a 3D image generation unit, and the main processing unit holds a plurality of images in advance. Of the superimposing data, the first superimposing data is superimposed on the first viewpoint image obtained from the first viewpoint, and the slave processing unit holds in advance in synchronization with the main processing unit. Of the plurality of data for superimposition, the first superimposition data is superimposed on a second viewpoint image obtained from a second viewpoint different from the first viewpoint, and the 3D image generation unit An image processing method including a step of generating a 3D image composed of the first viewpoint image and the superimposed second viewpoint image.
(8) A main processing unit that superimposes the first superposition data among a plurality of superposition data held in advance on the first viewpoint image obtained from the first viewpoint, and the main processing unit. A sub-process for superimposing the first superimposition data on a second viewpoint image obtained from a second viewpoint different from the first viewpoint among the plurality of superposition data held in synchronization And a program for functioning as a 3D image generation unit that generates a 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition.

The series of processes described above can be executed by hardware or software, for example. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose computer or the like.

[Computer configuration example]
FIG. 4 shows a configuration example of a computer that executes the above-described series of processing by a program.

CPU (Central Processing Unit) 81 executes various processes according to a program stored in ROM (Read Only Memory) 82 or storage unit 88. A RAM (Random Access Memory) 83 appropriately stores programs executed by the CPU 81, data, and the like. These CPU 81, ROM 82, and RAM 83 are connected to each other by a bus 84.

The CPU 81 is also connected with an input / output interface 85 via the bus 84. The input / output interface 85 is connected to an input unit 86 made up of a keyboard, a mouse, a microphone, etc., and an output unit 87 made up of a display, a speaker, etc. The CPU 81 executes various processes in response to commands input from the input unit 86. Then, the CPU 81 outputs the processing result to the output unit 87.

The storage unit 88 connected to the input / output interface 85 is composed of, for example, a hard disk, and stores programs executed by the CPU 81 and various data. The communication unit 89 communicates with an external device via a network such as the Internet or a local area network.

Further, the program may be acquired via the communication unit 89 and stored in the storage unit 88.

The drive 90 connected to the input / output interface 85 drives a removable medium 91 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the program or data recorded therein. Etc. The acquired program and data are transferred to and stored in the storage unit 88 as necessary.

As shown in FIG. 4, a recording medium for recording (storing) a program installed in a computer and ready to be executed by the computer is a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc- Removable media 91, which is a package media made up of a read-only memory, DVD (digital versatile disc), a magneto-optical disk (including MD (mini-disc)), or a semiconductor memory, or the program is temporarily or It is composed of a ROM 82 that is permanently stored, a hard disk that constitutes the storage unit 88, and the like. Recording of a program on a recording medium is performed using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 89 that is an interface such as a router or a modem as necessary. Is called.

In the present specification, the steps describing the series of processes described above are not limited to the processes performed in time series according to the described order, but are not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

Further, the present disclosure is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present disclosure.

DESCRIPTION OF SYMBOLS 1 Image processing device, 21 ₁ , 21 ₂ lens, 22 ₁ , 22 ₂ imaging device, 23 ₁ main processing unit, 23 ₂ sub processing unit, 24 operation unit, 25 data storage unit, 26, 27 relay unit, 28 generation display part, 41 _1, 41 ₂ image processing unit, 42 _1, 42 ₂ AD conversion unit, 43 _1, 43 ₂ CPU, 44 _1, 44 ₂ holding part, 45 _1, 45 ₂ RAM, 46 _1, 46 ₂ superimposing unit, 47 ₁ , 47 ₂ communication unit, 61 3D image generation unit, 62 display control unit, 63 display unit

Claims (8)

A main processing unit that superimposes first superimposition data among a plurality of superposition data held in advance on a first viewpoint image obtained from a first viewpoint;
A second viewpoint image obtained from a second viewpoint different from the first viewpoint from among the plurality of superimposing data held in advance in synchronization with the main processing unit. A slave processing unit superposed on
An image processing apparatus comprising: the first viewpoint image after superimposition; and a 3D image generation unit that generates a 3D image composed of the second viewpoint image after superimposition. A first output unit that outputs the first viewpoint image to the main processing unit;
The image processing apparatus according to claim 1, further comprising: a second output unit that outputs the second viewpoint image to the slave processing unit. The main processing unit communicates a synchronization signal for performing processing in synchronization with the main processing unit to the sub processing unit via a dedicated communication path,
The slave processing unit superimposes the first superimposition data on the second viewpoint image in synchronization with the synchronization signal communicated from the main processing unit via the dedicated communication path. Item 3. The image processing apparatus according to Item 2. The main processing unit
A first holding unit that holds the plurality of superimposing data in advance;
A first changing unit for changing an internal state of the main processing unit based on a user input operation;
Among the plurality of superposition data held in the first holding unit, the first superposition data indicating the internal state of the main processing unit changed by the first change unit is the first superposition data. The image processing apparatus according to claim 2, further comprising: a first superimposing unit that superimposes on one viewpoint image. The slave processing unit
A second holding unit that holds the plurality of superimposing data in advance;
In synchronization with the change of the internal state of the main processing unit, a second changing unit that changes the internal state of the sub processing unit to the same internal state as the main processing unit,
Of the plurality of superimposing data held in the second holding unit, the first superimposing data indicating the internal state of the slave processing unit changed by the second changing unit is the first superimposing data. The image processing apparatus according to claim 4, further comprising: a second superimposing unit that superimposes the two viewpoint images. The image processing apparatus includes a plurality of sub processing units,
2. The 3D image generation unit generates a 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition obtained from the plurality of sub-processing units, respectively. An image processing apparatus according to 1. In an image processing method of an image processing apparatus that generates a 3D image,
The image processing apparatus includes a main processing unit, a sub processing unit, and a 3D image generation unit,
The main processing unit superimposes the first superposition data among the plurality of superposition data held in advance on the first viewpoint image obtained from the first viewpoint,
The slave processing unit obtains the first superimposition data from a second viewpoint different from the first viewpoint among the plurality of superposition data held in advance in synchronization with the main processing unit. Superimposed on the second viewpoint image,
An image processing method including a step in which the 3D image generation unit generates a 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition. Computer
A main processing unit that superimposes first superimposition data among a plurality of superposition data held in advance on a first viewpoint image obtained from a first viewpoint;
A second viewpoint image obtained from a second viewpoint different from the first viewpoint from among the plurality of superimposing data held in advance in synchronization with the main processing unit. A slave processing unit superposed on
A program for functioning as a 3D image generation unit that generates a 3D image composed of the first viewpoint image after superimposition and the second viewpoint image after superimposition.

Images