JPH08102924A - Image input / output device - Google Patents

Abstract

(57) [Summary] [Structure] An image forming apparatus having a display function and an imaging function, the liquid crystal panel having a pixel drive element array 104 in which pixel drive elements for driving pixels for display are arranged in a matrix. 2 and an image pickup panel 6 having an image pickup element array 3 in which image pickup elements, which are a plurality of light receiving elements that perform light detection, are arranged in a matrix, are arranged to face each other. By providing the thin image pickup panel 6 which is a panel type image pickup means, the entire image input / output device can be made thin and the portability can be improved, and the display area and the image pickup area are the same, which makes the device thinner. While trying to
When used in an interactive display device such as a conference system, the eyes look natural and the user does not feel uncomfortable.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an input / output integrated image input / output device having both an image pickup function and a display function.

[0002]

2. Description of the Related Art Today, an input / output integrated image input / output device having both an image pickup function and a display function has been actively developed. As such an image input / output device, for example, there is a so-called video movie, which is a camera-integrated VTR (Video Tape Recorder) commercialized by the applicant of the present application under the product name of “liquid crystal view cam”. In this, the captured image can be immediately viewed on the spot. In addition to this, as a kind of image input / output device, for example, a TV (Television) telephone, a TV
So-called interactive display devices such as conference systems have also been developed. It should be noted that such a function as an interactive display device is provided in a part of the video movies including the video movie of the above-mentioned product name "liquid crystal view cam".

On the other hand, in a portable camera having only an image pickup function, a disc-shaped (disc-shaped) film or a semiconductor memory is used as a recording medium in order to reduce the size, weight and thickness in order to improve portability. The existing products have been proposed, and some have been commercialized. For example, JP-A-1-176616
In JP-A-8, a semiconductor memory is used as a recording medium,
An electronic still has been proposed in which the thickness of the camera body excluding the lens portion is 10 mm or less. Furthermore, JP-A-2-358
In Japanese Patent Laid-Open No. 74, there is proposed a configuration in which a solid-state image pickup device (CCD (Charge Coupled Device) device) chip is arranged on a card-shaped film wiring substrate and is attachable to and detachable from the camera body.

However, even in the cameras proposed in the above publications, the thickness of the lens portion is restricted and there is a limit to the reduction in thickness. Therefore, as one method of improving this, a "panel type image sensor" having a layered structure by combining a lens array, a pinhole array, and a photodiode array has been proposed (Electronics 1993. May issue p62-). p65).

[0005]

By the way, in the case of the conventional image input / output device, it is true that the display means is made extremely thin and the size and weight thereof are reduced while the flat panel such as a liquid crystal display device is being developed. Has been. However, in the configuration of the conventional image pickup means, as in the case of the portable camera having only the image pickup function, there is a limit to the thinning due to the thickness of the lens portion, and further thinning cannot be expected, This prevents the image input / output device from becoming thinner.

The thinning of the image input / output device is of course a problem in portability in a video movie used as a portable type and a portable information terminal of a TV phone, and it is a desktop type in a TV phone or a TV conference system. Even in the case of terminals, the proportion of them on the desk becomes large, which causes problems such as narrowing the work space on the desk.

In addition, with the recent progress of the information-oriented society, it is expected that there will be more opportunities for an individual to talk with a distant partner while watching the video of the partner, regardless of whether it is a portable information terminal or a desktop terminal. It However, in the configuration of the conventional image input / output device, since the screen of the display unit and the lens unit of the image pickup unit are placed apart from each other, the line of sight of the user looking at the screen and the image pickup direction of the image pickup unit are different from each other. Are not exactly the same, so that, for example, even if a user is in a video conference, the user with whom the user interacts sees another place even though the user is looking at the screen and talking. You may feel uncomfortable, such as feeling like you are talking.

[0008] Therefore, in order to prevent such a feeling of strangeness, with the video conference system in mind, for example, a. In Japanese Patent Laid-Open No. 4-315390, a semitransparent mirror is arranged on the entire screen of the display means, the image of the subject (user) is reflected on the semitransparent mirror, and the reflected light is projected by the image pickup means. A method of making the line of sight natural, or b. An optical shutter composed of a scattering type liquid crystal cell is provided on the back surface of the display means, and an image pickup means is arranged further behind it, and the scattering type liquid crystal cell is set in a scattering state during display, and an image pickup period is provided between the displays, and at that moment. A method has been proposed in which the line of sight is made natural by setting the scattering type liquid crystal cell in a transmissive state.

However, in the case of a portable terminal, if the method A disclosed in Japanese Patent Laid-Open No. 4-315390 is used, not only the size of the device increases but also the thickness of the device itself increases and the device becomes thin. There is a problem that you can not achieve. In a portable information terminal, not only weight reduction but also thinness is an important factor. In addition, when the optical shutter including the above-mentioned scattering type liquid crystal cell is used and the method B in which the imaging period is provided between the displays is used, the size and weight cannot be reduced, and there is a problem that the display flickers. The same thing can be said for a desktop type terminal, but the above-mentioned methods A and B are not preferable in terms of space saving because the size cannot be reduced.

The present invention has been made in view of the above problems, and its object is to reduce the thickness of the device itself by focusing on the above-mentioned panel type image sensor and adopting it for the image pickup function of the image input / output device. And an image input / output device in which the line of sight is natural and does not cause a feeling of strangeness when used as an interactive display device such as a TV phone without sacrificing the advantage of being thin. Especially.

[0011]

In order to solve the above-mentioned problems, an image input / output device according to a first aspect of the present invention comprises a panel-type display means in which pixels for display are arranged in a matrix. It is characterized in that it has a panel type image pickup means in which a plurality of light receiving elements for performing light detection are arranged in a matrix, and these display means and image pickup means are arranged so as to face each other.

In order to solve the above problems, an image input / output device according to a second aspect of the present invention is the image input / output device according to the first aspect, in which a substrate on which the pixels in the display means are arranged is arranged, It is characterized in that the substrate on which the light receiving elements in the image pickup means are arranged is the same substrate.

An image input / output device according to a third aspect of the present invention is, in order to solve the above-mentioned problems, in the image input / output device according to the second aspect, a pixel drive element for driving a pixel in the display means, It is characterized in that the light receiving element in the image pickup means is the same element.

According to a fourth aspect of the present invention, in order to solve the above-mentioned problems, an image input / output device according to the first, second or third aspect is provided.
In the image input / output device described above, a light guide unit is arranged on the front side of the image pickup unit, and the light guide unit guides light in parallel onto the light receiving element or has a condensing point. The condensing point is located on the back surface side or the front surface side of the light receiving element when viewed from the subject to be imaged.

According to a fifth aspect of the present invention, in order to solve the above problems, in the image input / output apparatus according to the fourth aspect, the light guide means is formed with a light receiving element in the image pickup means. It is characterized in that it is formed by configuring the substrate to be formed from a substrate having a light guiding function.

According to a sixth aspect of the present invention, there is provided an image input / output apparatus according to the fourth or fifth aspect, in order to solve the above problems, the front side of the light guide means or the light receiving unit. It is characterized in that light incident angle limiting means for regulating the incident angle of light incident on the light guide means or the light receiving element is arranged on the front side of the element.

[0017]

According to the structure of the present invention, the image pickup means makes the reflected light from the subject incident on the plurality of light receiving elements of the panel type arranged in a matrix, and the information of the subject is obtained by the compound eye of the insect. It is designed to be obtained like an image. Therefore, the thickness of this image pickup means is much thinner than the thickness of the image pickup means adopting the conventional monocular structure, and such a thin image pickup means is arranged to face the panel type display means. The image input / output device configured as described above is thin and has excellent portability.

Moreover, since the display area and the image pickup area are equal, when adopted in an interactive display device such as the above-mentioned TV phone or TV conference system, the caller can see the face of the other party displayed on the screen. Even if you make a call while watching,
The lines of sight of both parties are matched, and it becomes possible to talk with a natural line of sight.

According to the structure of claim 2, since the substrate on which the pixels in the display means are arranged and the substrate on which the light receiving elements in the image pickup means are arranged are the same substrate, each of them is a different substrate. The thickness of one substrate can be reduced as compared with the configuration. In the case of such display means and image pickup means, the thickness and weight of the substrate itself occupy a large proportion of the total thickness and weight. The output device can be made thinner, and the portability can be further improved.

According to the structure of claim 3, the pixel drive element of the display means and the light receiving element of the image pickup means are the same element. Therefore, in addition to the effect of the structure of claim 2, each element is a separate element. The number of manufacturing steps can be reduced as compared with the one formed from. Furthermore, since the area occupied by these elements can be reduced, the aperture ratio of the panel or the resolution can be improved.

According to the structure of claim 4, the light is guided in parallel on the light receiving element or has a condensing point on the front side of the imaging means, and this condensing point is from the object to be imaged. Since the light guide means on the back surface side or the front surface side of the light receiving element is arranged, the image pickup system of the image input / output device can be determined in addition to the operation according to the first, second, or third aspect of the invention. . That is, when the light guide means guides light in parallel onto the light receiving element, an equivalent imaging system (mirror) is used, and when the light converging point of the light guide means is on the back surface side of the light receiving element when viewed from the subject, a wide-angle lens is used. A wide-angle (reduction) imaging system using a lens and a telephoto (enlargement) imaging system using a telephoto lens can be respectively configured when the condensing point is on the surface side of the light receiving element.

According to the structure of claim 5, the light guide means is formed by forming the substrate on which the light receiving element in the image pickup means is formed from a substrate having a light guide function.
In addition to the effect of the configuration of claim 4, the number of components and the number of manufacturing steps can be reduced as compared with the configuration in which the light guide means and the substrate are separately provided.

According to the structure of claim 6, the light incident angle limiting means for restricting the incident angle of the light incident on the light guiding means or the light receiving element on the front side of the light guiding means or the front side of the light receiving element. Are arranged, the field of view of the light finally condensed on the light receiving element can be narrowed, in addition to the effect of the configuration of claim 4 or 5. As a result, the resolution of the captured image can be increased and the occurrence of blurring can be suppressed.

[0024]

【Example】

[Embodiment 1] The following will describe one embodiment of the present invention with reference to FIGS. 1 to 5. Here, an image input / output device using liquid crystal as a display means, particularly an active matrix type image input / output device will be described. However, the image input / output device according to the present invention is integrated with the active matrix type liquid crystal panel. The display device is not limited to the device, but may be another display device such as a PDP (Plasma).
It goes without saying that the present invention can also be applied to an image input / output device integrated with a display panel) or the like.

The image input / output apparatus 1 according to this embodiment is shown in FIG.
As shown in FIG. 3, the image capturing direction and the display direction are opposite to each other, and the person A on the side opposite to the screen 1a side is captured by the image capturing unit (not shown) provided in the image input / output device 1. It is displayed on the screen 1a of the display means and is used like a so-called ordinary "camera" so that the person B on the screen 1a side looks at it.

As shown in FIG. 1, the image input / output apparatus 1 includes an active matrix type liquid crystal panel 2 in which pixels for display are arranged in a matrix as a panel type display means, and a panel type display means. The image input / output unit 10 includes image pickup panels 6 each having an image pickup device formed in a matrix as an image pickup unit.

Liquid crystal panel 2 in image input / output unit 10
Is provided with translucent first and second substrates 101 and 102 made of, for example, glass or plastic. On the surface of the second substrate 102 facing the first substrate 101, a large number of TFTs, which are pixel driving elements, are provided together with a reflection electrode (not shown).
A pixel drive element array 104 in which (thin film transistors) are arranged in a matrix is formed, while a counter electrode 106 is formed on the surface of the first substrate 101 facing the second substrate 102. And both of these substrates 1
The liquid crystal layer 105 is sandwiched between 01 and 102,
Liquid crystal layer 1 between each pixel driving element and the counter substrate 106
Pixels are formed at 05.

On the other hand, the image pickup panel 6 is inspired by the above-mentioned panel type image sensor, and the image pickup means such as a conventional video camera has a monocular structure of vertebrate, whereas this image pickup panel 6 has a monocular structure. , Having an insect compound eye structure.

The image pickup panel 6 is provided with a translucent third substrate 103 made of, for example, glass or plastic,
An image sensor array 3 in which a plurality of image sensors are arranged in a matrix is formed on one surface. Although the details will be described later, the image pickup device includes a light receiving device.

Further, a light guide member (light guiding means) 4 for guiding the reflected light from the subject to the image pickup element array 3 is arranged on the front side of the image pickup element array 3. The light guide member 4
Is the presence or absence of the light condensing point, and the light condensing point is
The image pickup system of the image input / output device 1 is determined depending on the position of the image input / output device 1.

That is, in each drawing of FIG. 3, the reflected light (light for imaging) from the subject 7 is guided to the image sensor array 3 by the light guide member 4, and an image 7 ′ is formed on the image sensor array 3. As shown in FIG. 3A, the light guide member 4 is imaged.
When the condensing point P is on the back surface side of the image sensor array 3 when viewed from the subject 7, a wide-angle (reduction) imaging system using a wide-angle lens is used, as shown in FIG. When the reflected light does not exist and is guided to the image sensor array 3 in parallel, the equivalent image pickup system (mirror) is used, and as shown in FIG. If it is on the side, a telephoto (enlargement) image pickup system using a telephoto lens is configured respectively.

The light guide member 4 can be composed of, for example, a pinhole array. However, when the light guide member 4 is composed of a pinhole array, as shown in FIG. 4A, the light passing through the pinholes 8a in the pinhole array 8 is imaged in both the light coming from the a direction and the light coming from the b direction. An image is formed on the element array 3. Then, the field of view of each image formed on the image pickup element array 3 becomes large, and not only the light of a desired portion but also the light of other portions are incident on the image pickup element, which lowers the image resolution.
Alternatively, the image will be blurred. Therefore, a light incident angle limiting member (light incident angle limiting means) 5 that regulates the incident angle of the light imaged on the image sensor array 3 is provided (FIG. 1).
reference).

By providing such a light incident angle limiting member 5, as shown in FIG. 4B, the light which is about to enter the image pickup element array 3 is regulated by the light incident angle limiting member 5. As a result, the light traveling straight to the pinhole 8a passes through the pinhole 8a and forms an image on the image sensor array 3, but the light incident at a certain angle θ is absorbed by the side surface of the pinhole array 8 and imaged. No light reaches the element array 3 or light greatly attenuated reaches it.

The light incident angle limiting member 5 may be composed of a microlens array having a plurality of microlenses 9a respectively corresponding to the pinholes 8a of the pinhole array 8 as shown in FIG. it can. still,
Such a microlens may be a convex lens as shown or a concave lens.

With such a configuration, the image pickup panel 6 can form an image of a subject on an image pickup element corresponding to each eye of the compound eye to obtain an entire image of the subject.

In the case of the image input / output unit 10, as shown in FIG. 1, the liquid crystal panel 2 and the image pickup panel 6 are opposite to the pixel drive element array 104 side of the second substrate 102 in the liquid crystal panel 2. And a surface of the image pickup panel 6 opposite to the image pickup element array 3 side of the third substrate 103 are attached and integrated with each other via the adhesive layer 107. Further, the pixel drive element array 10 of the liquid crystal panel 4
4, the image pickup element array 3 of the image pickup panel 6, the connection wiring 1
09 and 109 are connected to the circuit boards 108 having these drive circuits, respectively.

Further, in the image input / output device 1, since the image pickup direction and the display direction are different, the image input / output unit 1 is
The individual image pickup device and the individual pixel drive device in 0 are
Each element may be formed on the entire screen and each element may be formed in the same area on the screen. Also, FIG.
In the figure, other components such as a color filter, a polarizing plate, a light shielding film (layer), a protective plate (sheet), etc. are not shown, but it goes without saying that these are added as necessary. This also applies to the following description of the image input / output unit.

Thus, the image input / output device 1 of this embodiment
In the above configuration, the image capturing means in the image input / output unit 10 is composed of the image pickup panel 6 and is arranged so as to face the flat plate-shaped liquid crystal panel 2. Therefore, an image including such an image input / output unit 10 is provided. The input / output device 1 is significantly thinner than the conventional image input / output device having a photographing means having a monocular structure, and portability can be improved.

In the configuration of the image input / output unit 10 described above, the image pickup light (reflected light from the object) incident on the image pickup element array 3 is transmitted through the light incident angle limiting member 5 and the light guide member 4. Since the light is directly incident on the image sensor array 3 through a transparent substrate, a clear image without blur can be obtained.

In the image input / output unit 10 of the image input / output device 1 of this embodiment, when the liquid crystal panel 2 and the image pickup panel 6 are attached to each other, the second substrate 102 and the third substrate 103 are provided with an adhesive layer. Although it was adhered via 107, as shown in FIG. 5, for example, a transparent substrate 110 for sandwiching the substrate is arranged between the second substrate 102 and the third substrate 103, and the substrate 110 is The image input / output unit 11 in which the second and third boards 102 and 103 are mounted may be used. Further, in this case, an image pickup control circuit for driving the image pickup element array 3 on the substrate 110 and / or
Alternatively, a circuit for display control which drives the pixel driving element array 104 may be formed.

[Second Embodiment] The following will describe another embodiment of the present invention with reference to FIG.

The image input / output apparatus according to the present embodiment has a structure in which the image pickup direction and the display direction are opposite to each other, as in the image input / output apparatus 1 shown in FIG. The image input / output unit 12 shown in FIG. 6 is provided.

The image input / output unit 12 has a structure in which the third substrate 103 of the image pickup panel 6 shown in FIG. 1 and the second substrate 102 of the liquid crystal panel 2 are commonly used. In the output unit 10, the imaging element array 3 includes the second substrate 10 on which the pixel driving element array 104 is formed.
Although it was formed on the third substrate 103 different from 2,
The image pickup element array 3 is formed on the surface of the second substrate 102 opposite to the pixel drive element array 104 side. With such a configuration, the thickness of the image input / output unit 12 becomes thinner and lighter by the reduction of one substrate that influences the thickness and weight of the entire image input / output unit. It is possible to make the device thinner and lighter than the image forming apparatus of No. 1.

[Embodiment 3] Another embodiment of the present invention will be described below with reference to FIGS. 7 to 26.

The image input / output device according to the present embodiment has a structure in which the image pickup direction and the display direction are opposite to each other, as in the image input / output device 1 shown in FIG. The image input / output unit 13 shown in FIG. 7 is provided.

As shown in FIG. 7, the image input / output unit 13 includes
In the image input / output unit 12 of the second embodiment, the image pickup device array 3 is the pixel drive device array 10 on the second substrate 102.
The image pickup element array 3 is formed on the same surface as the pixel drive element array 104, while the image pickup element array 3 is formed on the surface opposite to the 4th surface. In this case, the image sensor array 3
In order to suppress the influence of the refraction index and the like that the image pickup light incident on the second substrate 102 transmits, the light guide member 4 and the light incident angle limiting member 5 are also included in the image pickup element array 3.
It is preferable to provide the same side. With such a configuration, like the image forming apparatus of the second embodiment, the image input / output apparatus 1 of the first embodiment can be made thinner and lighter, and the same surface as the pixel driving element array 104 can be obtained. Since the image pickup element array 3 is formed in the above, there is an advantage that the manufacturing process can be simplified.

Although the image pickup element array 3 and the pixel drive element array 104 appear to have a laminated structure in the drawing, in reality, the image pickup element and the pixel drive element are the second one.
It is formed on the surface of the substrate 102. Further, when the image pickup element array 3 and the pixel drive element array 104 are formed on the same substrate as described above, they may be formed separately or may be an image pickup element / pixel drive element array. The image input / output unit 13 of this embodiment is formed like the former. The latter case has the advantage that the number of manufacturing steps can be further reduced.

In the image input / output unit 13, the transparent second substrate 102 is used as the substrate on which the image pickup element array 3 and the pixel drive element array 104 are formed.
As shown in, the light guide member 4 is also provided with a function.
For example, using the fiber substrate 111, the image input / output unit 14
It is possible to reduce the number of members and the manufacturing process. Further, when such a substrate 111 having a light guiding property is used, the light incident angle limiting member 5 may be provided on the side opposite to the image sensor array 3 instead of being provided on the image sensor array 3 side.

In each of the image input / output units 10 to 14 in the above-described first to third embodiments, the reflection type liquid crystal panel 2 is used.
Although the image input / output unit having the above is illustrated, a transmissive liquid crystal panel may be used. When a transmissive liquid crystal panel is used, basically, a substrate having pixels (that is, the pixel driving element array 1
04 or a substrate on which the counter electrode 106 is formed),
When the substrate having the image pickup device (that is, the substrate on which the image pickup device array 3 is formed) is separate, it is necessary to dispose the display light source between the substrates, but the substrate having pixels and the substrate having the image pickup device It is also possible to use external light as a light source for display by providing a scattering layer between and. This scattering layer may be, for example, a polymer film or PDLC (Po
lymer Dispersed Liquid Crystal).

When the liquid crystal panel and the image pickup panel 6 share a substrate and the pixel driving element array 104 and the image pickup element array 3 are formed on the same substrate, (1) the display light source and the image pickup light are used. Or separate the path that is taken by (2)
A method of dividing the display and the imaging period by time division can be considered. In the latter case, for example, on the back surface of the display unit, a light scattering layer capable of controlling the light scattering state and the transmission state from the outside, such as a PD
It can be realized by providing a layer made of LC.

Next, the structure of the image input / output unit 13 included in the image input / output apparatus of this embodiment will be described in more detail.

As described above, in the case of the image input / output unit 13, the image pickup element and the pixel driving element are formed on the same substrate. The relationship between the light receiving direction of the image sensor and the image capturing direction is roughly classified into the same direction (direct input) and the opposite direction (reflection input). In the opposite case, separately from the display.
A light transmission direction control means dedicated to the image pickup device, for example, a reflection plate may be formed. However, in the case of a reflection type such as the liquid crystal panel 2 included in the image input / output unit 13, a reflection plate for display and an image pickup device The image input / output unit 13 uses the surface of the reflective electrode opposite to the display side to perform imaging (for the reflective liquid crystal display device, See Japanese Patent Application No. 4-331810).

As shown in FIG. 9, a light transmissive substrate 20.
The imaging element 3a is formed on the same surface as the pixel driving element 104a on the second substrate 102 (corresponding to the second substrate 102 in FIG. 7). The light transmitted through the light guide member (not shown) is reflected by the reflection electrode 2.
It is reflected downward at 1 and input to the image sensor 3a. still,
The reflection surface of the reflection electrode 21 does not have to be an irregular reflection surface as shown in FIG. 9 and may be a flat surface as shown in FIG. 10A, but it is preferably completely horizontal. It is preferable to use a structure having a constant inclined surface shown in FIG. 6B so that light is incident on the image sensor 3a or a wavefront structure shown in FIG. Furthermore, when viewed from the top of the wavefront structure, as shown in FIG.
It may be a ripple-like structure spreading from the point to the surroundings,
Furthermore, the surface of the reflective electrode 21 opposite to the display side may form a kind of lens, a so-called reflecting mirror.

Further, the pixel driving element 104a and the image pickup element 3
As shown in FIG. 11, a may be shared by the same pixel driving element / light receiving element 22. When the pixel driving element / light receiving element 22 is used, the number of manufacturing steps can be reduced and the area occupied by these elements can be reduced as compared with the case where each element is formed from another element. There is an advantage that the improvement or the resolution can be improved.

Note that the above-mentioned points of attention of the image input / output units 14 can be applied to the image input / output unit 13 described here as needed. The same effect as described above can be expected. Further, it goes without saying that the present invention can be applied not only to a monochrome image but also to a color image.

Next, the image pickup device 3 in the image input / output unit 13
A specific configuration for capturing an image using a and an example of transmitting the captured image will be described.

In the pixel portion, when the pixel driving element 104a and the image pickup element 3a are formed, they are formed as separate elements, or they are the same element, or at least a part of the pixel drive element 104a and the image pickup element 3a are formed. It has already been described above that the case of sharing, that is, two cases of forming the pixel driving element / imaging element 22 can be considered.

In the former case, as shown in FIG. 12A, the pixel driving element 104a and the image pickup element 3a are separately provided in the pixel section 30 as separate elements, or as shown in FIG. For example, switch means 31 provided if necessary
And the image input / output unit 13
Has the configuration shown in FIG.

On the other hand, in the latter case, the pixel driving element 104
As shown in FIG. 13A, the configuration in which the image sensor 3a is shared with at least a part of a is, for example, information storage means 32 that temporarily (temporarily) stores information on a captured image.
Is required. The information storage means 32 may share, for example, the LC capacitance or the auxiliary capacitance provided to the pixel driving element 104a. Further, when the pixel driving element / imaging element 22 is used, as shown in FIG.
The pixel transistor TR ₁ may also serve as an image sensor.
Incidentally, FIG. 7C shows a structural example of the pixel transistor TR ₁ having the image pickup function shown in FIG.

Next, regarding the structure of the signal lines for transmitting the display scanning signal, the display data signal, the image scanning signal, and the image data signal in the above-mentioned pixel portion 30, as shown in FIG. It is conceivable that the image scanning signal line 41 and the image data signal line 42 used for the transfer to the outside of the pixel display section are separately provided. Reference numeral 43 in FIG. 14 denotes a display scanning signal line,
Reference numeral 44 indicates a display data signal line. In addition to this, FIG.
As shown in FIG. 5, a configuration in which a display / image data signal line 45 and a display / image scanning signal line 46 that share both display and image are provided is conceivable, and these may be used in combination. Incidentally, as shown in FIG.
When each signal line is provided in common, it is necessary to provide a switching means for switching each signal line as required, and a switching for switching between the display data signal sent to the display / image data signal line 45 and the image data signal. Means SW ₁ are provided.

The combination of the pixel drive element 104a and the image pickup element 3a and the combination of the scanning line / data line described above is arbitrary, and these combinations may be changed according to need. A concrete example of this is shown in FIG. In the figure, the display scanning signal line 43 and the image scanning signal line 41 are provided independently, and the display data signal line and the image data signal line are both display and image data signal line 4.
5 shows the case of sharing. In the figure, the pixel driving element 104a is composed of a pixel transistor TR ₂ and includes a liquid crystal LC and an auxiliary capacitance C _S added as needed.
It has. On the other hand, the image pickup element 3a includes a light receiving element OPT.
And a load capacitance R ₁ and a hold capacitance C _H for accumulating image data. The image input / output unit 13 employs this configuration.

Here, the image data is directly (including the case where the image data is transferred via another constituent element, for example, the pixel transistor TR ₂ ) as an image data signal (display / image data signal line). ), The hold capacitance C _H can be omitted.

On the other hand, the configuration of the light receiving element OPT is arbitrary, and a phototransistor is used as shown in FIG. 17A, a photodiode or a photodiode as shown in FIG.
As shown in FIG. 7C, a solar cell or the like may be used as shown in FIG. 7C, or a laminated structure in which several layers of each element are stacked may be used. Further, not only one type of light receiving element OPT but also several types of light receiving element OPT may be used in combination. As described above, in one image input / output device, when a plurality of types of light receiving elements OPT are used in combination, the light receiving characteristics due to the wavelength or the intensity of light can be corrected.
The bias means shown in (a) to (c) of FIGS. 16 and 17 may be added as needed (depending on the element structure).

Next, the image input / output unit 13 (see FIG.
6 has a configuration of 6),
This will be described with reference to FIGS. 18 and 19. FIG. 18 shows a method of collectively writing an image and transferring image data after the end of the display period. FIG. 19 shows a method of writing an image and image data before refreshing the display data for each scanning signal line. The method of performing the transfer of is shown.

In FIG. 18, during the display period (display data writing period), for example, from the first row of the scanning signal line,
.. The mth line, ..., The nth line and the gate of the pixel transistor TR ₂ are turned on (see (a) and (b)), and the data of the display / image data signal line (source bus line) is transferred to the pixel unit 30.
Write (see (e)). After the end of the display period (display data writing period), the source bus line potential is once refreshed to a constant potential, and the pixel transistors TR ₂ connected to the respective display scan signal lines are simultaneously turned on to bring the constant potential to each pixel section. Write to 30 (LC capacity, auxiliary capacity). It should be noted that this refresh is performed so that the picked-up image data does not fluctuate due to the charges charged in the pixel section 30 or the bus line. In the subsequent image data transfer period, the images on the material are picked up in order from the first row, and the image data is transferred via the data bus line. That is, from the first line ...
The pixel transistor TR ₂ forming the same pixel is turned on in synchronization with the m-th row, the n-th row and the turning-on of the transistor TR ₃ (see (a) and (b)).

The pixel transistor TR ₂ connected to each scanning signal line is turned on in synchronization with the transistor TR _3.
However, this may take another operation timing, for example, the pixel transistors TR ₂ may be turned on all at once during a certain period of the image data transfer period. On the contrary, all transistors TR ₃
Are turned on all at once, the image data is once taken in to the pixel display section, and then the pixel transistors TR ₂ connected to the respective source bus lines are turned on sequentially for each scanning signal line,
The data stored in the pixel display unit may be sequentially transferred via the source bus line. In the case where the pixel unit 30 is configured to pick up an image once, it is necessary to pay attention to the amplification factor / NF of the amplifier provided for data reception, as compared with the case of directly writing the image data to the bus line. On the other hand, there is an advantage that the charge generated by the light receiving element OPT is at least good.

Next, a method of transferring the image data not through the transfer of the image data all at once but immediately before refreshing the respective display data through the bus line for each scanning signal line will be described with reference to FIG. . According to this method, it is possible to suppress the flickering of the display, which is expected to occur when image data is collectively transferred, and further, the interruption of the display. In FIG. 19, the pixel transistor T of each line
The R ₂ gate ON period includes a reset period, an image data transfer period, and a display data writing period. After the display period of a certain field ends, the display data is newly added to the pixel unit 30.
Before writing to the
The capacitors, storage capacitors, etc.) are once refreshed to the reference potential (reset period), and then image data is picked up and transferred (image data transfer period) (see (a) and (b)). In the image data transfer period, the transistors TR ₃ connected to the same pixel are turned on in synchronization (see (d) and (e)). After the end of the image data transfer period, the display data writing period starts, and the bus line data is written in the pixel capacitance (see (c)).

Data may be fetched from the light receiving element OPT to the data storage means C _{H at} all times, but if necessary, the period may be limited and the data may be transferred to the data signal line in a synchronous relationship. For example, they may be performed at the same time. In such a case, for example, as shown in FIG. 20, a transistor TR ₄ may be further provided as a switch means in the light receiving element OPT which is a light receiving means to control the reading of the image. Particularly, when the light emitting element OPT is formed for each pixel unit 30, the light emitting element OPT may be sequentially turned on corresponding to the pixel for reading the image data, or may be randomly turned on in order to prevent interference by the adjacent light emitting element OPT. Good.
It goes without saying that the reading of the image data and the transfer of the image data do not have to be completed within one field period and may be performed over a plurality of fields.

Next, the system configuration of the image input / output device having the image input / output unit 13 described above will be described. FIG. 21 is a block diagram showing the system configuration of the image input / output device. When the image input / output device operates as an image output device, that is, a pixel display device, in FIG.
The switch SW ₂ is tilted to the a side, the display data is transferred from the display data transfer driver 52 to the image display unit 50, and the display scan driver 53 sets the display scan driver output for the display data storage pixel row to the active state and the data signal line Data is stored in the pixel unit 30.

On the other hand, when the image input / output device operates as the image input device, the image scanning driver 54 is turned on sequentially or collectively according to the configuration to read the image, or simultaneously with the reading, display is performed if necessary. When transferring the read data, the switch SW ₂ is tilted to the b side, and the image data amplification circuit 55 provided as necessary amplifies the image data signal,
The amplified signal is transferred to the image data processing circuit 56 composed of a CPU / storage means or the like. Of the signals processed by the image data processing circuit 56, the signals used for display are transferred to the display data transfer driver,
It is sent to the image display unit 50 as a display data signal.

The configuration shown in FIG. 21 is merely an example, and the display data transfer driver 5 is provided if another configuration, for example, circuit conditions such as element breakdown voltage as shown in FIG.
2 and the image data amplifier circuit 55 may be directly connected without a switch SW ₂ (via a bus line if necessary), and the display scan driver 53 and the image scan driver 54 (via a bus line if necessary). You may connect directly. In addition, when the image scanning signal line and the display scanning signal line are used in common, the image scanning driver 54 and the display scanning driver 53 may be shared. Further, the image data amplification means 55 may be provided in the image display section 50.

By the way, since the light receiving level varies depending on the image pickup condition, for example, the brightness of the ambient light, it may be necessary to correct the pickup data according to the external light. FIG. 23 shows an example thereof. That is, the external light level detection means 57 is provided in the image display section 50 or its periphery, and the detection signal is input to the signal processing circuit 58 composed of the CPU storage means or the like as indicated by, and the data is converted based on a predetermined correction coefficient. Make a correction. Alternatively, as shown in (1), the output of the external light level detecting means 57 is input to the image data amplifying circuit 55, and the amplification degree of the circuit 55 is changed to correct the image data. Further, it is also possible to correct the level of the image data while confirming the display data by a manual operation without automatically judging the outside light. This is also effective in other configurations in which outside light is not used as the image pickup light.

Further, due to the problem of the response speed of the light receiving element OPT, the time for picking up one picture may be controlled from the outside when the light receiving level cannot be sufficiently obtained in a predetermined time.

In addition, when so-called blurring occurs in the picked-up image due to the problem of the light receiving optical system, the picked-up image data is corrected by comparing the picked-up image data with, for example, a library provided in the apparatus itself. Good. This kind of method is particularly effective when a character, a document, or the like is photographed.

In each of the above explanations, the ratio of providing the image pickup device 3a and the pixel driving device 104a is set to 1: 1. However, the present invention is not limited to this, and one pixel unit 30 is provided as shown in FIG. A plurality of imaging elements 3a, preferably an odd number of imaging elements 3a may be provided for one pixel driving element 104a. In this way, a plurality of image pickup devices 3
One advantage of providing a is that the occurrence of errors in pickup image data (imaging data) is suppressed. That is, as an example, as shown in FIG. 25A, the logic determination can be performed using the majority logic. In other words, if one of the three light receiving elements 3a in the same pixel is
o. No. 1 and No. 1 No. 2, the rest 3, and when the pickup image data detected by each light receiving element 3a is A, A, B, the larger one is determined as A.

Further, as shown in FIG. 7B, although it takes time to read one image, one pixel is read several times, in this example, three times, and the logic of majority decision is used. It is also possible to make a judgment. The latter method is
There is an advantage that it is possible to suppress a decrease in aperture ratio. Although the majority logic is used here as a method for suppressing the occurrence of an error, another method such as a method using an average value may be used. Moreover, the judgment described above is
It may be performed in the pixel area, or the image data processing circuit 5 including the CPU storage means shown in FIGS.
6 or in the signal processing circuit 58.

[Fourth Embodiment] FIG. 26 shows another embodiment of the present invention.
The following is a description with reference to FIG.
For convenience of explanation, members having the same functions as those shown in the first to third embodiments are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIGS. 26 (a) and 26 (b), in the image input / output device 60 according to the present embodiment, the image pickup direction and the display direction are mutually different in the case of the image input / output devices of the first to third embodiments. In contrast to the opposite direction, the imaging direction and the display direction are the same. Such an image input / output device 60 is
For example, as shown in (a) of the figure, it is applied to a TV telephone system or the like between a plurality of image input / output devices 60. That is,
The person A looking at the screen 60a of the image input / output device 60 is
An image is picked up by an image pickup means (not shown) of the image input / output device 60, and this video signal data is transferred to the image input / output device 60 ′ viewed by the person B who is an interlocutor, and the screen 60a ′ viewed by the person B is displayed. While the person B is also displayed on the screen 60a of the image input / output apparatus 60 of the person A, the person B is also imaged by an image pickup unit (not shown) provided in the image input / output apparatus 60 ′.

Further, as shown in FIG. 7B, the person A himself looking at the screen 60a of the image input / output device 60 is imaged by an image pickup means (not shown) provided in the image input / output device 60. As it is displayed on the screen 60a,
It is also used in a "mirror" state.

The image input / output unit mounted in the image forming apparatus 60 of the present embodiment has a basic configuration of each image input / output unit shown in FIGS. 1 and 6 to 9 of the above-described Embodiments 1 to 3. 10-1
4 is the same as that of the fourth embodiment, but in the case of a reflection type liquid crystal panel, as a matter of course, as shown in the figure, the reflection electrode (reflection plate) is the image input / output of the first embodiment shown in FIG. Part 10
Although it is provided on the second substrate 102 side, it is provided on the first substrate 101 opposite to the display side.

On the other hand, in the case of a transmissive liquid crystal panel, there is no such limitation. In addition, even if the image pickup element array 3 and the pixel drive element array 104 are provided on the same substrate,
It is not necessary to pay particular attention to the points (1) or (2) described above. However, as in the above embodiments,
When the image pickup direction and the display direction are opposite to each other, even if the image pickup device 3a of the image pickup device array 3 and the pixel drive device 104a of the pixel drive device array 104 are formed from separate elements and are arranged on the entire screen, nothing happens. Although there is no problem, in a device having the same image pickup direction and display direction as in the image input / output device 60 of the present embodiment, (3) the area for forming the pixel drive element array 104 and the image pickup element array 3 are arranged. It should be noted that the area to be formed is divided and arranged, or (4) the same element is used in time division.

The pixel drive element array 10 of (3) above.
The configuration in which the region forming 4 and the region forming the image sensor array 3 are divided and arranged is as already described in the third embodiment.

According to this, for example, when it is adopted in the interactive display device such as the above-mentioned TV telephone or TV conference system,
Even if the caller makes a call while looking at the face of the other party displayed on the screen 60a, the lines of sight of both parties are matched, and the call can be made with a natural line of sight. As a result, it is possible to obtain an image forming apparatus having a thin and excellent portability and a natural line of sight, and in the future, with the progress of the information society, a dramatic effect in the field of interactive display devices. Can bring.

When used as a "mirror", a true mirror-like image is obtained.

[0085]

As described above, the image input / output device according to the first aspect of the present invention has a panel type display means in which pixels for display are arranged in a matrix and a plurality of light receiving elements for detecting light. And a panel type image pickup means arranged in a matrix, and the display means and the image pickup means are arranged to face each other.

As a result, the thickness of the image pickup means becomes much thinner than the thickness of the image pickup means adopting the conventional monocular structure, and such a thin image pickup means faces the panel type display means. The image input / output device configured by arranging the components is thin and excellent in portability. Moreover, since the display area and the imaging area are the same, when adopted in an interactive display device such as the above-mentioned TV telephone or TV conference system, the caller can talk while looking at the face of the other party displayed on the screen. Even if the user makes a call, the lines of sight of both parties are matched, and the call can be made with a natural line of sight. As a result, it is possible to obtain an image forming apparatus having a natural line of sight and excellent portability.

As described above, the image input / output device according to claim 2 of the present invention is the image input / output device according to claim 1, in which the substrate on which the pixels of the display means are arranged and the image pickup means are arranged. The substrate on which the light receiving elements are arranged is the same substrate.

As a result, in addition to the effect of the structure of claim 1, the image input / output device can be made thinner, and the portability can be further improved.

As described above, the image input / output device according to claim 3 of the present invention is the image input / output device according to claim 2, in which the pixel drive element for driving the pixel in the display means and the image pickup means are provided. The light receiving element and the light receiving element are the same element.

As a result, in addition to the effect of the structure of the above-mentioned claim 2, as compared with the case where each is formed from another element,
Since the number of manufacturing steps can be reduced and the area occupied by these elements can be reduced, the aperture ratio of the panel or the resolution can be improved.

As described above, the image input / output device according to claim 4 of the present invention is the image input / output device according to claim 1, 2 or 3, in which the light guide means is arranged on the front side of the image pickup means. This light guiding means guides light in parallel onto the light receiving element, or has a converging point, and this condensing point is viewed from the subject to be imaged, on the back surface side of the light receiving element, or It is a structure on the front side.

Thus, in addition to the effects of the above-described structure of claim 1, 2 or 3, the image input / output devices of the equivalent image pickup system (mirror), the wide-angle (reduction) image pickup system, and the telephoto (enlargement) image pickup system are obtained. There is an effect that can be.

As described above, the image input / output device according to a fifth aspect of the present invention is the image input / output device according to the fourth aspect, wherein the light guide means is a substrate on which a light receiving element in the image pickup means is formed. This is a structure formed by being composed of a substrate having a light guiding function.

As a result, in addition to the effect of the structure of claim 4, the number of parts and the number of manufacturing steps can be reduced as compared with the structure in which the light guide means and the substrate are separately provided.

As described above, the image input / output device according to a sixth aspect of the present invention is the image input / output device according to the fourth or fifth aspect, wherein the image input / output device is provided on the front surface of the light guide means or the front surface side of the light receiving element. The light incident angle limiting means for controlling the incident angle of the light entering the light guiding means or the light receiving element is arranged.

As a result, in addition to the effects of the above-mentioned structure of claim 4 or 5, the field of view of the light finally condensed on the light receiving element can be narrowed, the resolution of the picked-up image can be increased, and the blurring can occur. This has the effect of suppressing the occurrence of

[Brief description of drawings]

FIG. 1 is a cross-sectional view of an image input / output unit included in an image input / output device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an image pickup direction and a display direction in the image input / output device, and an image displayed on a screen.

3A to 3C show that light for image pickup is guided to an image pickup element array by a light guide member provided in the image input / output device, and an image is formed on the image pickup element array. FIG. 4A is an explanatory diagram showing a wide angle (reduction) when the light condensing point of the light guide member is on the back surface side of the image sensor array when viewed from the subject.
FIG. 2B shows an equivalent image pickup system (mirror) when the reflected light is guided in parallel to the image pickup element array without the light collection point, and FIG. Shows a telephoto (enlarged) image pickup system when is on the front surface side of the image pickup element array.

FIG. 4A is an explanatory diagram showing a direction of light passing through the pinhole when the light of an image pickup from a subject directly enters the pinhole when the light guide member is composed of a pinhole array; (B) is explanatory drawing which shows the direction of the light which passes the light-incidence angle limiting member arrange | positioned at the light-incidence side of a pinhole array, (c) shows the microlens array (FIG. Is an explanatory view showing a direction of light passing through a pinhole when a microlens is arranged.

FIG. 5 shows another embodiment of the present invention and is a cross-sectional view of an image input / output unit provided in the image input / output device.

FIG. 6 shows another embodiment of the present invention and is a cross-sectional view of an image input / output unit provided in the image input / output device.

FIG. 7 shows another embodiment of the present invention and is a cross-sectional view of an image input / output unit provided in the image input / output device.

FIG. 8 shows another embodiment of the present invention and is a cross-sectional view of an image input / output unit provided in the image input / output device.

9 is an explanatory diagram showing a configuration of a main part of the image input / output unit shown in FIG. 7, showing a configuration in which a reflective electrode for display and a reflective electrode for an image sensor are shared in a reflective liquid crystal panel. ing.

10 (a) to 10 (d) are explanatory views showing another structure of the reflective electrode shown in FIG. 9, (a) being flat in the horizontal direction, and (b) having a constant inclined surface. (C) shows one having a wave-like structure, and (d) shows one having a ripple-like structure spreading from one point to the periphery.

FIG. 11 shows a case where a pixel driving element and a light receiving element in which a reflective electrode for display and a reflective electrode for an image pickup element are commonly used in a reflective liquid crystal panel and which has functions of a pixel drive element and an image pickup element are used. It is explanatory drawing which shows the structure comprised.

12A is a schematic diagram showing a configuration of a main part of the image input / output unit shown in FIG. 7, showing a configuration in one pixel in which a pixel driving element and an imaging element are independently provided in each pixel. Shows,
(B) is a schematic diagram showing another configuration in one pixel in which an image pickup device and a pixel drive device are connected via a switch means.

13A and 13B are schematic diagrams showing a configuration of one pixel of the pixel driving element / light receiving element shown in FIG. 11, and FIG. 13A shows at least a part of the pixel driving element and an imaging element. The configuration in the case of sharing is shown, (b) shows the configuration in which the image pickup element is also provided in the pixel drive element, and (c) shows the structural example of the pixel drive element having the image pickup function shown in (b). FIG.

FIG. 14 is a schematic diagram showing a configuration in which an image scanning signal line and an image data signal line, and a display scanning signal line and a display data signal line are separately provided.

FIG. 15 is a schematic diagram showing a configuration in which a display / image data signal line and a display / image scanning signal line for both display and image are provided.

16 illustrates a configuration of a main part of the image input / output unit illustrated in FIG. 7, in which a display scanning signal line and an image scanning signal line are independently provided, and the display data signal line and the image data signal line serve as a display. FIG. 6 is a circuit diagram showing a configuration shared by image data signal lines.

17 (a) to (c) are circuit diagrams showing specific examples of a light receiving element which is an image pickup element, (a) using a phototransistor, (b) using a photodiode, (C) shows one using a solar cell or the like.

18 is a timing chart showing an operation example in the image input / output unit shown in FIG. 7, showing a method of collectively writing an image and transferring image data after the display period ends.

19 is a timing chart showing another operation example in the image input / output unit shown in FIG. 7, showing a method of writing an image and transferring the image data before refreshing the display data for each scanning signal line. Shows.

FIG. 20 is a circuit diagram of a pixel portion in the case where data is taken in from the light receiving element OPT to the data storage means C _H at the same time as a period in which data is transferred to the data signal line by limiting the period as necessary. .

21 is a block diagram showing a system configuration of an image forming apparatus including the image input / output unit 13 of FIG.

FIG. 22 is a block diagram showing another system configuration of the image input / output device.

FIG. 23 is a block diagram showing another system configuration of the image input / output device.

FIG. 24 is an explanatory diagram showing another configuration of the pixel portion and showing a configuration in which a plurality of light receiving elements are provided for one pixel driving element in one pixel.

25 (a) and 25 (b) are explanatory views showing a logic determination method performed when a plurality of light receiving elements are provided as in FIG.

26 (a) and 26 (b) are explanatory views showing an image pickup direction and a display direction of an image input / output device according to another embodiment of the present invention, and an image displayed on a screen.

27 is a cross-sectional view of an image input / output unit provided in the image input / output device shown in FIG.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image input / output device 2 Liquid crystal panel (display means) 3 Imaging element array 3a Imaging element (light receiving element) 4 Light guide member (light guiding means) 5 Light incident angle limiting member (light incident angle limiting means) 8 Pinhole array ( Light guide means) 10 image input / output unit 11 image input / output unit 12 image input / output unit 13 image input / output unit 14 image input / output unit 30 pixel unit 60 image input / output device 61 image input / output unit 101 first substrate 102 second substrate 103 third substrate 104 pixel driving element array 104a pixel driving element 106 counter electrode

Claims (6)

[Claims] 1. A panel type display means in which pixels for display are arranged in a matrix, and a panel type image pickup means in which a plurality of light receiving elements for detecting light are arranged in a matrix. An image input / output device, wherein the image pickup unit and the image pickup unit are arranged to face each other. 2. The image input according to claim 1, wherein the substrate on which the pixels in the display means are arranged and the substrate on which the light receiving elements in the image pickup means are arranged are the same substrate. Output device. 3. The image input / output device according to claim 2, wherein the pixel drive element for driving the pixel in the display means and the light receiving element in the image pickup means are the same element. 4. A light guide means is disposed on the front side of the image pickup means, and the light guide means guides light in parallel on a light receiving element or has a condensing point. 4. The image input / output device according to claim 1, wherein the condensing point is on the back surface side or the front surface side of the light receiving element when viewed from the object to be imaged. 5. The light guide unit is formed by forming a substrate on which a light receiving element in the image pickup unit is formed from a substrate having a light guide function.
The image input / output device described. 6. A light incident angle limiting means for restricting an incident angle of light incident on the light guiding means or the light receiving element is arranged on the front surface side of the light guiding means or the front surface side of the light receiving element. The image input / output device according to claim 4 or 5, characterized in that