JP2008109362A - Camera server, and processing method for data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce disorder of a picture displayed on a client side by distributing image data having small noise to the client. <P>SOLUTION: A camera server 3 which distributes image data of an image picked up by a camera device 2 having an imaging element (image sensor 25) to the client 5 through a network 6 has a data selecting means 38 of judging whether or not the image data are predetermined image data and outputting the image data judged to be the predetermined image data to the side of the network 6 and not outputting the data to the side of the network 6 otherwise. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a camera server and a data processing method.

  2. Description of the Related Art Conventionally, a camera server for distributing image data of an image captured by a camera device including an image sensor to a client via a network is known (for example, see Patent Document 1). Such a camera server is configured to distribute all data signals output from the camera device to the client side.

JP-A-2005-318412

  However, a data signal output from the camera device may include a noise data signal that is not an image data data signal. Therefore, the client distributes not only image data but also a noise data signal. Therefore, when displaying a video based on the data transmitted from the camera server on the client side, there is a problem that the video is disturbed by a noise data signal.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a camera server and a data processing method that distribute image data with less noise to a client and reduce disturbance of video displayed on the client side.

  In order to solve the above-described problem, a camera server that distributes image data of an image captured by a camera device including an image sensor to a client via a network determines whether the image data is predetermined image data. Data selection for image data that is determined to be predetermined image data is output to the network side, and data that is determined that the image data is not predetermined image data is not output to the network side Means shall be provided.

  In such a configuration, image data with less noise can be distributed to the client, so that the client can obtain image data with less noise.

  According to another invention, in addition to the above-described invention, the data selecting means determines whether or not the image data is predetermined image data based on information of a header added to the image data.

  When configured in this way, it is possible to easily determine whether the image data is predetermined image data.

  In another invention, in addition to the above-described invention, the camera server synchronizes the image data and the sound data based on the header information of the image data and the sound data when the sound data is input in addition to the image data. The data synchronization means synchronizes the predetermined sound data of the sound data with the data portion that is determined not to be the predetermined image data by the data selection means.

  In such a configuration, image data with less noise can be distributed to the client, so that the client can obtain image data with less noise. As for the sound data, sound data synchronized with the image data that has not been distributed is also distributed. Therefore, the client can listen to the collected sound without interruption.

  In order to solve the above-described problem, in a data processing method for distributing image data of an image captured by a camera device including an image sensor to a client via a network, whether or not the image data is predetermined image data. If the image data is determined to be predetermined image data, the image data is output to the network side. If the image data is determined not to be the predetermined image data, the image data is not output to the network side. I will do it.

  In such a configuration, image data with less noise can be distributed to the client, so that the client can obtain image data with less noise.

  According to another invention, in addition to the above-described invention, whether or not the image data is predetermined image data is determined based on header information added to the image data.

  When configured in this way, it is possible to easily determine whether the image data is predetermined image data.

  In order to solve the above-described problem, in a data processing method for distributing image data of an image captured by a camera device including an image sensor and sound data of sound collected by a microphone to a client via a network, the image data Determines whether the image data is predetermined image data. The image data determined to be the predetermined image data is output to the network side, and the image data is not the predetermined image data. The determined data is processed so as not to be output to the network side, and the sound data and the image data output to the network side are synchronized with each other based on header information added to each data, The specified sound data of the sound data is also synchronized with the data portion determined not to be image data. To be.

  In such a configuration, image data with less noise can be distributed to the client, so that the client can obtain image data with less noise. As for the sound data, sound data synchronized with the image data that has not been distributed is also distributed. Therefore, the client can listen to the collected sound without interruption.

  According to the present invention, it is possible to distribute image data with less noise to a client, and to reduce disturbance of video displayed on the client side.

  Hereinafter, a camera server and a data processing method according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a system configuration diagram showing a camera server system 1 according to an embodiment of the present invention. The camera server system 1 includes a camera device 2 and a camera server 3. The camera device 2 and the camera server 3 are connected by a USB (Universal Serial Bus) cable 4. The camera server system 1 and the client 5 are connected via the network 6, and the camera server system 1 receives a command from the personal computer 7 on the client 5 side. The camera server 3 distributes image data of an image captured by the camera device 2 to the personal computer 7 (client 5) based on a command received from the personal computer 7. One or a plurality of clients 5 exist, and the camera server system 1 distributes the image data to each client 5 according to a request from each client 5.

  The camera device 2 includes a cubic housing 8. The housing 8 is provided with a fish-eye lens 9, a USB connector 10, an audio / video output connector 11, a sound output connector 12, and the like. The fisheye lens 9 is disposed on the upper surface of the housing 8. In addition to the fisheye lens 9, a vent hole 14 for the microphone 13 (see FIG. 2) is formed on the upper surface of the housing 8. The USB connector 10, the audio / video output connector 11, and the sound output connector 12 are disposed on the side surface of the housing 8.

  The camera server 3 has a rectangular parallelepiped housing 15. The housing 15 has an external connector such as a USB connector 16 serving as an input unit for inputting data from the camera device 2, a microphone connector 18 to which an expansion microphone 17 that is appropriately used as necessary is connected, and an infrared sensor. General-purpose input / output port (GPIO) 19 for connecting devices, an audio output connector 20 for outputting an audio signal, a video output connector 21 for outputting a video signal, a speaker connector 22 to which a speaker is connected, a network A communication interface 23 connected to the power supply switch 24, a power switch 24 also serving as a reset switch, and the like are provided.

  FIG. 2 is a block diagram showing an electric circuit of the camera device 2.

  In addition to the fisheye lens 9, USB connector 10, audio / video output connector 11, and sound output connector 12, the camera device 2 includes a microphone 13, an image sensor 25 as an image sensor, and an FPGA (Field Programmable Gate Array). A processing unit 26, a sound processing unit 27 configured by a sound IC (Integrated Circuit), an audio / video encoder 28, a streaming generation unit 29, a CPU (Central Processing Unit) 30 that controls operation of each of the above-described components, and the like Have

  The microphone 13 collects sounds and sounds around the camera device 2. The microphone 13 generates a sound signal that is an electrical signal corresponding to the collected sound and outputs the sound signal to the sound processing unit 27. The sound processing unit 27 generates digital sound data based on the electrical signal from the microphone 13. Note that the sound processing unit 27 outputs an electrical signal corresponding to the sound from the microphone 13 to the sound output connector 12 without converting it into sound data (digital format). Therefore, the sound collected by the microphone 13 can be heard by connecting, for example, a speaker, headphones or the like to the sound output connector 12.

  For example, a CMOS (Complementary Metal-Oxide Semiconductor) is used for the image sensor 25. An image by the fisheye lens 9 is formed on the image sensor 25. The image sensor 25 outputs luminance distribution data, which is an imaging signal, to the color conversion processing unit 26 as digital image data. In the present embodiment, the image sensor 25 performs imaging at an imaging speed of 1/15 frames per second, for example. Therefore, the image sensor 25 outputs image data for one frame every 1/15 second.

  The color conversion processing unit 26 replaces the data of each pixel of the luminance distribution data using a color conversion table (not shown) to generate digital image data. A header is added to the generated image data for each frame by a header adding unit 31 formed by the CPU 30. A header number M (M = 1, 2, 3,...) That is a serial number is assigned to the header according to the imaging order.

  The CPU 30 includes a JPEG encoder unit 32 as an encoder for converting image data into compressed image data in JPEG (Joint Photographic Groups Group) format. When the JPEG encoder unit 32 converts the image data into JPEG format image data, information indicating that the content of the data is JPEG converted image data is added to the header.

  By the way, noise data other than image data may be output from the image sensor 25, or noise data may be mixed between the image sensor 25 and the JPEG encoder unit 32. In the JPEG encoder unit 32, when the data input to the JPEG encoder unit 32 is image data, the data is converted into JPEG format image data, and the data with the header added to the header is the JPEG format image. Gives information that the data is data.

  On the other hand, if the data to be converted is not image data, the JPEG encoder unit 32 does not convert the image data to JPEG format. That is, noise data that is not image data among data input to the JPEG encoder unit 32 as image data output from the image sensor 25 is not converted to JPEG image data. Information indicating data is not given. The JPEG encoder unit 32 may be configured not to convert the image data into JPEG format image data when noise data of a predetermined level or more is included in the image data.

  Accordingly, based on the header information, it can be determined whether the content of the data to which the header is added is image data or noise data.

  Also for the sound data, a header is added by the header adding unit 31 with a data amount of a predetermined unit. For example, in this embodiment, a header is added to sound data for each data length of 1/30 second. In the header of the sound data, information that the data to which the header is added is sound data, and information of a header number N (N = 1, 2, 3,...) That is a continuous number as time passes. Is granted.

  The image data has a data amount of 15 frames per second, whereas the sound data has a data amount of 1/30 second. Therefore, two units (1/30 seconds × 2) of sound data correspond to one frame of image data.

  The image data and the sound data are output from the CPU 30 to the streaming generation unit 29 in a state in which the timing at the time of imaging and the time of sound collection are synchronized with each other based on the header number. For example, the header number M of the image data is 1, 2, 3, 4, 5,..., And the header number N of the sound data is 1, 2, 3, 4, 5, 6, 7, 8, 9,. Suppose that 10,. Then, the sound data of the header number N = 1 and 2 correspond to the image data of the header number M = 1, and the image data of the header number M = 1 and the sound data of the header number N = 1, 2 are time. The data is synchronized with the target timing. Similarly, the image data having the header number M = 2 and the sound data having the header numbers N = 3 and 4 are data having synchronized timings. That is, the image data of the header number M and the sound data of the header numbers N = 2M-1 and 2M are data in which the temporal timing when the image pickup and the sound collection are synchronized.

  The streaming generation unit 29 reads image data and sound data output from the CPU 30 and generates data having these content data as streaming data. The streaming generation unit 29 supplies the generated digital data signal to the communication processing unit 33 and transmits the data signal from the USB connector 10 to the camera server 3 via the USB cable 4.

  In this way, data having content data of image data and sound data is generated, and this digital data is sent to the camera server 3 via the USB cable 4. Thus, digital image data can be transmitted to the camera server 3 without being converted into an analog data signal.

  The audio / video encoder 28 reads the image data output from the color conversion processing unit 26 without converting the image data into JPEG format image data, and uses the NTSC (National TV Standards Committee) or PAL (Phase Alternating Line) video. Generate a signal. The audio / video encoder 28 converts the sound data in the digital format into a sound signal that is an electrical signal. Then, the video signal and the sound signal are output to the audio / video output connector 11.

  FIG. 3 is a block diagram showing an electric circuit of the camera server 3.

  The camera server 3 includes a USB connector 16, a microphone connector 18 to which an extension microphone 17 is connected, a general-purpose input / output port 19 to which an external device such as an infrared sensor is connected, an audio output connector 20, a video output connector 21, a speaker connector 22, In addition to the communication interface 23 and the power switch 24, the audio / video input connector 34, the audio encoder 35, the memory 36, and an MCU (Microcontroller Unit) 37 serving as control means for controlling the operation of each of the above-described components.

  The MCU 37 includes a data selection unit 38 that is a data selection unit, a data synchronization unit 39 that is a data synchronization unit, a data distribution unit 40 that is a data distribution unit, and the like. The data selection unit 38, the data synchronization unit 39, and the data distribution unit 40 are functionally realized by reading the control program stored in the memory 36 into the MCU 37.

  The data selection unit 38 determines the content of the image data transmitted from the camera device 2 based on the header information of the image data of each frame. As described above, when the data in the header is JPEG-converted image data, information to that effect is given to the header, and when the data in the header is noise data, the header No information is given. That is, the data selection unit 38 determines whether the content of the data for each header is predetermined image data or data other than the predetermined image data based on the header information.

  Then, the data selection unit 38 selects data to be output to the data synchronization unit 39 based on the determination result. That is, when the data selection unit 38 determines that the content of the data is predetermined image data, the image data is transmitted from the communication interface unit 23 via the data synchronization unit 39 and the data distribution unit 40. The data is distributed to the personal computer 7 of the client 5 through the network 6. On the other hand, data that is not predetermined image data is not output to the data synchronization unit 39 side.

  As described above, the data selection unit 38 is provided so that data that is not predetermined image data is not distributed to the client 5 side, so that the image data distributed to the client 5 side does not include noise data. It becomes like this. Therefore, the video displayed on the monitor of the personal computer 7 of the client 5 is a beautiful video with little noise. Further, since noise data is not transmitted, the amount of data transmitted from the camera server 3 to the client 5 is reduced, and the transmission speed of image data from the camera server 3 to the client 5 can be increased.

  On the other hand, since the sound data is not selected as in the case of image data, all the sound data is transmitted from the communication interface unit 23 via the network 6 via the data synchronization unit 39 and the data distribution unit 40. Delivered to PC 7. On the client 5 side, sound such as voice collected by the microphone 13 can be heard from a speaker built in the personal computer 7 or the like.

  By the way, since the sound data is not selected as in the case of image data, the header number N of the sound data input from the data selection unit 38 to the data distribution unit 40 is a continuous number without any loss. On the other hand, as described above, for the image data, the data selection unit 38 selects the data to be output to the data distribution unit 40, so that the header of the image data input from the data selection unit 38 to the data distribution unit 40 is selected. The number M may have a defect.

  Therefore, for example, when the image data is distributed with the image data of the portion where the header number M is missing, the sound data is based on the sound data when distributed to the client 5 side without compressing the data. The video based on the image data precedes the content of the sound, and the video and the sound are not synchronized. For example, if the sound data synchronized with the missing image data is deleted, the video and the sound can be synchronized, but there is a problem that the content of the sound is skipped.

  Therefore, in the data synchronization unit 39, the sound collection timing and the imaging timing of the sound data and the image data output from the data selection unit 38 are synchronized with each other so that the sound content is not skipped. Process to synchronize.

  Specifically, for example, the header number of the image data output from the data selection unit 38 is 1, 2, _, 4, 5,... (The image data of the header number 3 is missing because it is noise data. If the header number of the sound data is 1, 2, 3, 4, 5, 6,..., The sound data of the header numbers 1 and 2 is added to the image data of the header number 1. The sound data of header numbers 3 and 4 are synchronized with the image data of header number 2. Although the image data of the header number 3 is missing, the sound data of the header numbers 5 and 6 are synchronized with this portion. The sound data of header numbers 7 and 8 is synchronized with the image data of header number 4. As described above, in a state where the sound data and the image data are synchronized, the data is distributed from the data distribution unit 40 through the communication interface 23 to the network 6 side.

  On the client 5 side, only the sound data is distributed for the portion of the header number 3 in which the image data is missing, and the sound based on the sound data of the header numbers 5 and 6 is reproduced from the speaker of the personal computer 7. become.

  Note that the viewer software of the personal computer 7 of the client 5 is configured to continuously display the video based on the immediately preceding image data on the monitor until new image data is distributed. It is possible to prevent an unnatural state in which nothing is displayed on the monitor of the personal computer 7 in the case of missing. When the viewer software is configured in this way, the image data based on the image data with the header number 2 is displayed until the image data with the header number 2 is distributed and then the image data with the header number 4 is distributed. During this time, the sound based on the sound data of header numbers 5 and 6 is reproduced.

  The data distribution unit 40 simultaneously distributes data signals to these clients 5 when a plurality of clients 5 are accessing the camera server 3.

  By the way, the USB connector 16 is supplied with a data signal having image data in digital format and sound data in digital format from the camera device 2 via the USB cable 4 and is input to the MCU 37. The digital data signal input to the MCU 37 is output from the communication interface 23 to the network 6 as a digital data signal.

  Therefore, since the camera server 3 outputs the digital data signal output from the camera device 2 to the network side as the digital signal, there is no signal degradation. Further, it is not necessary to provide the camera server 3 with an AD conversion function for converting an analog signal into a digital signal. Note that the image data is transmitted by the TCP protocol or the UDP protocol by the half-duplex communication method. The sound data is transmitted by the TCP protocol or the UDP protocol in a duplex communication system.

  As described above, the image data is converted into JPEG format compressed image data in the camera device 2 and then transmitted to the camera server 3, and is distributed from the data distribution unit 40 to the client 5 through the communication interface 23. Is done. As described above, the image data is converted into the JPEG format in the camera device 2 and the image data in the JPEG format is sent to the camera server 3, thereby reducing the volume of data transmitted from the camera device 2 to the camera server 3. be able to. For this reason, the transmission speed of the image data in the camera server system 1 can be improved. As a result, the speed at which the image data is transmitted to the client 5 can be increased.

  On the other hand, for example, the conversion of the image data to the JPEG format is performed in the MCU 37 of the camera server 3 and the conversion to the JPEG format is performed each time the image data is transmitted from the client 5. Is performed, the speed at which the image data is distributed from the camera server 3 to the client 5 is reduced.

  In addition to the overall control of the camera server 3, the MCU 37 performs many processes related to the control of the camera device 2 and the communication with the client 5. Therefore, in addition to these processes, if an encoding function for JPEG conversion is provided, the load on the MCU 37 increases, the processing speed of the MCU 37 decreases, and the encoding processing speed for JPEG conversion also decreases. As a result, the transmission speed of the image data from the camera server 3 to the client 5 becomes slow.

  In addition, when conversion to the JPEG format is executed each time in response to a request for transmission of image data from the client 5, while image data transmission processing is being performed for one client 5. Since the other clients 5 have to wait, the transmission speed of the image data to the clients 5 becomes slow.

  On the other hand, JPEG conversion speed can be increased by performing JPEG conversion using the CPU 30 on the camera device 2 side, which has fewer processing items than the MCU 37 on the camera server 3 side. In addition, the MCU 37 does not perform the JPEG conversion process, so that the processing speed does not decrease, the processing speed of the data selection unit 38, the data distribution unit 40, and the like can be increased, and the image data is distributed to the client 5. The processing speed can be increased.

  Further, since the image data is simultaneously distributed to the plurality of clients 5 from the data distribution unit 40, the client 5 waits for the distribution to the other clients 5 to receive the distribution of the image data. This eliminates the possibility of speeding up delivery.

  The JPEG encoder unit 32 is provided in the CPU 30 of the camera apparatus 2 and, as indicated by a dotted line in FIG. It is also possible to configure so that image data that has been subjected to JPEG conversion by the JPEG encoder 32A is input to the MCU 37. Since the JPEG encoder unit 32A is provided as a processing device different from the MCU 37 provided with the data selection unit 38, the data synchronization unit 39, the data distribution unit 40, and the like, the processing speed of the MCU 37 is not reduced.

  Further, a dedicated JPEG encoder may be provided on the camera device 2 side separately from the CPU 30, and JPEG conversion may be performed by the JPEG encoder.

  In the audio encoder 35, digital sound data input from the communication interface 23 to the camera server 3 is read from the MCU 37, converted into a sound signal that is an electrical signal, and output to the speaker connector 22. Therefore, by connecting a speaker or the like to the speaker connector 22, for example, sound data input from the communication interface 23 can be reproduced.

  When a sound signal and a video signal output from the audio / video output connector 11 on the camera device 2 side are input to the audio / video input connector 34, the sound signal and the video signal are respectively transmitted to the audio output connector 20 and the video. Output to the output connector 21. Accordingly, when a television or the like is connected to the audio output connector 20 and the video output connector 21, the video and audio captured and collected by the camera device 2 can be viewed.

  A camera device control signal for controlling the camera device 2 is transmitted from the camera server 3 to the camera device 2 via the USB cable 4. The camera device control signal is generated by the device control unit 41 configured in the MCU 37. For example, the camera device control signal controls the start or stop of the operation of the camera device 2, or the camera device 2 receives image data. Those that request transmission. Also, for a command transmitted from the personal computer 7 on the client 5 side connected to the network to the camera server 3, the command is interpreted and a camera device control signal is transmitted to the camera device 2. For example, from a personal computer 7 or the like, pan control for performing pan shooting for moving the shooting direction horizontally, tilt control for moving the shooting direction up and down, zoom control for performing zoom shooting for narrowing or widening the shooting angle of view. Is requested, the device control unit 41 interprets the command and transmits a camera device control signal to the camera device 2. In addition to the camera device 2, the device control unit 41 also controls the external device when an external device is connected to the GPIO.

  By the way, the microphone 13 built in the camera device 2 is limited in the size that can be used due to the restrictions of the image sensor 25 disposed in the housing 8 and the circuit configuration, and cannot obtain sufficient sound collecting power. There is a case. In such a case, the extension microphone 17 is connected to the microphone connector 18.

  When the MCU 37 recognizes that the extension microphone 17 is connected to the microphone connector 18, the MCU 37 transmits a camera device control signal for stopping the sound collection by the microphone 13 to the camera device 2. Upon receiving this camera device control signal, the CPU 30 of the camera device 2 controls the sound processing unit 27 to stop generating sound data. For this reason, the camera device 2 outputs data without sound data.

  A sound signal is read from the microphone connector 18, and digital sound data is generated by the sound processing unit 42 configured in the MCU 37 of the camera server 3. The sound data and the image data transmitted from the camera device 2 are supplied from the data distribution unit 40 to the communication interface 23 and distributed to the network 6.

  As described above, when the extension microphone 17 is connected, the extension microphone 17 collects sound instead of the microphone 13 built in the camera device 2. Sound data suitable for the environment can be obtained and placed on data for distribution.

  When the operation of the camera server 3 is started (the power switch 25 is turned on), the application of the camera server 3 sets the sound data of the microphone 13 sent from the USB cable 4 as the default sound input. Thereafter, the application of the camera server 3 searches for the presence or absence of connection of the extension microphone 17 of the microphone connector 18. When the application of the camera server 3 recognizes the connection of the extension microphone 17, as described above, the device control unit 41 transmits a camera device control signal for stopping sound collection by the microphone 13 to the camera device 2. . When the application of the camera server 3 does not recognize the connection of the extension microphone 17, the sound data of the microphone 13 sent from the USB cable 4 is used as it is.

  Note that the determination by the data selection unit 38 may be performed based on other information instead of based on the header information added to the image data. For example, when image data is inspected for each header and noise data exceeding a predetermined level is detected, the image data of the header may not be transmitted to the network 6 side.

It is a figure which shows the structure of the camera server system which concerns on embodiment of this invention. It is a block diagram which shows the electric circuit of the camera apparatus in the camera server system shown in FIG. It is a block diagram which shows the electric circuit of the camera server in the camera server system shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera server system 2 ... Camera apparatus 3 ... Camera server 6 ... Network 25 ... Image sensor (imaging element)
38 ... Data selection part (data selection means)
39 ... Data synchronization unit (data synchronization means)

Claims (6)

In a camera server that distributes image data of an image captured by a camera device including an image sensor to a client through a network,
It is determined whether or not the image data is predetermined image data, and the image data determined to be the predetermined image data is output to the network side, and the image data is A camera server, comprising: a data selection unit that does not output data determined not to be the predetermined image data to the network side.   The camera server according to claim 1, wherein the data selection unit determines whether or not the image data is the predetermined image data based on information of a header added to the image data. A data synchronization means for synchronizing the image data and the sound data based on header information of the image data and the sound data when sound data is input in addition to the image data;
3. The data synchronization means synchronizes predetermined sound data of the sound data with respect to a data portion determined by the data selection means to be not the predetermined image data. The camera server described in 1. In a data processing method for distributing image data of an image captured by a camera device including an image sensor to a client through a network,
It is determined whether or not the image data is predetermined image data, and the image data determined to be the predetermined image data is output to the network side, and the image data is A data processing method, wherein the data determined not to be the predetermined image data is not output to the network side.   5. The data processing method according to claim 4, wherein whether or not the image data is predetermined image data is determined based on header information added to the image data. In a data processing method for distributing image data of an image captured by a camera device including an image sensor and sound data of sound collected by a microphone to a client via a network,
It is determined whether the image data is predetermined image data, and the image data determined to be the predetermined image data is output to the network side, and the image data is For the data determined not to be the predetermined image data, a process for not outputting to the network side is performed,
The sound data and the image data output to the network side are synchronized with each other based on header information added to each data,
A data processing method characterized by synchronizing predetermined sound data of the sound data with a data portion determined not to be the predetermined image data.

Images