CN110956211B - Multi-mode information fusion-based method in livestock and poultry farm - Google Patents

Abstract

The invention discloses a method based on multi-mode information fusion in a livestock farm, which comprises the following steps: acquiring livestock and poultry image modal information, livestock and poultry sound modal information and environment parameter modal information in a livestock and poultry farm; establishing an LPF data format, wherein image mode information is divided into R layer data, G layer data and B layer data for storage, the R layer data, the G layer data, the B layer data, sound data and environment parameter data are sequentially arranged in the data format, and a flag bit is inserted between each data; and extracting the three modal information according to the acquisition time sequence, and carrying out permutation and fusion according to the data format to obtain fusion data. By setting the data format and fusing according to the method, the method can be used for conveniently packaging the information in the livestock and poultry farm, is convenient for subsequent transmission, storage and management, and has the advantages of small storage capacity, large load information quantity, high transmission speed and the like.

Description

A method based on multimodal information fusion in livestock and poultry farms

technical field

The invention relates to the technical field of livestock and poultry intelligent breeding, and more specifically, relates to a method based on multimodal information fusion in a livestock and poultry farm.

Background technique

In recent years, the rapid development of the livestock and poultry breeding industry has transformed the traditional self-employed farming methods into large-scale and intensive livestock and poultry farms. The livestock and poultry breeding method has also changed from the traditional artificial breeding method to the breeding technology based on modern methods such as computer technology and Internet of Things technology. Therefore, a large amount of multimodal information will be generated during the breeding process, such as images, sounds, temperature, humidity, etc.

At present, traditional livestock and poultry breeding enterprises still transmit multi-modal information in a classified manner, but the traditional single-modal information fusion and transmission method not only has the disadvantages of large data volume and slow transmission speed, but also the transmission method between different modes It cannot be interoperable, resulting in the inability of good fusion of multi-modal information, so it is impossible to manage multi-modal information well. How to integrate these multi-modal information more quickly and efficiently has become a problem that must be faced in the intelligent management of livestock and poultry farms.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a method based on multimodal information fusion in livestock and poultry farms, which can fuse multimodal information into one modal information, which is convenient for transmission and storage and management.

The purpose of the present invention is achieved through the following technical solutions: a method based on multimodal information fusion in a livestock and poultry farm, comprising:

Obtain the modal information of livestock and poultry images, the modal information of livestock and poultry sounds, and the modal information of environmental parameters in livestock and poultry farms;

Construct a *.LPF data format, in which the image modality information is divided into R-layer data, G-layer data and B-layer data for storage. In the data format, R-layer data, G-layer data, B-layer data, sound data and The environmental parameter data is arranged in sequence, and a flag is inserted between each data;

The above three modal information are extracted in order of collection time, arranged and fused according to the data format, to obtain fused data.

The present invention can integrate various modal information by adopting the *.LPF data format. Compared with the traditional method of separately transmitting image information in JPG format, sound information in MP3 format, and environmental parameter information, multiple sets of The packet header, data length and various flag bits of the modal information data format are combined into one set, which effectively reduces the length of the data information and facilitates the rapid storage and transmission of multi-modal information.

Preferably, the specific structure of the *.LPF data format is as follows:

File signature field, file length, file start bit, R layer start flag, R layer data, R layer end flag, G layer start flag, G layer data, G end flag, B layer start Flag bit, B layer data, B layer end flag bit, sound data start flag bit, sound data, sound data end flag bit, environment parameter data start flag bit, environment parameter data, environment parameter data end flag bit, CRC calibration Check code, end-of-file flag.

Preferably, the various flag bits are used to identify the storage locations of various modality information in the LPF file.

Preferably, the file signature field is used to identify whether the file is an LPF file.

Preferably, the environmental parameter information includes temperature information, humidity information, harmful gas concentration, air velocity, dust concentration, and light intensity.

Preferably, after obtaining the livestock and poultry image, the image is preprocessed, and the preprocessing steps are as follows:

The livestock and poultry images are normalized, and the images are uniformly processed into a fixed-resolution RGB three-channel color livestock and poultry image, and the modality information of the livestock and poultry image is the above-mentioned RGB three-channel color livestock and poultry image information.

Preferably, after the sound information of livestock and poultry is acquired, the sound information is preprocessed, and the preprocessing steps are as follows: the sampling rate of the sound information is adjusted to a uniform frequency.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The invention provides a method based on multimodal information fusion in livestock and poultry farms. Compared with the traditional single data format of image information, sound information, and environmental parameter information, the new data format LPF proposed by the invention has the ability to store Small capacity, large amount of load information, fast transmission speed and other advantages. The method of the present invention integrates three different modal information into a new type of modal information for transmission, which is simplified and convenient for subsequent management systems to store and manage different modal information.

Description of drawings

Fig. 1 is a flowchart of a method based on multimodal information fusion in a livestock and poultry farm according to the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Example

As shown in Figure 1, the present invention proposes a method based on multimodal information fusion in livestock and poultry farms, which can perform fusion processing on acquired livestock and poultry images, livestock and poultry sounds, and environmental parameter information in livestock and poultry farms , this kind of fusion processing is realized by creating a new *.LPF data format, through the above fusion processing, the storage capacity can be reduced and the efficiency of data transmission can be improved.

In this embodiment, a plurality of high-definition cameras and sound sensors are set up in the livestock and poultry house, and various environmental sensors are set up in various places in the livestock and poultry farm. The environmental sensors include temperature sensors, humidity sensors, harmful gas concentration sensors, airflow Speed sensor, dust concentration sensor, light intensity sensor, etc. Collect images of livestock and poultry through high-definition cameras, collect sound modal information of livestock and poultry through sound sensors, and collect modal information of environmental parameters in livestock and poultry farms through various environmental sensors.

For the obtained livestock and poultry images, the images are preprocessed first, and the livestock and poultry images are uniformly processed into RGB three-channel color livestock and poultry image information with a resolution of 800*600. The RGB three-channel information corresponds to the R layer data in the subsequent data mode , G layer data, B layer data.

For the obtained livestock and poultry sounds, preprocessing is performed first, and the sampling rate of livestock and poultry sound information is uniformly adjusted to 44100Hz.

For the obtained environmental parameter information, in order to facilitate the reception and identification of the subsequent server in practical applications, the data collected by each sensor is arranged and stored in sequence.

After the corresponding preprocessing and sorting of the above data, the three modal information are fused. The fusion method is:

Construct a data format (*.LPF), the data format is shown in Table 1, including the file signature field, file length, file start bit, R layer start flag, R layer data, R layer end flag, G Layer start flag, G layer data, G layer end flag, B layer start flag, B layer data, B layer end flag, sound data start flag, sound data, sound data end flag, environment Parameter data start flag, environment parameter data, environment parameter data end flag, CRC check code, file end flag.

Table 1 Data format

When performing fusion, the image modality information is divided into R-layer data, G-layer data and B-layer data, extracted in order of acquisition time, arranged and fused according to the data format, to obtain fused data. The R layer start flag indicates the start flag of the R channel of the livestock and poultry image, and the size is 2 bytes. The R layer data represents the data of the R channel of the livestock and poultry image. The end flag of the R layer indicates the end flag of the R channel of the livestock and poultry image, and the size is 2 bytes. The G layer start flag bit indicates the start flag of the G channel of the livestock and poultry image, and the size is 2 bytes. The G layer data represents the data of the G channel of the livestock and poultry image. The end flag of the G layer indicates the end flag of the G channel of the livestock and poultry image, and the size is 2 bytes. The start mark bit of the B layer indicates the start mark of the B channel of the livestock and poultry image, and the size is 2 bytes. The B layer data represents the data of the B channel of the livestock and poultry image. The end flag bit of the B layer indicates the end flag of the B channel of the livestock and poultry image, and the size is 2 bytes. The sound data start mark bit represents the start mark of the livestock and poultry sound data, and the size is 2 bytes. The sound data represents livestock sound data. The sound data end mark bit represents the end mark of the livestock and poultry sound data, and the size is 2 bytes. The initial flag bit of the environmental parameter data represents the initial flag of the environmental parameter information in the livestock and poultry house, and the environmental parameter data represents the environmental parameter information in the livestock and poultry house. The end flag bit of the environmental parameter data indicates the end flag of the environmental parameter information in the livestock and poultry house, and the size is 4 bytes. The CRC check code indicates the check of the environmental parameter information in the livestock and poultry house, and the size is 4 bytes. The end-of-file flag is used to indicate the end of the file, and the size is 4 bytes.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the technical scope disclosed in the present invention. Modifications or replacements shall all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (6)

1. A method for multi-modal information fusion in a livestock farm, comprising:

acquiring livestock and poultry image modal information, livestock and poultry sound modal information and environment parameter modal information in a livestock and poultry farm;

establishing an LPF data format, wherein image mode information is divided into R layer data, G layer data and B layer data for storage, the R layer data, the G layer data, the B layer data, sound data and environment parameter data are sequentially arranged in the data format, and a flag bit is inserted between each data; the specific structure of the LPF data format is as follows:

file signature field, file length, file start bit, R layer data, R layer end bit, G layer start bit, G layer data, G layer end bit, B layer start bit, B layer data, B layer end bit, voice data start bit, voice data end bit, environmental parameter data start bit, environmental parameter data end bit, CRC check code, file end bit;

and extracting the three modal information according to the acquisition time sequence, and carrying out permutation and fusion according to the data format to obtain fusion data.

2. The method for multi-modal information fusion based on the livestock and poultry farm according to claim 1, wherein various flag bits are used for identifying the storage positions of various modal information in the LPF file.

3. The method of claim 1, wherein the file signature field is used to identify whether the file is an LPF file.

4. The method based on multi-modal information fusion in a livestock farm according to claim 1, wherein the environmental parameter information includes temperature information, humidity information, harmful gas concentration, airflow rate, dust concentration, illumination intensity.

5. The method based on multi-modal information fusion in a livestock farm according to claim 1, wherein the preprocessing of the images after the acquisition of the livestock images is as follows:

normalizing the livestock and poultry image, uniformly processing the image into an RGB three-channel color livestock and poultry image with fixed resolution, wherein the livestock and poultry image modal information is the RGB three-channel color livestock and poultry image information.

6. The method for multi-modal information fusion based on the livestock and poultry farm according to claim 1, wherein after the voice information of the livestock and poultry is obtained, the voice information is preprocessed, and the preprocessing steps are as follows: the sampling rate of the sound information is adjusted to a uniform frequency.

Images