WO2019049529A1 - Condition monitoring system and method - Google Patents

Abstract

This state monitoring system for monitoring the state of a worker working at a work site is provided with: a work information acquisition unit (34) which acquires, as work information, information for identifying the work to be performed by the worker at the work site; a biological information detection unit which detects biological information that is varied according to the work performed by the worker; a biological index acquisition unit (33c) which, on the basis of the biological information, acquires a biological index that serves as an index for determining the state of the worker; and a work information/biological index storage unit (31b) which stores the work information and the biological index in association with each other.

Description

Condition monitoring system and method

The present invention relates to a state monitoring system and a state monitoring method for monitoring the state of a worker who works at a work site.

Conventionally, various apparatuses for detecting biological information of a subject have been proposed. For example, Patent Document 1 discloses an apparatus which is attached to the head of a subject and detects a brain activity signal of the subject. This device is configured by arranging an electrode for detecting a brain activity signal inside a member (frame) such as a headband. In recent years, such a wearable biological information detection apparatus has been realized.

Further, for example, Patent Document 2 proposes a head-mounted device provided with a sensor that monitors the movement of the driver's eyes and a motion sensor (for example, an acceleration sensor). The former sensor can detect the direction of the driver's eye, and the latter sensor can detect vibration to detect whether the vehicle is moving or stationary. Therefore, by using the information detected by both the sensors, it is possible to monitor drowsiness and carelessness during driving of the driver and to take appropriate measures as necessary. For example, if the driver is gazing downward at a longer time than a predetermined time and the vehicle is moving, an alert is issued on the assumption that the driver is careless.

Further, for example, in Patent Document 3, a working state management system for a power tool is proposed, which enables appropriate work according to the working state by transmitting work navigation information giving work instructions according to the working state to the power tool. It is done.

Unexamined-Japanese-Patent No. 2017-108759 (refer Claim 1, FIG. 1 etc.) JP-A-2011-528242 (see claim 1, paragraphs [0001], [0007], [0016], FIG. 1, etc.) JP-A-2017-62556 (see claim 1, paragraph [0011], FIG. 1, etc.)

Incidentally, for example, a site such as a factory where product production (including product processing and assembly) is performed, a site such as a distribution warehouse where parts and products are put in and out (hereinafter, these sites are also referred to as “work sites”) In the case of), improvement of work aiming at improvement of work efficiency is considered daily. However, in the past, the number of product defects (mistakes) and the length of tact time have been used as indicators to identify the work to be improved, and it is determined whether the workers can comfortably work in the improvement of the work. It was rare to be considered. Therefore, for example, the working time is short, but the worker's physical condition is not good, and the hard work etc. is not a candidate for improvement of the work, and as a result the worker continues hard work hard, resulting in an accident or disease. Even in the case of carrying out the work of (2), there may be a situation where the efficiency of the other work is lowered due to the accumulation of fatigue.

Therefore, in order to improve the work of the worker at the work site, it is necessary to improve the work in consideration of the "comfort of the work (easiness of work)" of the worker. With regard to the work performed by the worker, it is necessary to be able to grasp the worker's physical and mental burden and to understand the worker's “comfort”. However, no system or method has been proposed for making it possible to grasp the "comfort of the work" in relation to the work performed by the work site.

In addition, the patent document 1 mentioned above only discloses an example of the apparatus which detects biological information (brain activity signal), and the relationship between the biological information and the operation | work which a subject performs is considered at all. In addition, in relation to the work performed by the subject, "comfort of work" can not be pursued and grasped. Further, Patent Document 2 detects the direction of eyes while the driver is driving on a vehicle and detects the presence or absence of carelessness. It is not intended to detect biological information or improve work at the work site where it is used. Moreover, patent document 3 transmits operation | movement navigation information to an electric tool, and it is disclosed in the disclosure of the point which guide | induces operation | work, and the disclosure of the point which makes it possible to grasp "comfort of operation" about the operation | work which an operator performs. There is no

The present invention has been made to solve the above-mentioned problems, and the purpose thereof is to grasp the "comfort of the work" of the worker in relation to the work performed by the worker at the work site. An object of the present invention is to provide a condition monitoring system and a condition monitoring method capable of preventing an accident or a disease during the work of a worker and improving the work efficiency by improving the work based on the "comfort of the work".

The state monitoring system according to one aspect of the present invention is a state monitoring system that monitors the state of a worker who works at a work site, and includes information for specifying the work performed by the worker at the work site, the work information And a biological information detection unit for detecting biological information that changes along with the work of the worker, and an index for determining the state of the worker based on the biological information And a task information / bioindex storage unit for storing the task information and the bioindex in association with each other.

A state monitoring method according to another aspect of the present invention is a state monitoring method for monitoring the state of a worker who performs work at a work site, the information for specifying the work performed by the worker at the work site, An indicator for determining the state of the worker based on the work information acquisition process to be acquired as information, the living body information detection process for detecting the living body information that changes along with the work of the worker, and the living body information And a storage step of associating and storing the work information and the biometric index.

Work information / bioindicator storage unit by associating work information (for example, work name) about the work performed by the worker at the work site and a biometric index (for example, parasympathetic nerve activity, blink frequency, α wave value / β wave value) By storing in the third party, the third party can grasp the information stored in the work information / biometric index storage unit, for example, by the access from the external terminal. Then, the third party grasps the physical and mental burden of the worker about the work of the worker performed on the work site based on the grasped biometric index, and "comfort of the worker" It becomes possible to grasp. Therefore, the third party can improve the work of the worker based on the "comfort of the work", and by the improvement of the work, it is possible to prevent an accident or a disease during the work of the worker, It becomes possible to improve work efficiency.

It is an explanatory view showing the schematic structure of the state surveillance system concerning one embodiment of the present invention. It is a block diagram showing an outline composition of a camera which the above-mentioned state surveillance system has. It is an explanatory view expanding and showing a living body information detecting device which the above-mentioned state surveillance system has. It is a block diagram which shows the structure of the principal part of the said biometric information detection apparatus. It is a block diagram showing an outline composition of a server which the above-mentioned state surveillance system has. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system. It is a graph which shows an example of a temporal change of the heart rate detected by the above-mentioned living body information detecting device. It is a graph which shows the time series data of RR interval obtained based on the data of the above-mentioned heart rate. It is a graph which shows the frequency analysis result in time range (A) of FIG. It is a graph which shows the frequency analysis result in time range (B) of FIG. It is an explanatory view showing a schematic structure of a state surveillance system concerning other embodiments of the present invention. It is a perspective view expanding and showing a living body information detecting device and a head-worn-type image display device which the above-mentioned state surveillance system has. It is a block diagram which shows the structure of the principal part of the said biometric information detection apparatus. It is a block diagram which shows the structure of the principal part of the said head-mounted image display apparatus. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system. It is sectional drawing which shows the schematic structure of the video display part which the said head-mounted type | mold video display apparatus has. It is explanatory drawing which shows a schematic structure of the state monitoring system which concerns on the further another embodiment of this invention. It is a perspective view expanding and showing a living body information detecting device and HMD which the above-mentioned state surveillance system has. It is a block diagram which shows the structure of the principal part of the said biometric information detection apparatus. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system. It is an explanatory view showing the composition of the modification of the above-mentioned state surveillance system. It is a block diagram which shows the structure of the terminal device which the state monitoring system of the said modification has. It is explanatory drawing which shows the rough flow of the process in the state monitoring system of the said modification. It is explanatory drawing which shows a schematic structure of the state monitoring system which concerns on the further another embodiment of this invention. It is a block diagram which shows the schematic structure of the barcode reader which the said status monitoring system has. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system. It is an explanatory view showing the composition of the modification of the above-mentioned state surveillance system. It is explanatory drawing which shows the rough flow of the process in the state monitoring system of the said modification. It is explanatory drawing which shows a schematic structure of the state monitoring system which concerns on the further another embodiment of this invention. It is a block diagram which shows the schematic structure of the terminal device which the said state monitoring system has. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system. It is an explanatory view showing an example of a total result by a total processing part which the above-mentioned state surveillance system has. It is explanatory drawing which shows a schematic structure of the state monitoring system which concerns on the further another embodiment of this invention. It is a block diagram which shows the structure of the principal part of the head mounted image display apparatus which the said state monitoring system has. It is an explanatory view showing a rough flow of processing in the above-mentioned state surveillance system.

It will be as follows if each embodiment of the present invention is described based on a drawing. In addition, a common member number may be added to the configuration common to each embodiment, and the description thereof may be omitted. Further, in the present specification, when the numerical range is represented as a to b, the values of the lower limit a and the upper limit b are included in the numerical range.

In the present specification, “work site” refers to a site where a worker works. Here, for the work performed by the worker, in addition to work such as product manufacture (including processing and assembly) and inspection, etc., distribution warehouses (storage of parts necessary for manufacturing and products manufactured using the above parts) It also includes work to get in and out of places and storage rooms. For example, after receiving an order for a product, an act of picking the product from the warehouse is also included in the work at the work site. Therefore, the "work site" includes a distribution warehouse as well as a site such as a factory that manufactures products. Note that an action performed outside the work site (for example, a driving action of a vehicle performed to carry out a product from the work site to another place) is not included in the “work performed at the work site”.

First Embodiment
(Overall configuration of status monitoring system)
FIG. 1 is an explanatory view showing a schematic configuration of the state monitoring system 1 of the present embodiment. The condition monitoring system 1 is a system for monitoring the condition of the worker A who is working at the work site, and includes a camera 10, a biological information detection device 20, and a server 30. The camera 10 and the server 30, and the biological information detection apparatus 20 and the server 30 are communicably connected via a communication line such as a wireless LAN (Local Area Network). The mutual communication form may be wireless communication based on the Bluetooth (registered trademark) standard. The camera 10 and the biological information detection apparatus 20 are provided according to the number of workers A.

The server 30 is communicably connected to a terminal device 100 outside the system via a communication line. Thus, a third party other than the worker A (for example, a manager or supervisor of a work, hereinafter also referred to as a manager or the like) accesses the server 30 from the terminal device 100 and is stored in the server 30 It is possible to grasp the existing information. The terminal device 100 may be included in the state monitoring system 1 (see FIG. 21) or may be configured integrally with the server 30. Hereinafter, the details of each part constituting the state monitoring system 1 will be further described.

(Configuration of camera)
FIG. 2 is a block diagram showing a schematic configuration of the camera 10. The camera 10 includes an imaging unit 11, a storage unit 12, a communication unit 13, and a control unit 14. The camera 10 is installed at a position where the worker A can be included in the field of view.

The imaging unit 11 is a block that captures an image by capturing the worker A and his work, and an imaging lens, an imaging device (for example, a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS)), a focusing mechanism, and an aperture A mechanism, a drive circuit, an A / D (analog / digital) conversion circuit, and the like are included. The storage unit 12 is a memory for temporarily storing data of an image acquired by the imaging unit 11 in addition to the operation program of the control unit 14, and, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), It is configured to include a non-volatile memory and the like. The communication unit 13 is a communication interface for transmitting the data of the image to the outside, and includes a transmission circuit, an antenna, and the like. The control unit 14 is configured by, for example, a CPU (Central Processing Unit), and controls the operation of each unit of the camera 10.

(Configuration of biological information detection device)
FIG. 3 is an explanatory view showing the biological information detection apparatus 20 in an enlarged manner. The living body information detection device 20 is a wearable detection device that is attached to the worker A and detects the living body information of the worker A. In the present embodiment, the living body information detection apparatus 20 is attached to the surface of the belt body 20a which is passed through and fixed to the arm of the worker A and the surface of the belt body 20a, and a display unit for displaying various information detected And 24.

FIG. 4 is a block diagram showing the configuration of the main part of the biological information detection apparatus 20. As shown in FIG. The biological information detection device 20 includes a biological information detection unit 21, a storage unit 22, a communication unit 23, and a control unit 25 in addition to the display unit 24 described above.

The living body information detection unit 21 is configured of a heart rate sensor 21 a that detects a heart rate as the living body information that changes in accordance with the work performed by the worker A. The storage unit 22 is a memory for temporarily storing biological information (here, heart rate data) detected by the biological information detection unit 21 in addition to the operation program of the control unit 25. For example, RAM, ROM, non-volatile memory Memory is included. The communication unit 23 is a communication interface for transmitting the living body information to the outside or receiving information transmitted from the outside, and includes a transmitting circuit, a receiving circuit, an antenna, and the like. The control unit 25 includes, for example, a CPU, and controls the operation of each unit of the biological information detection apparatus 20. The living body information detection apparatus 20 may separately include, for example, a sensor that detects a position, an amount of ultraviolet light, an acceleration, an atmospheric pressure, and the like, in addition to the above-described heart rate sensor 21a.

For example, “Microsoft Band 2” manufactured by Microsoft Corporation can be used as the living body information detection apparatus 20 having the above configuration.

(Server configuration)
FIG. 5 is a block diagram showing a schematic configuration of the server 30. As shown in FIG. The server 30 includes a storage unit 31, a communication unit 32, and a control unit 33.

The storage unit 31 is configured of, for example, a hard disk, and includes a work procedure data storage unit 31 a, a work information / biometric index storage unit 31 b, and an operation program storage unit 31 c. The work procedure data storage unit 31a stores electronic data indicating a work procedure (manual). The work information / biometric index storage unit 31 b includes work information (for example, work name) acquired by the work information acquisition unit 34 described later, and a biometric index acquired by the later-described biological index acquisition unit 33 c (e.g. The parasympathetic nerve activity obtained based on the heart rate of Note that “in association” indicates that the work information and the biometric index correspond to each other on a one-on-one basis. The operation program storage unit 31c stores an operation program to be executed by the control unit 33.

The communication unit 32 is a communication interface for transmitting and receiving information to and from the camera 10 and the biological information detection apparatus 20, and includes a transmission circuit, a reception circuit, an antenna, and the like.

The control unit 33 includes an overall control unit 33a, a specification processing unit 33b, and a biometric index acquisition unit 33c. The overall control unit 33a, the specific processing unit 33b, and the biometric index acquisition unit 33c may be integrally configured by one CPU, or may be configured by separate CPUs. The overall control unit 33 a controls the operation of each unit of the server 30.

The identification processing unit 33 b is based on the image acquired by the imaging unit 11 of the camera 10 (image at work of the worker A) and the electronic data indicating the work procedure stored in the work procedure data storage unit 31 a. The task name of the task performed by the worker A is identified, and the identified task name is acquired as task information. The specific method of specifying the work name will be described later. The imaging unit 11 of the camera 10 described above and the identification processing unit 33b of the server 30 constitute a work information acquisition unit 34 that acquires, as work information, information specifying the work performed by the worker A at the work site. ing.

Based on the biological information detected by the biological information detection unit 21 of the biological information detection apparatus 20, the biological index acquisition unit 33c acquires a biological index serving as an index for determining the state of the worker A. In addition, the specific method of acquiring a biometric index is mentioned later.

(About the state monitoring method)
FIG. 6 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the present embodiment. Hereinafter, the state monitoring method of the present embodiment will be described with reference to FIG. Here, it is assumed that the worker A carries out the work according to the work procedure manual prepared in advance, and the electronic data of the work procedure manual is stored in the work procedure data storage unit 31a.

When the imaging unit 11 of the camera 10 captures the work of the worker A and acquires an image (# 1), the control unit 14 transmits the data of the image (work image) acquired by the imaging unit 11 to the communication unit It transmits to the server 30 through 13 (# 2).

In addition, when the biological information detection unit 21 (heart rate sensor 21a) of the biological information detection device 20 detects a heart rate that changes with the work of the worker A (# 3; biological information detection step), the control unit 25 Data of the heart rate acquired by the heart rate sensor 21a is transmitted to the server 30 via the communication unit 23 (# 4).

In the server 30, the control unit 33 (in particular, the specific processing unit 33b) reads electronic data indicating the work procedure from the work procedure data storage unit 31a (# 5). When the server 30 receives the data of the work image transmitted from the camera 10 via the communication unit 32 (# 6), the specification processing unit 33b receives the electronic data indicating the work procedure and the data of the work image. The work name of the work performed by the worker A is specified based on the above, and the specified work name is acquired as work information (# 7; work information acquisition process).

For example, the specification processing unit 33b extracts (analyzes and recognizes) a part (or a tool) used by the operator A from the work image by image analysis (image recognition), and obtains electronic data indicating a work procedure. Reference is made to the parts (or tools) appearing in the work registered in the work procedure to determine the match / mismatch. In addition to the case in which the shapes completely match, the case in which the two match each other includes the case in which the shapes are nearly perfect matches. If both parts (or tools) match, the specific processing unit 33 b is the work name of the work that is determined to be matched, which is registered in the electronic data of the work procedure. Is extracted and acquired as work information.

On the other hand, when the server 30 receives the heart rate data transmitted from the biological information detection apparatus 20 via the communication unit 32 (# 8), the biological index acquisition unit 33c determines that the living body is based on the heart rate data. As an index, the parasympathetic nerve activity which is related to concentration is calculated and acquired (# 9; biomarker acquisition step).

Specifically, based on the data of the heart rate, the biomarker acquisition unit 33c calculates a time of one beat (RR interval, heartbeat interval) to obtain time series data of the RR interval. For example, as shown in FIG. 7, when the heart rate per minute (BPM: Beat Per Minute) is obtained from the heart rate sensor 21a every second, for each heart rate,
RR interval (seconds) = 60 / heart rate (BPM)
Thus, as shown in FIG. 8, time-series data of RR intervals every one second can be obtained.

Next, when a certain time range of time series data of RR interval is frequency analyzed by fast Fourier transform, graphs as shown in FIG. 9 and FIG. 10 are obtained. 9 shows the frequency analysis result in the time range (A) of FIG. 8 as an example, and FIG. 10 shows the frequency analysis result in the time range (B) of FIG. 8 as an example. 0.04 to 0.15 Hz is an LF (Low Frequency) component, and 0.15 to 0.40 Hz is an HF (High Frequency) component. In the relaxed state, since the HF component becomes relatively larger than the LF component, here, the integrated value of the value (power, intensity) of 0.15 to 0.40 Hz is used as the parasympathetic nerve activity. That is, it is considered that worker A has more concentration as the degree of parasympathetic nerve activity is higher. The biomarker acquisition unit 33c calculates and acquires the parasympathetic nerve activity by such frequency analysis.

Next, the control unit 33 (for example, the overall control unit 33a) associates the task information (task name) acquired in # 7 with the biometric index (parasympathetic nerve activity level) acquired in # 9 to associate the task information / The index storage unit 31b stores the index (# 10; storage step). For example, when worker A performs work 1 for tightening a screw, the parasympathetic nerve activity is a1 and for work 2 which a hole is made in the substrate, the parasympathetic nerve activity a2 (for example, a2 <a1) In the assembly work 3 of the parts, information such as that the parasympathetic nerve activity is a3 (for example, a3> a1) is stored in the work information / biometric index storage unit 31b.

As described above, in the state monitoring system 1 according to the present embodiment, work management is performed because work information (work name) and biometric index (parasympathetic nerve activity activity) are associated with each other and stored in the work information / bioindex storage unit 31b. A third party such as a person can grasp the information stored in the work information / biometric index storage unit 31b by, for example, access from the external terminal device 100 (see FIG. 1). Then, the third party can easily grasp the physical and mental burden of the worker A regarding the work of the worker A performed at the work site based on the biometric index related to the work information. . That is, in the above example, for task 2, since the parasympathetic nervous activity is lower than that of other tasks, worker A's concentration power is low and it is easily judged that physical and mental burden is large. Can.

As described above, the physical and mental burden on the work of the worker A can be "visualized" by associating the work information with the biometric index and storing the result in the work information / biometric index storage unit 31b. . Thereby, the third party can easily grasp the "comfort of the work" about the work currently performed by the worker A, and it is determined whether the work of the worker A is "hard work" Can be easily grasped. If the work of the worker A is "hard work", the third party "comfortable work" such as, for example, the review of the work content, replacement of the work, and a break during the work Work to improve the As a result, it is possible to prevent accidents and diseases during the work of the worker A, and to improve the performance of the worker A and to improve the work efficiency.

In addition, the third party can confirm which work the concentration of the worker A is often reduced by appropriately checking the information stored in the work information / biometric index storage unit 31 b. As a result, the work can be improved with priority over the work in which the concentration power is reduced.

Further, the biological information detection unit 21 is configured of a heartbeat sensor 21a. As a result, based on the heart rate data acquired by the heart rate sensor 21a, the biomarker acquisition unit 33c can reliably calculate and acquire the parasympathetic nerve activity as a biomarker.

Also, as described above, it is considered that worker A has more concentration as the parasympathetic activity is higher, so the parasympathetic activity is a biological index for determining the concentration of worker A. It can be said. Since such a biometric index is stored in the work information / biometric index storage unit 31b, the third party uses the above-mentioned biometric index stored in the work information / biometric index storage unit 31b to determine the concentration of the worker A. It is possible to accurately determine the physical and mental burden on the worker by making an accurate judgment.

The work information acquisition unit 34 is configured to include the imaging unit 11 and a specific processing unit 33 b. As a result, by the processing in the specific processing unit 33b using the work image of the worker A acquired by the imaging unit 11, the work information (work name) of the worker A can be reliably acquired.

In the present embodiment, an example will be described in which the server 30 performs calculation of parasympathetic nerve activity by frequency analysis (acquisition of a biological index) and specification of an operation based on an operation procedure and an operation image (acquisition of operation information). However, it may be performed outside the server 30. For example, the control unit 25 of the biological information detection apparatus 20 may calculate the degree of parasympathetic nerve activity by frequency analysis and transmit the result to the server 30. That is, the control unit 25 of the biological information detection apparatus 20 may have the function of the biological index acquisition unit 33 c of the server 30. Further, electronic data indicating the work procedure is stored in the storage unit 12 of the camera 10, and the control unit 14 identifies the work based on the work procedure and the work image, and transmits the result to the server 30. You may do so. That is, the control unit 14 of the camera 10 may have the function of the identification processing unit 33 b of the server 30.

Further, in the present embodiment, the parasympathetic nervous activity is used as a biological index, but the parasympathetic nervous activity (hereinafter referred to as parasympathetic nervous activity ref) at the time of worker A's inactivity is measured in advance. The ratio of parasympathetic activity during work A (hereinafter referred to as parasympathetic activity work) to parasympathetic activity ref, ie, (parasympathetic activity work) / (parasympathetic activity ref) is a biomarker It may be

Therefore, as in the present embodiment, in the case where the biological information detection unit 21 includes the heart rate sensor 21a, the biological index is the parasympathetic nerve activity obtained based on the heart rate detected by the heart rate sensor 21a, or It can be said that the ratio of the nerve activity to the reference value ((parasympathetic nerve activity work) / (parasympathetic nerve activity ref)) may be used. In this case, the third party ensures the physical and mental burden of the worker A on the work of the worker A performed at the work site on the basis of the biometric index regardless of using any of the above-mentioned biomarkers. It is possible to

In the state monitoring method of the present embodiment, the work information acquisition process (# 7) for acquiring, as the work information, information specifying the work performed by the worker A at the work site, and the above-described work of the worker A Biometrics acquisition process (# 9) which acquires a biometrics index which is an index for judging the state of the worker A based on the living body information detection process (# 3) which detects living body information which changes, and the above-mentioned living body information And a storage step (# 10) for associating and storing the work information and the biometric index. Thereby, the same effect as the effect by the configuration of the state monitoring system 1 described above can be obtained.

Further, the state monitoring method further includes an imaging step (# 1) for photographing the work of the worker A to acquire an image (work image), and in the work information acquisition step (# 7), The work name of the work is specified based on the electronic data indicating the procedure of the work stored in advance in the work procedure data storage unit 31a, and the work name is acquired as the work information. By the processing based on the work image of the worker A and the electronic data, the work information (work name) of the worker A can be reliably acquired.

Second Embodiment
FIG. 11 is an explanatory view showing a schematic configuration of the state monitoring system 1 of the present embodiment. The condition monitoring system 1 of the present embodiment replaces the living body information detection device 20 of the first embodiment with a living body information detection device 40 and replaces the camera 10 with an HMD that is a head mounted image display device (see-through virtual image display device). The configuration is the same as that of the first embodiment except that a (Head-Mounted Display) 50 is provided. Further, the living body information detection apparatus 40 and the server 30, and the HMD 50 and the server 30 can communicate via a communication line such as a wireless LAN or by wireless communication based on the Bluetooth (registered trademark) standard. The living body information detection device 40 and the HMD 50 are provided corresponding to each worker A.

FIG. 12 is an enlarged perspective view showing the biological information detection apparatus 40 and the HMD 50. As shown in FIG. The living body information detection device 40 and the HMD 50 both have the shape of glasses, and the worker A can perform the work in a state where the living body information detection device 40 and the HMD 50 are overlapped.

(Configuration of biological information detection device)
FIG. 13 is a block diagram showing the configuration of the main part of the biological information detection apparatus 40. As shown in FIG. The biological information detection apparatus 40 includes a biological information detection unit 41, a storage unit 42, a communication unit 43, and a control unit 44.

The living body information detection unit 41 is configured of an electro-oculogram sensor 41a that detects the presence or absence of blinking as living body information that changes with the work of the worker A. Generally, the cornea side of human eye is positively charged and the retina side is negatively charged. For this reason, when a person moves his eye or blinks, the potential of the skin near the cornea changes. The electro-oculogram sensor 41a can detect the movement of the human eye and the presence or absence of blinking by detecting the electric potential.

The storage unit 42 is a memory for temporarily storing the biological information detected by the biological information detection unit 41 in addition to the operation program of the control unit 44, and includes, for example, a RAM, a ROM, a non-volatile memory, etc. ing. The communication unit 43 is an interface for communicating with the outside (for example, the server 30), and includes a transmission circuit, a reception circuit, an antenna, and the like. The control unit 44 includes, for example, a CPU, and controls the operation of each unit of the biological information detection apparatus 40.

As the living body information detection apparatus 40 of the said structure, "JINS MEME" made from Ginzu can be used, for example.

(Configuration of HMD)
As shown in FIG. 12, the HMD 50 is a display unit configured to have an image display unit 61 that displays an image to be presented to the observer (operator A) and a holding member 62. The holding member 62 corresponds to a frame and a temple of the glasses, holds the image display unit 61 in front of the eye of the operator A (for example, in front of the right eye), and integrally integrates the visor 62a located in front of both eyes of the observer Hold it. The image display unit 61 is held by the holding member 62 by clipping, and is a so-called clip-on type. The HMD 50 may have a lens integrated structure in which a part of the image display unit 61 doubles as a lens of glasses. In addition, the HMD 50 may be of a type in which two video display units 61 are arranged and the video can be observed with both eyes. The detailed configuration of the video display unit 61 will be described later.

FIG. 14 is a block diagram showing the configuration of the main part of the HMD 50. As shown in FIG. The HMD 50 acquires in advance task content to be presented to the worker A and a task name corresponding to the task content as task information, and functions as the task information acquisition unit 50a. The HMD 50 also functions as a work navigation system that sequentially presents the work contents to the worker A in the form of video. Such an HMD 50 includes a display element 51, a storage unit 52, a communication unit 53, and a control unit 54.

The display element 51 is an element that displays an image to be presented to the worker A, and is configured of, for example, a liquid crystal display (LCD). The storage unit 52 is a memory for storing work contents corresponding to the work contents and contents to be presented to the worker A in addition to the operation program of the control unit 54 as work information in advance, and may be RAM, ROM, nonvolatile memory It is configured. The communication unit 53 is an interface for communicating with the outside (for example, the server 30), and includes a transmission circuit, a reception circuit, an antenna, and the like. The control unit 54 includes, for example, a CPU, and controls the operation of each unit of the HMD 50.

(About the state monitoring method)
FIG. 15 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the present embodiment. Hereinafter, the state monitoring method according to the present embodiment will be described with reference to FIG.

First, in the HMD 50, the work content stored in the storage unit 52 is sequentially displayed as a video on the display element 51 (# 11). Then, the worker A carries out the work in accordance with the displayed contents. In addition, the HMD 50 transmits a work name corresponding to the displayed work content as work information to the server 30 (# 12).

On the other hand, the presence or absence of blink during the work of the worker A is detected and acquired by the electro-oculogram sensor 41a of the biological information detection apparatus 40 (# 13; biological information detection process). The blink presence / absence data of the worker A is transmitted to the server 30 (# 14).

When the server 30 receives the blink data transmitted from the biological information detection apparatus 40 (# 15), the biological data acquiring unit 33c is related to fatigue as a biological data based on the blink data described above. The blink count per unit time is calculated and acquired (# 16; biomarker acquisition step). Then, the control unit 33 (for example, the overall control unit 33a) associates the work information (work name) transmitted from the HMD 50 in # 12 with the biometric index (the number of blinks per unit time) acquired in # 16. The operation information / biometric index storage unit 31b stores the information (# 17; storage step). For example, when worker A performs work 1 for tightening a screw, the number of blinks per unit time is b1 and when work 2 for drilling a hole in a substrate, the number of blinks per unit time is b2 (for example, b2 <b1), and at the time of the assembly work 3 of the component, information such as the number of blinks per unit time is b3 times (for example, b3> b1) is the work information / biometric index storage unit 31b Is stored in

Also in the condition monitoring system 1 of the present embodiment, the work information (work name) and the biometric index (the number of blinks per unit time) are associated with each other and stored in the work information / bioindex storage unit 31b. A third party such as a manager grasps the information stored in the work information / biometric index storage unit 31b by, for example, access from the external terminal device 100 (see FIG. 1), and physically removes the worker A's work.・ It becomes possible to easily grasp the mental burden. That is, in the above-mentioned example, since the number of blinks per unit time is smaller than that of the other operations, the fatigue of the worker A is large, and the physical and mental burden (degree of fatigue) is large. It can be easily judged.

Therefore, as in the first embodiment, the third party can easily grasp the "comfort of work" for the work currently performed by the worker A. And, by aiming to improve the work as necessary, it is possible to obtain the same effect as that of the first embodiment, such as preventing accidents and diseases during the work of the worker A. In addition, the third party can confirm which work the worker A tends to get tired by appropriately checking the information stored in the work information / biometric index storage unit 31 b. As a result, the work can be improved with priority to the work that is easy to get tired.

Further, the biological information detection unit 41 is configured by an electro-oculogram sensor 41 a. As a result, the vital sign acquiring unit 33c can reliably calculate and acquire the number of blinks per unit time as the vital index based on the blink / not presence data acquired by the electro-oculogram sensor 41a.

Further, since it is considered that the fatigue of the worker A is larger as the number of blinks per unit time is smaller, it can be said that the number of blinks per unit time is a biological index for determining the fatigue of the worker A. Since such a biometric index is stored in the work information / biometric index storage unit 31b, the third party can determine the degree of fatigue of the worker A from the above-mentioned biometric index stored in the work information / biometric index storage unit 31b. Can accurately determine the physical and mental burden on the worker.

In addition, the work information acquisition unit 50a acquires (stores in the storage unit 52) the work content to be presented to the worker A and the work name corresponding thereto as work information in advance, The HMD 50 functions as a work navigation system to sequentially present in In this configuration, by transmitting the work information (work name) of the worker A from the HMD 50 to the server 30, it is not necessary to perform the process of specifying the work name on the server 30 side as in the first embodiment. There is no need to store electronic data indicating the work procedure on the server 30 side. As a result, the specific processing unit 33 b and the work procedure data storage unit 31 a of the server 30 can be omitted, the configuration of the server 30 can be simplified, and the processing load on the server 30 side can be reduced. In addition, since the work content is sequentially presented as a video by the HMD 50, the worker A can appropriately proceed the work while checking the presented video (work content).

Further, in the state monitoring method of the present embodiment, the work content is displayed to the worker A by the HMD 50 which previously obtains (stores in the storage unit 52) the work content and the corresponding work name as the work information. A work content display process (# 11) sequentially presented as a video on A and an output process (# 12) for outputting the work information from the HMD 50, and in # 17, the work information output from the HMD 50 is Store in association with the indicator. As a result, similar to the above, the worker A can appropriately perform the work, can simplify the configuration of the server 30, and can reduce the processing load on the server 30 side.

In the present embodiment, although the example in which the calculation of the number of blinks per unit time is performed on the server 30 side has been described, the calculation may be performed other than the server 30. For example, an arithmetic function may be added to the biological information detection device 40, and the biological information detection device 40 may calculate the number of blinks per unit time and then transmit the calculated data to the server 30. . That is, the control unit 44 of the biological information detection apparatus 40 may have the function of the biological index acquisition unit 33 c of the server 30.

Further, in the present embodiment, the blink count per unit time is used as the biological index, but the blink count per unit time (hereinafter referred to as blink count ref per unit time) at the time of operator A's calmness is measured in advance. Besides, the ratio between the number of blinks per unit time during work of worker A (hereinafter referred to as the number of blinks work per unit time work) and the number of blinks per unit time ref, that is, (a blink per unit time The number of times work) / (the number of blinks per unit time ref) may be used as the biomarker.

Therefore, as in the present embodiment, in the case where the biological information detection unit 41 is configured by the electro-oculogram sensor 41a, the biological index is the number of blinks per unit time detected by the electro-oculogram sensor 41a, or per unit time It may be said that the ratio of the number of blinks to the reference value ((number of blinks per unit time work) / (number of blinks per unit time ref)) may be used. In this case, the third party ensures the physical and mental burden of the worker A on the work of the worker A performed at the work site on the basis of the biometric index regardless of using any of the above-mentioned biomarkers. It is possible to

(Configuration of video display unit)
Next, details of the configuration of the video display unit 61 of the HMD 50 of the present embodiment described above will be described. FIG. 16 is a cross-sectional view showing a schematic configuration of the video display unit 61. As shown in FIG. The image display unit 61 guides light from the outside world to the pupil of the observer (operator A), thereby making the observer observe the outside world, and displaying an image as a virtual image in a part of the observer's field of view Includes a display optical system 61a provided to the user. The display optical system 61a includes, for example, an illumination optical system 62, a polarizing plate 63, a polarization beam splitter (PBS) 64, the display element 51 described above, and an eyepiece optical system 66. It is not limited to this configuration.

In addition, the direction is defined as follows for convenience of explanation. An axis optically connecting the center of the optical pupil P formed by the eyepiece optical system 66 and the center of the display surface of the display element 51 and an extension of the axis are taken as an optical axis. Then, a direction perpendicular to the optical axis plane of the HOE (holographic optical element; holographic optical element) 83 of the eyepiece optical system 66 is taken as an X direction. The optical axis plane of the HOE 83 refers to a plane including an incident ray and a reflected ray when a ray coincident with the optical axis enters the HOE 83. Further, in a plane perpendicular to the surface normal at the intersection point of each optical member with the optical axis, a direction perpendicular to the X direction is taken as the Y direction. Then, a direction perpendicular to the X direction and the Y direction is taken as the Z direction. Using such a definition, for example, it includes the normal line of the display element 51 and the normal lines of two parallel planes 81 b and 81 c of the eyepiece optical system 66 described later, and the center of the display plane of the display element 51 The cross section included is the YZ cross section.

The illumination optical system 62 illuminates the display element 51, and includes a light source 71, an illumination mirror 72, and a diffusion plate 73.

The light source 71 is configured by an RGB integrated LED that emits light corresponding to each color of R (red), G (green), and B (blue). The plurality of light emitting points (each light emitting point of RGB) are arranged in a substantially straight line in the horizontal direction (X direction). The wavelength of light emitted from the light source 71 is, for example, the peak wavelength of light intensity and the wavelength width of half light intensity: 462 ± 12 nm (B light), 525 ± 17 nm (G light), 635 ± 11 nm (R light) It is. The light source 71 may be a laser light source.

The illumination mirror 72 reflects light (illumination light) emitted from the light source 71 toward the diffusion plate 73 and bends the illumination light so that the optical pupil P and the light source 71 become substantially conjugate in the Y direction. It is an optical element.

The diffusion plate 73 is a one-way diffusion plate that diffuses incident light, for example, 40 ° in the X direction in which a plurality of light emitting points of the light source 71 are arranged, and does not diffuse incident light in the Y direction. The diffusion plate 73 is held on the surface of the polarizing plate 63.

The polarizing plate 63 transmits light of a predetermined polarization direction out of light incident through the diffusion plate 73 and guides the light to the PBS 64.

The PBS 64 reflects the light transmitted through the polarizing plate 63 in the direction of the reflective display element 51, and of the light reflected by the display element 51, light corresponding to the image signal ON (transmitted through the polarizing plate 63) Light is a flat plate-like polarization separation element that transmits light having orthogonal polarization directions), and is attached to a light incident surface 81 a of an eyepiece prism 81 described later of the eyepiece optical system 66.

The display element 51 is a display element that modulates the light from the illumination optical system 62 to display an image, and is formed of, for example, a reflective LCD. The display element 51 may be configured to have a color filter, or may be time-division so that an RGB image corresponding to an emission color is displayed in synchronization with time-division light emission for each of RGB of the light source 71. It may be driven. The display element 51 is disposed such that the longitudinal direction of the rectangular display surface is the X direction and the short direction is the Y direction.

The eyepiece optical system 66 is an optical system for guiding image light from the display element 51 to the pupil (optical pupil P) of the observer, and has non-axially symmetric (non-rotationally symmetric) positive optical power. . The eyepiece optical system 66 includes an eyepiece prism 81, a deflection prism 82, and an HOE 83.

The eyepiece prism 81 internally guides image light incident from the display element 51 via the PBS 64, and transmits light from the outside (external light), so that the upper end of the parallel plate is directed to the upper end It is configured to be thicker and thinner at its lower end toward the lower end.

In the eyepiece prism 81, the surface to which the PBS 64 is attached is the light incident surface 81a on which the image light from the display element 51 is incident, and the two surfaces 81b and 81c located substantially parallel to the optical pupil P and facing each other It is a total reflection surface that guides image light by total reflection. Among them, the surface 81 b on the optical pupil P side also serves as an emission surface of image light diffracted and reflected by the HOE 83.

The eyepiece prism 81 is bonded to the deflection prism 82 with an adhesive so as to sandwich the HOE 83 disposed at the lower end thereof. In the present embodiment, among the surfaces constituting the eyepiece prism 81, the surfaces (light incident surfaces 81a and 81b) through which the image light passes other than the surface 81d in contact with the HOE 83 are flat surfaces, but are curved surfaces The surface may be a combination of a flat surface and a curved surface.

The deflection prism 82 is bonded to the eyepiece prism 81 via the HOE 83 to form a substantially parallel plate. By attaching the deflection prism 82 to the eyepiece prism 81, the refraction when external light passes through the lower end of the eyelid prism 81 can be canceled by the deflection prism 82, and an image observed (visually recognized) as the external world It is possible to prevent distortion from occurring (external image).

The HOE 83 is a volume phase reflective type holographic optical element that is provided in contact with the eyepiece prism 81 and diffracts and reflects the image light guided inside the eyepiece prism 81. HOE 83 diffracts light in three wavelength ranges of, for example, 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) at the peak wavelength of diffraction efficiency and the half wavelength width of diffraction efficiency. (Reflect) That is, the RGB diffraction wavelengths of the HOE 83 substantially correspond to the wavelength of the RGB image light (the emission wavelength of the light source 71).

In the above-described configuration, the light emitted from the light source 71 of the illumination optical system 62 is reflected by the illumination mirror 72 and diffused only in the X direction by the diffusion plate 73, and then only the light of a predetermined polarization direction is polarized 63 passes through. Then, the light transmitted through the polarizing plate 63 is reflected by the PBS 64 and is incident on the display element 51.

In the display element 51, incident light is modulated according to the image signal. At this time, the image light corresponding to the image signal ON is converted into light whose polarization direction is orthogonal to that of the incident light by the display element 51 and emitted, so that it passes through the PBS 64 and enters the eyepiece prism 81. Incident from the surface 81a. On the other hand, since the image light corresponding to the image signal OFF is emitted without the polarization direction being converted by the display element 51, the image light is blocked by the PBS 64 and does not enter the eyepiece prism 81.

In the eyepiece prism 81, the incident image light is totally reflected once by each of the two opposing surfaces 81c and 81b of the eyepiece prism 81, and then enters the HOE 83 where it is diffracted and reflected and emitted from the surface 81b. The pupil P is reached. Therefore, at the position of the optical pupil P, the observer can observe the image displayed on the display element 51 as a virtual image. That is, the observer (worker A) can perform the work while visually recognizing the work content presented in the video.

On the other hand, since the eyepiece prism 81, the deflecting prism 82 and the HOE 83 transmit almost all the external light, the observer can observe the external world by see-through. Therefore, the virtual image of the image displayed on the display element 51 is observed to be superimposed on a part of the outside world in the field of view (view) of the observer. Therefore, the observer (operator A) can perform work such as processing on the processing object at hand observed in the see-through while confirming the work content presented in the video.

In the present embodiment, a reflective LCD is used as the display element 51. However, a transmissive LCD may be used, and the optical design of the image display unit 61 may be changed accordingly.

Third Embodiment
FIG. 17 is an explanatory view showing a schematic configuration of the condition monitoring system 1 of the present embodiment. The condition monitoring system 1 of the present embodiment has the same configuration as that of the second embodiment except that the biological information detection device 40 of the second embodiment is replaced with a biological information detection device 90. The biometric information detection device 90 and the server 30 can communicate via a communication line such as a wireless LAN or by wireless communication based on the Bluetooth (registered trademark) standard. The point that the biological information detection device 90 and the HMD 50 are provided corresponding to the worker A is the same as that of the second embodiment.

FIG. 18 is a perspective view showing the biological information detection apparatus 90 and the HMD 50 in an enlarged manner. The worker A can perform work in a state in which both the biological information detection apparatus 90 and the HMD 50 are attached to the head. At this time, the living body information detection device 90 is attached to the head of the worker A so as to cover at least the back of the worker A and to avoid interference with the HMD 50.

(Configuration of biological information detection device)
FIG. 19 is a block diagram showing the configuration of the main part of the biological information detection apparatus 90. As shown in FIG. The biological information detection apparatus 90 includes a biological information detection unit 91, a storage unit 92, a communication unit 93, and a control unit 94.

The living body information detection unit 91 is configured of a brain potential sensor 91 a that detects a brain potential as living body information that changes with the work of the worker A. The storage unit 92 is a memory for temporarily storing the biological information detected by the biological information detection unit 91 in addition to the operation program of the control unit 94, and includes, for example, a RAM, a ROM, a non-volatile memory, etc. ing. The communication unit 93 is an interface for communicating with the outside (for example, the server 30), and includes a transmission circuit, a reception circuit, an antenna, and the like. The control unit 94 includes, for example, a CPU, and controls the operation of each unit of the biological information detection apparatus 90.

For example, "Mind Wave Mobile" manufactured by Neurosky Inc. can be used as the biological information detection apparatus 90 configured as described above.

(About the state monitoring method)
FIG. 20 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the present embodiment. The state monitoring method of the present embodiment will be described below with reference to FIG.

First, in the HMD 50, the work content stored in the storage unit 52 is displayed on the display element 51 under the control of the control unit 54 (# 21). As a result, the worker A can carry out the work according to the display content. Then, the HMD 50 transmits the work name corresponding to the displayed work content as work information to the server 30 (# 22).

On the other hand, the brain potential during the work of the worker A is detected and obtained by the brain potential sensor 91a of the biological information detection apparatus 90 (# 23; biological information detection step). The data of the brain potential of the worker A is transmitted to the server 30 (# 24).

When the server 30 receives the brain potential data transmitted from the biological information detection device 90 (# 25), the biological index acquisition unit 33c determines that the concentration is related to the alpha wave, based on the above data. The value / β wave value is calculated and acquired (# 26; biomarker acquisition step). Specifically, the biological index acquisition unit 33c performs frequency analysis (fast Fourier transform) on time series data of the acquired brain potential, and takes an integrated value of values of 8 Hz to 13 Hz as an alpha wave value, and a value of 13 Hz to 30 Hz. The α wave value / β wave value is calculated, using the integrated value of as the β wave value. Then, the control unit 33 (for example, the overall control unit 33a) combines the work information (work name) transmitted from the HMD 50 in # 22 and the biological index (α wave value / β wave value) acquired in # 26. It associates and stores in the work information / biometric index storage unit 31b (# 27; storage step). For example, when worker A performs work 1 for tightening a screw, α wave value / β wave value is c1, and for work 2 when a hole is formed in a substrate, α wave value / β wave value is c2 (for example, c2 <c1), and information that the α wave value / β wave value is c3 (for example, c3> c1) at the assembly work 3 of the component is the work information / biometric index storage unit 31b Is stored in

As described above, also in the state monitoring system 1 according to the present embodiment, the task information (task name) and the biometric index (α wave value / β wave value) are associated and stored in the task information / bioindex storage unit 31b. Therefore, the third party such as the manager of the work grasps the information stored in the work information / biometric index storage unit 31b by, for example, access from the external terminal device 100 (see FIG. 1). It becomes possible to easily grasp the physical and mental burden of work. That is, in the above example, for the work 2, since the alpha wave value / beta wave value is smaller than the other work, the concentration of the worker A is reduced, and the physical and mental burden is large. It can be easily judged.

Therefore, as in the first or second embodiment, the third party can easily grasp the “comfort of work” for the work currently performed by the worker A. And, by aiming to improve the work as necessary, it is possible to prevent the accident and the disease during the work of the worker A, and the like, it is possible to obtain the same effect as the first embodiment or the second embodiment. In addition, since the third party can confirm by which work the concentration of the worker A is often reduced by appropriately checking the information stored in the work information / biometric index storage unit 31b. Work can be improved with priority to work with reduced concentration.

Further, the biological information detection unit 91 is configured of a brain potential sensor 91a. As a result, the biological index acquisition unit 33 c can reliably calculate and acquire the α wave value / β wave value as the biological index based on the brain potential data acquired by the brain potential sensor 91 a.

Further, as described above, it is considered that the concentration of worker A decreases as the α wave value / β wave value decreases, so the α wave value / β wave value determines the concentration of worker A. It can be said that it is a vital indicator for Since such a biometric index is stored in the work information / biometric index storage unit 31b, the third party uses the above-mentioned biometric index stored in the work information / biometric index storage unit 31b to determine the concentration of the worker A. It is possible to accurately determine the physical and mental burden on the worker by making an accurate judgment.

In the present embodiment, an example in which the calculation of the α wave value / β wave value is performed on the side of the server 30 has been described. For example, an arithmetic function is added to the living body information detection device 90, and the living body information detection device 90 calculates the α wave value / β wave value based on the brain potential data, and then sends the calculation data to the server 30. You may make it transmit. That is, the control unit 94 of the biological information detection apparatus 90 may have the function of the biometric index acquisition unit 33 c of the server 30.

Further, in the present embodiment, the alpha wave value / beta wave value is used as the biological index, but it is assumed that alpha wave value / beta wave value (hereinafter, alpha wave value / beta wave value ref) when operator A is at rest The ratio of α wave value / β wave value (hereinafter referred to as α wave value / β wave value work) to α wave value / β wave value ref during the work of worker A, ie, (Α wave value / β wave value work) / (α wave value / β wave value ref) may be used as the biological index.

Therefore, as in the present embodiment, when the biological information detection unit 91 is configured of the brain potential sensor 91a, the biological index is an α wave value / β obtained based on the brain potential detected by the brain potential sensor 91a. It can be said that the wave value or the ratio of the α wave value / β wave value to the reference value ((α wave value / β wave value work) / (α wave value / β wave value ref)) may be used. In this case, the third party ensures the physical and mental burden of the worker A on the work of the worker A performed at the work site on the basis of the biometric index regardless of using any of the above-mentioned biomarkers. It is possible to

(Modification)
FIG. 21 is an explanatory view showing the configuration of a modification of the state monitoring system 1 of the present embodiment. The state monitoring system 1 of this modification corresponds to a configuration in which the terminal device 100 is added to the configuration of FIG. 17 described above. The terminal device 100 and the server 30, and the terminal device 100 and the HMD 50 can communicate via a communication line such as a wireless LAN or by wireless communication based on the Bluetooth (registered trademark) standard.

(Configuration of terminal device 100)
FIG. 22 is a block diagram showing the configuration of the terminal device 100. As shown in FIG. The terminal device 100 includes a display unit 101, an input unit 102, a storage unit 103, a communication unit 104, and a control unit 105. The display unit 101 is configured of a display that displays various information. The input unit 102 includes a keyboard, a mouse, a touch pad, and the like that receive an input (instruction) by a third party. The storage unit 103 is a memory that stores an operation program of the control unit 105 and various information, and is configured of, for example, a RAM, a ROM, a non-volatile memory, a hard disk, and the like. The communication unit 104 is an interface for communicating with the outside, and includes a transmission circuit, a reception circuit, an antenna, and the like.

The control unit 105 includes an overall control unit 105 a and a comparison processing unit 105 b. The overall control unit 105a and the comparison processing unit 105b may be integrally configured by one CPU, or may be configured by separate CPUs. The overall control unit 105 a controls the operation of each unit of the terminal device 100. The comparison processing unit 105 b compares the biometric index (for example, α wave value / β wave value) transmitted from the server 30 with the tolerance range determined based on the preset threshold value for each operation, and the biometric index is the tolerance range If it is outside, it generates and outputs instruction information for instructing a worker to display a prompt for refreshing.

Such a terminal device 100 can be configured by, for example, a personal computer. The terminal device 100 may be configured separately from the server 30 as shown in FIG. 21, or may be configured integrally with the server 30, although not shown.

FIG. 23 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the modified example. In this modification, in addition to the processing shown in FIG. 20, the following processing is further performed.

It is assumed that the server 30 holds the threshold (biometric index threshold) of the alpha wave value / beta wave value for each operation set in advance in the storage unit 31 (# 28). The threshold value can be set, for example, as a value obtained by multiplying an α wave value / β wave value ref set in advance as a reference value by a predetermined ratio. The predetermined ratio is a ratio that indicates an increase in fatigue or a decrease in concentration. For example, since the direction in which the α wave value / β wave value ref decreases indicates the direction in which the concentration power decreases, in the present modification, the above ratio is set to about 0.5 to 0.9, and the α wave is set. A value approximately 0.5 to 0.9 times the value / β wave value ref is used as a threshold value of the α wave value / β wave value. When (α wave value / β wave value work) / (α wave value / β wave value ref) is used as a biological index, (α wave value / β wave value work) / (α wave value / β) A value about 0.5 to 0.9 times the wave value ref) may be used as a biomarker threshold, and the tolerance range of the biomarker may be set based on the biomarker threshold.

The biometric index (α wave value / β wave value) calculated in # 26 and the biometric index threshold value (α wave value / β wave value threshold) held in # 28 are terminals via the communication unit 32. It is sent to the device 100. Then, the control unit 105 (in particular, the comparison processing unit 105b) of the terminal device 100 calculates an allowable range of the biomarker based on the biomarker threshold, and compares the transmitted biomarker with the allowable range ((1) # 29). For example, since the direction in which the alpha wave value / beta wave value as the biometric index decreases indicates a decrease in concentration, the comparison processing unit 105b sets the range larger than the biomarker threshold as the allowable range, and the alpha wave value / beta It is determined whether the wave value is out of the above-mentioned allowable range (whether or not the value is below the allowable range) (# 30). If the α wave value / β wave value is within the allowable range, the process returns to # 29 to repeat the process (compare the allowable range with the above-described biometric index acquired for the next operation).

If the alpha wave value / beta wave value is out of the allowable range (if it falls below the allowable range), the comparison processing unit 105 b determines that the concentration of the worker A is lowered, and the operator A is refreshed. The instruction information for instructing to display (restoration of concentration) is transmitted to the HMD 50 (# 31). When the HMD 50 receives the above instruction information from the terminal device 100, the control unit 54 controls the display element 51 to prompt information such as "Please take a break." Or "Please perform stretching." Is displayed (# 32).

As described above, the comparison processing unit 105 b of the terminal device 100 compares the biometric index (α wave value / β wave value) transmitted from the server 30 with the allowable range for each operation, and the biometric index is out of the allowable range. If it is, the instruction information for instructing the worker A to display a request for refreshing is output to the HMD 50 (# 29 to # 31). Then, the HMD 50 performs a display for prompting the worker A to be refreshed by the display element 51 based on the above instruction information (# 32). As a result, the worker A can look at the image (refresh instruction information) displayed on the HMD 50, take appropriate measures such as taking a break, and can restore concentration. That is, according to the above-mentioned processing, the increase in fatigue and the decrease in concentration are monitored, and when they are detected, the information is fed back to the worker A in real time to promote the recovery of the fatigue and the concentration. This can improve the workability.

In particular, since the HMD 50 as a display unit is a head mounted image display device mounted on the head of the worker A, the worker A wears the HMD 50 on the head and performs work That is, the instruction information displayed on the HMD 50 can be easily confirmed without causing any trouble in the operation.

From the above, in the state monitoring method of the present modification, the comparison processing step (# 29, # 30) comparing the above-described biometric index with the allowable range determined based on the preset threshold value for each operation (# 29, # 30) And an output step (# 31) for outputting instruction information for instructing the operator to display a refresh when the biometric index is out of the allowable range, and a display for prompting the operator to refresh based on the instruction information. And displaying step (# 32). As a result, the worker can see an indication prompting the user to refresh (in the HMD 50), take appropriate measures such as taking a break, and can restore concentration.

Fourth Embodiment
FIG. 24 is an explanatory view showing a schematic configuration of the condition monitoring system 1 of the present embodiment. The condition monitoring system 1 of the present embodiment is an embodiment except that the biological information detection device 40 of the second embodiment is replaced with the biological information detection device 20 of the first embodiment, and a bar code reader 120 is further provided. The configuration is the same as that of No. 2. The biological information detection apparatus 20 and the server 30, the HMD 50 and the server 30, the barcode reader 120 and the server 30, and the barcode reader 120 and the HMD 50 communicate with each other via a communication line such as a wireless LAN or Bluetooth (registered trademark) standard. It can communicate by wireless communication based on.

FIG. 25 is a block diagram showing a schematic configuration of the barcode reader 120. As shown in FIG. The barcode reader 120 is a worker identification unit that identifies a worker by reading a worker's identification information (for example, a personal ID; identification). In particular, the barcode reader 120 identifies a worker by reading barcode information as identification information assigned to each worker. Such a barcode reader 120 includes a reading unit 121, a storage unit 122, a communication unit 123, and a control unit 124.

The reading unit 121 is configured to have a light source, a light receiving element, and the like, irradiates light to the bar code, and reads the bar code by receiving the reflected light, and the information attached to the bar code (bar code Get information). The storage unit 122 is a memory for temporarily storing bar code information acquired by the reading unit 121 in addition to the operation program of the control unit 124, and includes, for example, a RAM, a ROM, a non-volatile memory, etc. There is. The communication unit 123 is an interface for communicating with the outside (for example, the server 30 and the HMD 50), and includes a transmission circuit, a reception circuit, an antenna, and the like. The control unit 124 includes, for example, a CPU, and controls the operation of each unit of the barcode reader 120.

In the present embodiment, the barcode reader 120 is used as the worker identification unit, but for example, an IC chip or a matrix type two-dimensional code (QR code (registered trademark)) in which the worker personal ID is recorded is used. An apparatus or device for specifying a worker by reading may be used as the worker specifying unit. In addition, a device capable of performing biometric authentication for identifying a worker by detecting a fingerprint of the worker may be used as the worker identification unit.

FIG. 26 is an explanatory view showing a rough flow of processing in the condition monitoring system 1 of the present embodiment. Hereinafter, the state monitoring method of the present embodiment will be described with reference to FIG.

First, before the start of the operation, the worker A holds the bar code, to which the identification information (personal ID) of the worker A is attached, attached to the employee ID card of the worker A, to the barcode reader 120. Then, the reading unit 121 of the barcode reader 120 reads the barcode and acquires barcode information of the operator A (# 41). Then, the control unit 124 identifies the worker A from the barcode information acquired by the reading unit 121 (# 42). That is, at # 42, the control unit 124 acquires worker information including the worker name of the worker A. The worker information of the worker A identified is transmitted to the server 30 via the communication unit 123 (# 43).

The bar code information acquired in # 41 is transmitted to the HMD 50 via the communication unit 123. Under the control of the control unit 54, the HMD 50 reads out information on the work content of the worker A corresponding to the received barcode information from the storage unit 52, and causes the display element 51 to display the work content (# 44). As a result, the worker A can carry out the work according to the display content. Further, the HMD 50 transmits a work name corresponding to the displayed work content as work information to the server 30 (# 45).

On the other hand, the heart rate during the work of the worker A is detected and acquired by the heart rate sensor 21a (see FIG. 4) of the biological information detection apparatus 20 (# 46; biological information detection step). The heart rate data of the worker A is transmitted to the server 30 (# 47).

When the server 30 receives the heart rate data transmitted from the living body information detection device 20 (# 48), the living body index acquisition unit 33c determines, as a living body index, the parasympathetic nerve related to concentration based on the above data. The degree of activity is calculated and acquired (# 49; biomarker acquisition step). The method of calculating the parasympathetic nervous activity is the same as that of the first embodiment. Then, the control unit 33 (for example, the overall control unit 33a) receives the worker information (worker name) transmitted from the bar code reader 120 at # 43 and the work information (work name) transmitted from the HMD 50 at # 45. And the biometric index (parasympathetic nerve activity degree) acquired in # 49 are stored in the work information / bioindex storage unit 31b (# 50; storage step).

As described above, in the state monitoring system 1 according to the present embodiment, the worker information (worker name), the work information (work name), and the biometric index (parasympathetic nerve activity level) are associated with each other to create the work information / biometric index As stored in the storage unit 31b, a third party such as a work manager refers to the information stored in the work information / biometric index storage unit 31b by, for example, access from an external terminal device, and performs a specific work. It is possible to easily grasp the physical and mental burden of work on Person A. Therefore, the third party can easily grasp the "comfort of work" about the work currently performed by the specific worker A, and by improving the work as necessary, the specific person A can be identified. The same effects as in Embodiments 1 to 3 can be obtained, such as preventing accidents and diseases during work.

Further, the state monitoring method of the present embodiment includes a worker specifying step (# 41) for specifying the worker A by reading the identification information (for example, bar code information) of the worker A, and the storage step of # 50. Then, the information of the worker A specified in the worker specifying step, the work information transmitted in # 45, and the biometric index transmitted in # 47 are stored in association with each other. As a result, similar to the above, by improving the work as necessary, it is possible to obtain an effect such that an accident or a disease during the work can be prevented for the specific worker A.

In addition, the third party can confirm which work the concentration of the worker A is often reduced by appropriately checking the information stored in the work information / biometric index storage unit 31 b. As a result, the work can be improved with priority over the work in which the concentration power is reduced. In addition, even when, for example, a plurality of workers A perform work sharing, it is possible to share work so that each worker A can handle work that is easy to maintain concentration, and the entire work performed by a plurality of workers A The work efficiency of can be improved.

Further, in the present embodiment, since the barcode reader 120 is used as the worker identification unit, the worker identification unit is realized with a simple configuration, and the state monitoring system 1 of the present embodiment can be easily realized.

(Modification)
FIG. 27 is an explanatory view showing a configuration of a modification of the status monitoring system 1 of the present embodiment. The status monitoring system 1 of the present modification has the terminal device 100 of the third embodiment in the configuration of FIG. It corresponds to the structure which added (refer FIG. 21, FIG. 22).

FIG. 28 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the modified example. In this modification, in addition to the process shown in FIG. 26, the following process is further performed. In FIG. 28, for convenience, illustration of the process by the barcode reader 120 in FIG. 26 is omitted.

It is assumed that the server 30 holds the threshold (biometric index threshold) of the parasympathetic nerve activity for each worker A set in advance in the storage unit 31 (# 51). The threshold value can be set, for example, as a value obtained by multiplying a predetermined ratio of the parasympathetic activity ref set in advance as a reference value. The predetermined ratio is a ratio that indicates an increase in fatigue or a decrease in concentration. For example, since the direction in which the parasympathetic activity ref decreases indicates the direction in which the concentration declines, in the present embodiment, the ratio is set to about 0.5 to 0.9, and the parasympathetic activity ref is set. A value of about 0.5 to 0.9 times the value of is taken as a threshold of the parasympathetic nervous activity.

The vital sign (parasympathetic nerve activity) for worker A stored in # 50 and the vital sign threshold for each worker A held in # 51 (threshold for parasympathetic nerve activity) are the communication units. 32 to the terminal device 100. Then, the control unit 105 of the terminal device 100 (in particular, the comparison processing unit 105b, refer to FIG. 22) calculates an allowable range of the biomarker based on the biomarker threshold, and transmits the transmitted biomarker and the allowable range. Compare (# 52). For example, since the direction in which the parasympathetic nerve activity as the biomarker decreases shows a decrease in concentration, the comparison processing unit 105b sets the range larger than the biomarker threshold as the allowable range, and the parasympathetic nerve activity is in the allowable range. It is determined whether it is outside (whether or not it is below the allowable range) (# 53). If the parasympathetic nerve activity is within the allowable range, the process returns to # 52 to repeat the processing (compare the allowable range with the above-described biometric index acquired for the next task).

If the parasympathetic nerve activity is out of the allowable range (if it falls below the allowable range), the comparison processing unit 105 b determines that the concentration of the worker A is reduced, and refreshes the worker A (the concentration The instruction information for instructing the display prompting the recovery) is transmitted to the HMD 50 (# 54). When the HMD 50 receives the above instruction information from the terminal device 100, the control unit 54 controls the display element 51 to prompt information such as "Please take a break." Or "Please perform stretching." Is displayed (# 55).

As described above, in the present modification, the comparison processing unit 105 b of the terminal device 100 is based on the biometrics index (parasympathetic nerve activity level) of the worker A transmitted from the server 30 and the biometrics threshold for each worker A. Compare the tolerance with the tolerance determined, and when the biomarker for worker A is out of tolerance, output instruction information to instruct worker A to display a prompt for refreshing to HMD 50 (# 52 to # 54) . Then, the HMD 50 performs a display for prompting the worker A to refresh on the display element 51 based on the above instruction information (# 55). Thus, as in the modification of the third embodiment, in a state in which the worker A wears the HMD 50 and performs work, the worker A checks the displayed image (refresh instruction information) and takes a break, etc. It is possible to take appropriate measures and restore the concentration appropriately for each worker A.

It should be noted that the calculation of the parasympathetic nerve activity as a biological index may be performed by other than the server 30, and the ratio of the parasympathetic activity work during work of worker A to the parasympathetic activity ref as a biological index The point which may be used is the same as that of the first embodiment. Also, a value of about 0.5 to 0.9 times the value of (parasympathetic nerve activity work) / (parasympathetic nerve activity ref) is set as a biomarker threshold for each worker A, and based on the biomarker threshold, The allowable range of the biomarker may be set for each worker A.

From the above, in the state monitoring method of the present modification, the comparison processing step (# 52, # 53) comparing the above-mentioned biometric index with the allowable range determined based on the preset threshold value for each worker (# 52, # 53 And, an output step (# 54) for outputting instruction information for instructing the operator to display a refresh when the biometric index is out of the allowable range, and urging the worker to refresh based on the instruction information. It can be said that the display process (# 55) which displays is included. As a result, the worker can see an indication prompting the user to refresh (in the HMD 50), take appropriate measures such as taking a break, and can restore concentration.

Fifth Embodiment
FIG. 29 is an explanatory view showing a schematic configuration of the condition monitoring system 1 of the present embodiment. The state monitoring system 1 of the present embodiment is configured by further adding a terminal device 100 a and a printer 130 to the state monitoring system 1 of the second embodiment. The printer 130 can communicate with the terminal device 100a through a communication line such as a wireless LAN or by wireless communication based on the Bluetooth (registered trademark) standard, and an aggregation processing unit 105c described later of the terminal device 100a (see FIG. 30). The output part which outputs the result totaled by) is comprised.

FIG. 30 is a block diagram showing a schematic configuration of the terminal device 100a. The terminal device 100a has the same configuration as the terminal device 100 shown in FIG. 22 except that the control unit 105 further includes a tabulation processing unit 105c. The terminal device 100a can communicate with the server 30 via a communication line such as a wireless LAN or by wireless communication based on the Bluetooth (registered trademark) standard. The terminal device 100 a may be integrated with the server 30.

The tabulation processing unit 105 c periodically or information stored in the work information / biometric index storage unit 31 b of the server 30 (information transmitted from the server 30 when the terminal device 100 a is separate from the server 30) or Summarize at the set period and perform processing to create a report. Such an aggregation processing unit 105c may be configured by the same CPU as the overall control unit 105a and the comparison processing unit 105b, or may be configured by a CPU different from these.

Here, that the tabulation processing unit 105c periodically tabulates the above information means that the information is tabulated at predetermined intervals such as one month or quarter. In addition, tabulating in the set period means tabulating information in the tabulation period when the tabulation period is set and input as necessary using an input unit such as a keyboard. Further, aggregation means, for example, digitizing and summarizing a biomarker for each operation, or creating a graphed report.

FIG. 31 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the present embodiment. In the present embodiment, the processing up to # 17 is the same as the processing in the second embodiment shown in FIG. When the information stored in the work information / biometric index storage unit 31b in # 17 is transmitted from the server 30 to the terminal device 100a (# 18), the tabulation processing unit 105c of the terminal device 100a periodically performs the above information. Tabulate and create a report (# 19). For example, FIG. 32 shows that the tabulation processing unit 105c periodically blinks per unit time as a biometric index for each operation (for example, in one month from April 1, 2017 to April 30, 2017). ) Shows the tabulated results (reports). The result tabulated by the tabulation processing unit 105c is output to the printer 130, where it is printed on a recording medium (for example, paper) (# 20).

As described above, when the storage information of the work information / biometric index storage unit 31b of the server 30 is periodically output by the printer 130, the work manager or the like looks at the contents output by the printer 130, As a daily management, it is possible to easily check whether an abnormality has occurred in some work. Further, for example, a predetermined period before and after the work improvement is set by an input unit (not shown), and the tabulation processing unit 105c of the terminal device 100a tabulates the stored information in each set period, and the tabulated result is a printer If the output is performed at 130, the administrator or the like can simply perform the quantitative evaluation of the improvement effect of the work by comparing the output results in both periods.

In the present embodiment, an example has been described in which the counting processing unit 105 c counts the biometric index (the number of blinks per unit time) for each operation, but, for example, the barcode reader 120 (see FIG. When specifying the worker using the worker specifying unit such as FIG. 24) and associating the worker name, the work name, and the biometric index in the work information / biometric index storage unit 31b, the worker name is stored. The biometric index may be totaled for each time, and the total result may be output by the printer 130.

In the present embodiment, the printer 130 is used as an output unit for outputting the aggregation result. However, the aggregation result may be output by the display unit using a display unit such as a display. At this time, the display unit described above may be configured by the display unit 101 of the terminal device 100a, or may be configured by an external display unit attached to the terminal device 100a.

Sixth Embodiment
FIG. 33 is an explanatory view showing a schematic configuration of the state monitoring system 1 of the present embodiment. The state monitoring system 1 according to the present embodiment includes the server 30, the HMD 50, the biological information detecting device 90, and the terminal device 100, and the state monitoring system 1 according to the third embodiment (modified example) shown in FIG. Although it is common, it differs from the condition monitoring system 1 of the third embodiment in that the HMD 50 and the terminal device 100 function as a remote work support system.

The remote work support system is a system capable of displaying instruction information from a remote place on the HMD 50 and presenting it to the worker A. That is, in the remote work support system, an instruction (work information) on the work content of the worker A is input from the remote terminal device 100 and transmitted to the HMD 50, and the HMD 50 receives the instruction and displays the instruction contents as a video Do. As a result, it is possible to make the worker A visually recognize the displayed instruction content and support the work. Therefore, the terminal device 100 to which the work information of the worker A is input and the HMD 50 for receiving the work information from the terminal device 100 can obtain the work information acquisition unit 200 for acquiring the work information specifying the work of the worker A. It can be said that it is composed.

Here, FIG. 34 is a block diagram showing the configuration of the main part of the HMD 50 of the present embodiment. The HMD 50 of the present embodiment further includes an imaging unit 55 in addition to the configuration of FIG. 14. The imaging unit 55 is a block that captures the front view of the worker A to obtain an image, and includes an imaging lens, an imaging device (for example, CCD, CMOS), a focusing mechanism, an aperture mechanism, a drive circuit, and an A / D conversion circuit. And so on. Data of an image acquired by the imaging unit 55 can be transmitted to the terminal device 100 via the communication unit 53 and the communication line, and as a result, in the terminal device 100, the display unit 101 can display the image. It is possible. Therefore, in the present embodiment, the HMD 50 and the terminal device 100 can communicate in both directions.

As described above, when the HMD 50 and the terminal device 100 can communicate bi-directionally, the image acquired by the imaging unit 55 of the HMD 50 can be shared by the HMD 50 and the terminal device 100. Thereby, for example, the manager of the work or the like connects an instruction (for example, connector A and connector B) to the image while viewing the image displayed on the display unit 101 (see FIG. 22) of the terminal device 100. Etc., and transmits the instruction to the HMD 50, and an image reflecting the above instruction can be displayed on the HMD 50 to enable communication.

In the present embodiment, a personal computer is used as the terminal device 100, but another portable information terminal such as a smartphone or a tablet may be used.

FIG. 35 is an explanatory view showing a rough flow of processing in the state monitoring system 1 of the present embodiment. First, a work instruction is input at the input unit 102 (see FIG. 22) of the terminal device 100 located at a remote place, and transmitted to the HMD 50 (# 61). In the HMD 50, when the work instruction is received (# 62), the content of the work instruction is displayed on the display element 51 (# 63), and it is presented to the worker A as a video, and the instruction content is transmitted to the server 30. (# 64). The above-described processes of # 61 to # 62 correspond to a work information acquisition step of acquiring information (instruction content) specifying the work of the worker A as the work information.

On the other hand, the brain potential during the work of the worker A is detected and obtained by the brain potential sensor 91a of the biological information detecting device 90 (# 65; biological information detecting step). The data of the brain potential of the worker A is transmitted to the server 30 (# 66), and the server 30 performs the same process as in the third embodiment. That is, when the server 30 receives data of brain potentials transmitted from the biological information detection apparatus 90 (# 67), the biological index acquisition unit 33c calculates α wave value / β wave value as a biological index based on the above data And acquire (# 68; biomarker acquisition step). Then, the control unit 33 (for example, the overall control unit 33a) associates the instruction content of the work transmitted from the HMD 50 in # 64 with the biological index (α wave value / β wave value) acquired in # 68. The task information / biometric index storage unit 31b is stored (# 69; storage step).

As described above, in the state monitoring system 1 according to the present embodiment, the HMD 50 performs the work support by acquiring the instruction content input by the terminal device 100 as the work information and presenting the same to the worker A as a video. Thereby, the worker A can appropriately perform the operation based on the instruction from the remote place. In addition, the work information acquisition process includes an instruction content acquisition process (# 62) of acquiring the instruction content input by the terminal device 100 as operation information by the HMD 50. Then, the state monitoring system 1 of the present embodiment further includes an instruction content display step (# 63) for presenting the instruction content acquired in the instruction content acquisition step to the worker A in the form of a video by the HMD 50. Thereby, the same effect as the above can be obtained.

In addition, when performing work based on an instruction from a remote place, the contents instructed from the remote place are often non-stationary work that changes according to the situation of the place. For example, the operation of manufacturing a product while repeating trial and error, such as the manufacture of a product in a development stage, corresponds to the above-mentioned unsteady operation. When moving from the development stage to the stage of mass-producing products in full scale, it is necessary to make the above unsteady work into a manual and drop it into a steady work that anyone can do. Even in such a case, in the present embodiment, since the above-mentioned processing can “visualize” the comfort of work that was non-stationary, it improves the comfort of work as needed, and Work that was targeted can be dropped into routine work.

[Others]
It is also possible to configure the state monitoring system by appropriately combining the configurations of the embodiments described above. For example, it is also possible to configure a state monitoring system by combining the biological information detection device used in any of the embodiments and the work information acquisition unit used in any of the embodiments.

Further, as described in the third embodiment, the configuration in which the terminal device 100 issues a refresh instruction to the HMD 50 is applied to another embodiment, or the worker using the barcode reader 120 of the fourth embodiment. It is of course possible to apply the configuration to specify the above to the other embodiments, or to apply the configuration concerning aggregation and output of the biomarkers shown in the fifth embodiment to the other embodiments.

In addition, the biological information detection unit is not limited to the heartbeat sensor, the electro-oculography sensor, or the brain potential sensor described above, and a sensor for detecting the bio-information that can acquire a bio-indicator for determining fatigue or concentration is used. Just do it.

The state monitoring system and the state monitoring method of each embodiment described above may be expressed as follows.

That is, the state monitoring system described above is a state monitoring system that monitors the state of a worker who works at a work site, and uses information specifying the work performed by the worker at the work site as work information It becomes an index for judging the state of the worker based on the work information acquisition unit to be acquired, the living body information detection unit for detecting the living body information which changes along with the work of the worker, and the living body information A biometric index acquiring unit that acquires a biometric index, and a work information / bioindex storage unit that associates and stores the work information and the biometric index.

The biological information detection unit may be any of a heart rate sensor, an electro-oculogram sensor, and a brain-potential sensor.

The biometric index may be an index for determining fatigue or concentration of the worker.

The biological information detection unit is a heart rate sensor, and the biological index is a parasympathetic nerve activity obtained based on a heart rate detected by the heart rate sensor, or a ratio of the parasympathetic nerve activity to a reference value May be

The biological information detection unit may be an electro-oculogram sensor, and the bio-index may be the number of blinks per unit time detected by the electro-oculogram sensor or a ratio of the number of blinks to a reference value.

The biological information detection unit is a brain potential sensor, and the biological index is a ratio of an alpha wave value to a beta wave value obtained based on a brain potential detected by the brain potential sensor, or a value of the ratio It may be a ratio to a reference value.

The work information acquisition unit identifies a work name of the work based on an imaging unit that captures the work of the worker and acquires an image, the image, and electronic data indicating a procedure of the work. The information processing apparatus may further include a specific processing unit that acquires the work name as the work information.

The work information acquisition unit previously acquires, as the work information, the work content to be presented to the worker and the work name corresponding to the work content, and the head sequentially presents the work content to the worker by video. It may be configured of a part-mounted image display device.

The work information acquisition unit includes a head-mounted image display device and an external terminal device capable of communicating with the head-mounted image display device, and the head-mounted image display device includes the external terminal device. The input instruction content may be acquired as the work information, and may be presented as a video to the worker.

The above-mentioned state monitoring system further includes a worker specifying unit for specifying the worker by reading the worker identification information, and the work information / biometric index storage unit is specified by the worker specifying unit. The information on the worker, the work information, and the biometric index may be stored in association with each other.

The worker identification unit may identify the worker by reading bar code information as the identification information assigned to each worker.

The state monitoring system described above outputs an aggregation processing unit that aggregates information stored in the work information / biometric index storage unit periodically or in a set period, and the result of aggregation by the aggregation processing unit. And an output unit.

The condition monitoring system described above compares the biometric index with an allowable range determined based on a preset threshold value for each operation, and refreshes the worker when the biometric index is out of the allowable range. And a display unit for performing a display for prompting the worker to refresh based on the instruction information.

The condition monitoring system described above compares the biometric indicator with a tolerance determined based on a preset threshold value for each worker, and when the biometric indicator is out of the tolerance, the operator The display device may include a comparison processing unit that outputs instruction information for instructing a display prompting for refreshing, and a display unit for performing a display for prompting the worker to refresh based on the instruction information.

The display unit may be a head mounted image display device mounted on the head of the worker.

The state monitoring method described above is a state monitoring method for monitoring the state of a worker who performs work at a work site, and information specifying the work performed by the worker at the work site is used as the work information. It becomes an index for judging the state of the worker based on the work information acquisition process to acquire, the living body information detection process of detecting the living body information which changes along with the work of the worker, and the living body information It includes a biometric index acquisition step of acquiring a biomarker, and a storage step of associating and storing the work information and the biometric index.

The above state monitoring method further includes an imaging step of capturing the work of the worker and acquiring an image, and in the work information acquisition step, based on the image and electronic data indicating a procedure of the work. The work name of the work may be specified, and the work name may be acquired as the work information.

The state monitoring method described above includes the task contents shown by the head-mounted type video display device which acquires in advance the task content to be presented to the worker and the task name corresponding to the task content as the task information. And the output step of outputting the work information from the head-mounted image display device, and the storage step outputs the information from the head-mounted image display device. The task information may be stored in association with the biometric index.

The work information acquisition process further includes an instruction contents acquisition process of acquiring, as the work information, the instruction content input by the external terminal apparatus by the head mounted video display apparatus, the state monitoring method includes the instruction content acquisition The method may further include an instruction content display step of presenting the instruction content acquired in a step to the operator in the form of a video by the head mounted video display device.

The above state monitoring method further includes a worker identification step of identifying the worker by reading the identification information of the worker, and in the storage step, the worker identification process identified in the worker identification step Information, the work information, and the biometric index may be associated and stored.

As mentioned above, although embodiment of this invention was described, the scope of the present invention is not limited to this, and it can extend or change and carry out within the range which does not deviate from the main point of invention.

The present invention can be used, for example, in a state monitoring system and a state monitoring method for monitoring the state of a worker who works at a work site.

1 Condition Monitoring System 11 Imaging Unit (Work Information Acquisition Unit)
21 biometric information detection unit 21a heart rate sensor 31b work information / biometric index storage unit 33b specific processing unit (work information acquisition unit)
33c biometric index acquisition unit 34 work information acquisition unit 41 biometric information detection unit 41a electrooculogram sensor 50 HMD (head mounted image display device, display unit)
50a Work Information Acquisition Unit 91 Biological Information Detection Unit 91a Brain Potential Sensor 100 Terminal Device (External Terminal Device)
105b comparison processing unit 105c tabulation processing unit 120 bar code reader (worker identification unit)
130 Printer (output unit)
200 Work Information Acquisition Department

Claims (20)

A condition monitoring system for monitoring the condition of a worker who works at a work site, comprising:
A work information acquisition unit which acquires, as work information, information specifying the work performed by the worker at the work site;
A biological information detection unit that detects biological information that changes as the worker performs the work;
A biometric index acquisition unit that acquires a biometric index that is an index for determining the state of the worker based on the biological information;
A state monitoring system, comprising: work information / biometric index storage unit that associates and stores the work information and the biometric index. The state monitoring system according to claim 1, wherein the biological information detection unit is any one of a heart rate sensor, an electro-oculography sensor, and a brain-potential sensor. The condition monitoring system according to claim 1, wherein the biological index is an index for determining fatigue or concentration of the worker. The biological information detection unit is a heart rate sensor,
The condition monitoring system according to claim 1, wherein the vital sign is a parasympathetic nerve activity obtained based on a heart rate detected by the heart rate sensor, or a ratio of the parasympathetic nerve activity to a reference value. The biological information detection unit is an electro-oculogram sensor,
The condition monitoring system according to claim 1, wherein the biological index is the number of blinks per unit time or the ratio of the number of blinks to a reference value detected by the electro-oculography sensor. The biological information detection unit is a brain potential sensor,
The said biological index is a ratio of an alpha wave value to a beta wave value obtained based on a brain potential detected by the brain potential sensor, or a ratio of the value of the ratio to a reference value. Condition monitoring system. The work information acquisition unit
An imaging unit that captures the work of the worker and acquires an image;
The image processing system according to claim 1, further comprising: an identification processing unit that identifies an operation name of the operation based on the image and electronic data indicating the procedure of the operation, and acquires the operation name as the operation information. The condition monitoring system according to any one of 6. The work information acquisition unit previously acquires, as the work information, the work content to be presented to the worker and the work name corresponding to the work content, and the head sequentially presents the work content to the worker by video. The state monitoring system according to any one of claims 1 to 6, wherein the state monitoring system comprises a part-mounted type video display device. The work information acquisition unit
A head-mounted image display device,
An external terminal device capable of communicating with the head mounted image display device;
The state monitoring according to any one of claims 1 to 6, wherein the head-mounted image display device acquires an instruction content input by the external terminal device as the work information, and presents the content as an image to the worker. system. The system further includes a worker identification unit that identifies the worker by reading the worker identification information.
The work information / biometric index storage unit associates and stores the information of the worker specified by the worker identification unit, the work information, and the biometric index. State monitoring system as described. The condition monitoring system according to claim 10, wherein the worker specifying unit specifies the worker by reading bar code information as the identification information assigned to each worker. An aggregation processing unit that aggregates information stored in the work information / biometric index storage unit periodically or in a set period;
The state monitoring system according to any one of claims 1 to 11, further comprising: an output unit configured to output the results aggregated by the aggregation processing unit. An instruction to compare the biometric index with an allowable range determined based on a preset threshold value for each operation, and to instruct a display prompting the operator to refresh when the biometric index is out of the allowable range A comparison processing unit that outputs information;
The state monitoring system according to any one of claims 1 to 12, further comprising: a display unit that performs a display for prompting the worker to refresh based on the instruction information. The biometric index is compared with an allowable range determined based on a preset threshold value for each worker, and when the biometric index is out of the allowable range, a display prompting the worker to refresh is instructed. A comparison processing unit that outputs instruction information;
12. The state monitoring system according to claim 10, further comprising: a display unit that performs a display for prompting the worker to refresh based on the instruction information. The condition monitoring system according to claim 13, wherein the display unit is a head mounted image display device mounted on a head of the worker. A state monitoring method for monitoring the state of a worker who works at a work site, comprising:
A work information acquisition step of acquiring, as work information, information specifying the work performed by the worker at the work site;
A biological information detection step of detecting biological information that changes along with the work of the worker;
A biometric index acquisition step of acquiring a biometric index that is an index for determining the state of the worker based on the biological information;
A state monitoring method, including a storage step of associating and storing the work information and the biometric index. The method further includes an imaging step of capturing the work of the worker and acquiring an image;
The work information acquisition step specifies a work name of the work based on the image and electronic data indicating the procedure of the work, and acquires the work name as the work information. State monitoring method. An operation of sequentially presenting the work content to the worker in the form of a video by the head-mounted image display device which acquires in advance the task content to be presented to the worker and a task name corresponding to the task content as the task information Content display process,
Further including an output step of outputting the work information from the head mounted image display device;
The state monitoring method according to claim 16, wherein, in the storage step, the work information output from the head-mounted image display device is stored in association with the biometric index. The work information acquisition step further includes an instruction content acquisition step of acquiring the instruction content input by the external terminal device as the operation information by the head mounted video display device.
17. The method according to claim 16, wherein the state monitoring method further includes an instruction content display step of presenting the instruction content acquired in the instruction content acquisition step to the operator as a video by the head mounted video display device. State monitoring method. The method further includes a worker specifying step of specifying the worker by reading the worker identification information.
The state monitoring method according to any one of claims 16 to 19, wherein, in the storing step, the information of the worker specified in the worker specifying step, the work information, and the biometric index are stored in association with each other. .

Images