JP2018179344A - Refrigerator and refrigerator system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a refrigerator capable of supporting appropriate two-way communication between a user of a refrigerator and a person watching the refrigerator without requiring special operation of the user of the refrigerator. A refrigerator according to the present invention includes an opening/closing detection unit for detecting opening/closing of a door of a storage room, a data acquisition unit for externally acquiring voice data and a reproduction order designation data of the voice data, and a reproduction order designation data. Based on the following, the reproduction sound determination means for determining the next reproduction sound data, which is the sound data to be reproduced when the opening/closing detection means detects the opening/closing of the door, and the reproduction sound when the opening/closing detection means detects the opening/closing of the door. The sound reproducing means for reproducing the next reproduced sound data determined by the determining means, the sound recording means for recording the sound when the opening/closing detecting means detects the opening/closing of the door, and the sound recorded by the sound recording means as the target sound data to the outside. Data transmitting means for transmitting. [Selection diagram] Fig. 5

Description

  The present invention relates to a refrigerator and a refrigerator system.

  It has a refrigerator equipped with a door open / close sensor and voice input / output means, and an external device connectable to the refrigerator via the Internet, and receives transmission of a door open / close detection signal acquired by the door open / close sensor, voice input / output The initial utterance means performing speech control to the subject of safety confirmation through the means, the safety information acquisition means for acquiring the voice data inputted by the subject to the voice input / output means in response thereto, and analyzing the voice data Safety information by recognizing the safety condition and generating safety information, safety information is stored in the external device, connected to the external device via the Internet, and safety information can be browsed for safety information A safety confirmation system provided with confirmation means is known (see, for example, Patent Document 1). In the safety confirmation system of Patent Document 1, the initial utterance means has message information, and the utterance content by the initial utterance means includes the utterance content corresponding to the message information.

JP, 2016-148991, A

  As described above, the safety confirmation system including the refrigerator and the refrigerator as disclosed in Patent Document 1 can utter message information to the user of the refrigerator who is the target person of the safety confirmation. However, when there is a plurality of message information, there is a possibility that the message information may not be uttered in accordance with the intention of the person who watches the safety of the user of the refrigerator. On the other hand, if only one message information is transmitted, the next message information can not be transmitted until the message information is uttered to the user of the refrigerator. For this reason, it is difficult to support appropriate two-way communication between the user of the refrigerator and the person watching the safety.

  The present invention has been made to solve such problems. The purpose of the person who watches the refrigerator user's intention is only the daily operation of the user of the refrigerator to open and close the refrigerator door even if there are multiple voice information to be transmitted to the user of the refrigerator. Voice information can be played back to the user of the refrigerator, and proper two-way communication between the user of the refrigerator and the person watching over it without the need for special operation of the user of the refrigerator etc. To obtain a refrigerator and a refrigerator system capable of supporting communication.

  The refrigerator according to the present invention is based on open / close detection means for detecting opening / closing of a door of a storage room, data acquisition means for externally acquiring audio data and reproduction order designation data of the audio data, and the reproduction order designation data. Next, when the open / close detection unit detects the opening / closing of the door, the reproduction voice determination unit determines the next reproduced voice data which is the audio data to be reproduced when the opening / closing detection unit detects the opening / closing of the door A sound reproducing unit for reproducing the next reproduction sound data determined by the reproduction sound determining unit; a recording unit for recording a sound when the open / close detection unit detects the opening / closing of the door; And data transmitting means for transmitting voice as target voice data to the outside.

  Further, a refrigerator system according to the present invention includes the refrigerator configured as described above, a server device provided to be communicable with the refrigerator, and a terminal device provided to be communicable with the server device, The server device includes storage means for storing the voice data recorded by the terminal device and the reproduction order designation data generated by the terminal device, and the data acquisition means is stored in the storage means. The voice data and the reproduction order designation data are obtained from the server device, the storage means further stores the target voice data transmitted by the data transmission means, and the terminal device is stored in the storage means The target audio data can be acquired and reproduced.

  In the refrigerator and the refrigerator system according to the present invention, the daily operation of the refrigerator user to open and close the door of the refrigerator even when there is a plurality of voice information to be transmitted to the user of the refrigerator from the person watching over the user of the refrigerator The voice information according to the intention of the watcher can be reproduced to the user of the refrigerator, and the user of the refrigerator and the watcher can watch it without requiring any special operation of the user of the refrigerator. The effect of being able to support appropriate two-way communication between is produced.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the structure of the refrigerator system provided with the refrigerator which concerns on Embodiment 1 of this invention. It is a front view of the refrigerator concerning Embodiment 1 of this invention. It is a longitudinal cross-sectional view of the refrigerator concerning Embodiment 1 of this invention. It is a block diagram which shows the structure of the control system of the refrigerator which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the control apparatus of FIG. It is a block diagram which shows the structure of the terminal device with which the refrigerator system which concerns on Embodiment 1 of this invention is equipped. It is a block diagram which shows the structure of the server apparatus with which the refrigerator system which concerns on Embodiment 1 of this invention is provided. It is a figure which shows typically an example of the screen display of the terminal device which concerns on Embodiment 1 of this invention. It is a figure which shows typically an example of the screen display of the terminal device which concerns on Embodiment 1 of this invention. It is a figure which shows typically an example of the screen display of the terminal device which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of audio | voice reproduction | regeneration and sound recording operation of the refrigerator which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of the image transmission operation | movement of the refrigerator which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of the sound recording operation | movement of the terminal device which concerns on Embodiment 1 of this invention.

  An embodiment for carrying out the present invention will be described with reference to the attached drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and overlapping descriptions will be appropriately simplified or omitted. The present invention is not limited to the following embodiments, and various modifications can be made without departing from the spirit of the present invention.

Embodiment 1
1 to 13 relate to the first embodiment of the present invention, and FIG. 1 is a block diagram showing a configuration of a refrigerator system provided with a refrigerator, FIG. 2 is a front view of the refrigerator, and FIG. 4 is a block diagram showing the configuration of the control system of the refrigerator, FIG. 5 is a block diagram showing the configuration of the control device of FIG. 4, FIG. 6 is a block diagram showing the configuration of a terminal device provided in the refrigerator system, FIG. FIG. 8 is a block diagram showing an example of the screen display of the terminal device. FIG. 9 is a view schematically showing an example of the screen display of the terminal device. 11 schematically shows an example of screen display of the terminal device, FIG. 11 is a flow chart showing an example of voice reproduction and recording operation of the refrigerator, FIG. 12 is a flow chart showing an example of image transmission operation of the refrigerator, FIG. An example of voice recording operation of the device is shown It is a low view.

  As shown in FIG. 1, a refrigerator system provided with a refrigerator 100 according to Embodiment 1 of the present invention includes a refrigerator 100, a terminal device 200, and a server device 300. The refrigerator 100 and the server device 300 are communicably connected to each other. The terminal device 200 and the server device 300 are communicably connected to each other. Although not illustrated here, the refrigerator 100 and the terminal device 200 may be communicably connected to each other.

  The terminal device 200 is mainly used for a user (hereinafter, referred to as a "watching user") who watches the living condition of a user of the refrigerator 100 (hereinafter, referred to as a "fridge user"). The refrigerator 100 is installed, for example, in the home of a refrigerator user. The terminal device 200 is, for example, a personal computer (PC) installed in the home of the watching user. Besides, for example, the terminal device 200 may be a mobile terminal such as a smartphone or a tablet terminal used by the watching user.

  The server device 300 is, for example, a computer or the like installed in the home of a refrigerator user or in the home of a watching user. In addition, the server apparatus 300 may be installed in the place which is neither a refrigerator user's house interior nor a watching user's house interior.

  The number of the refrigerator 100 communicably connected to the server device 300 is not limited to one, and may be plural. Similarly, the number of terminal devices 200 communicably connected to the server device 300 is not limited to one, and may be plural. Further, the server device 300 may be a cloud server composed of a plurality of servers (server group).

  First, the configuration of the refrigerator 100 according to the first embodiment of the present invention will be described with reference to FIGS. 2 to 4. The refrigerator 100 has the heat insulation box 1 as shown in FIG. The front surface (front) of the heat insulation box 1 is opened, and a storage space is formed inside. The heat insulation box 1 has an outer case, an inner case and a heat insulating material. The outer box is made of steel. The inner box is made of resin. The inner box is placed inside the outer box. The heat insulating material is, for example, urethane foam, and is filled in the space between the outer case and the inner case. The storage space formed inside the heat insulation box 1 is divided by the one or more partition members into a plurality of storage rooms for storing food.

  As shown in FIG.2 and FIG.3, the refrigerator 100 is provided with the refrigerator compartment 10, the switching chamber 20, the icemaker 30, the freezer compartment 40, and the vegetable compartment 50 here as some storage compartments, for example. These storage chambers are arranged in four stages in the vertical direction in the heat insulation box 1.

  The refrigerator compartment 10 is disposed at the top of the heat insulation box 1. The switching chamber 20 is disposed on the left and right sides below the refrigerator compartment 10. The cold storage temperature zone of the switching chamber 20 can be switched by selecting one of a plurality of temperature zones. A plurality of temperature zones that can be selected as the cold storage temperature zone of the switching chamber 20 include, for example, a freezing temperature zone (for example, about -18 ° C), a refrigeration temperature band (for example, about 3 ° C), a chilled temperature zone (for example, about 0 ° C) It is a soft freezing temperature zone (for example, about -7 ° C). The ice making chamber 30 is disposed adjacent to the side of the switching chamber 20 and in parallel with the switching chamber 20, that is, on the other side of the refrigerator chamber 10 below the refrigerating chamber 10.

  The freezing room 40 is disposed below the switching room 20 and the ice making room 30. The freezer compartment 40 is mainly used for freezing storage objects for a relatively long period of time. The vegetable room 50 is disposed at the lowermost stage below the freezing room 40. The vegetable room 50 is mainly for storing vegetables, large-sized (for example, 2 L etc.) large plastic bottles, and the like.

  At an opening formed on the front of the refrigerator compartment 10, a rotary refrigerator compartment door 7 for opening and closing the opening is provided. Here, the refrigerator compartment door 7 is a double door type (a double door type), and is comprised by the right door 7a and the left door 7b. An operation panel 6 is provided on the outer surface of the refrigerator compartment door 7 (for example, the left door 7 b) on the front surface of the refrigerator 100.

  Each storage room (the switching room 20, the ice making room 30, the freezing room 40 and the vegetable room 50) other than the cold storage room 10 is opened and closed by a drawer type door. In these drawer-type doors, the depth direction (front-rear direction) of the refrigerator 100 is obtained by sliding a frame provided by being fixed to the door with respect to rails formed horizontally on the left and right inner wall surfaces of each storage room. Can be opened and closed.

  A camera 9 is installed on the inner surface of the refrigerator compartment door 7. The camera 9 captures the inside of the storage room (here, the cold storage room 10) and outputs a storage room image. The camera 9 has a view range from the ceiling surface inside the cold storage room 10 to the chilled room at the lowermost part. That is, the camera 9 can image | photograph the inside of the refrigerator compartment 10 from the same viewpoint as the refrigerator user who looked at the inside of the refrigerator compartment 10 from the refrigerator compartment door 7 side.

  In addition, the installation position of the camera 9 is not restricted to the surface inside the refrigerator compartment door 7 as shown in FIG. The camera 9 may be installed anywhere as long as it can capture an item such as food placed on the shelf of the refrigerator compartment 10. Further, the number of cameras 9 installed is not limited to one. That is, a plurality of cameras 9 may be installed. By installing a plurality of cameras 9, it is possible to suppress the occurrence of a dead angle due to the visual field being blocked by the food or the like stored in the refrigerator compartment 10. Furthermore, you may install the camera 9 not only in the refrigerator compartment 10 but in another storage compartment.

  The refrigerator 100 includes a refrigeration cycle circuit that cools air supplied to each storage chamber. The refrigeration cycle circuit is composed of a compressor 2, a condenser (not shown), a throttling device (not shown), a cooler 3 and the like. The compressor 2 compresses and discharges the refrigerant in the refrigeration cycle circuit. The condenser condenses the refrigerant discharged from the compressor 2. The throttling device expands the refrigerant flowing out of the condenser. The cooler 3 cools the air supplied to each storage chamber by the refrigerant expanded by the expansion device. The compressor 2 is disposed, for example, at the lower part on the back side of the refrigerator 100.

  The refrigerator 100 is formed with an air passage 5 for supplying the air cooled by the refrigeration cycle circuit to each storage chamber. The air passage 5 is mainly disposed on the back side of the refrigerator 100. The cooler 3 of the refrigeration cycle circuit is installed in the air passage 5. Further, in the air passage 5, a blower fan 4 for sending the air cooled by the cooler 3 to each storage room is also installed.

  When the blower fan 4 operates, the air (cold air) cooled by the cooler 3 is sent through the air passage 5 to the freezing chamber 40, the switching chamber 20, the ice making chamber 30 and the refrigerating chamber 10, and these storage chambers Cooling. The vegetable compartment 50 is cooled by introducing the return cold air from the cold storage compartment 10 into the vegetable compartment 50 via the return compartment for the cold storage compartment. The cool air that has cooled the vegetable compartment 50 is returned to the air duct 5 with the cooler 3 through the vegetable compartment return air duct (these feedback air ducts are not shown). Then, the air is cooled again by the cooler 3 and cold air is circulated in the refrigerator 100.

  A damper (not shown) is provided at a midway point from the air passage 5 to each storage room. Each damper opens and closes a portion leading to each storage room of the air passage 5. By changing the open / close state of the damper, it is possible to adjust the blowing amount of cold air supplied to each storage chamber. In addition, the temperature of the cold air can be adjusted by controlling the operation of the compressor 2.

  The refrigeration cycle circuit including the compressor 2 and the cooler 3 provided as described above, the blower fan 4, the air passage 5 and the damper constitute a cooling means for cooling the inside of the storage chamber.

  A control device 8 is accommodated at, for example, an upper portion on the back side of the refrigerator 100. The control device 8 is provided with a control circuit and the like for performing various controls necessary for the operation of the refrigerator 100.

  Next, the configuration of a control system of the refrigerator 100 will be described with reference to FIG. The control device 8 includes, for example, a microcomputer, and includes a processor 8a and a memory 8b. The control device 8 executes a process set in advance by the processor 8a executing a program stored in the memory 8b, and controls the refrigerator 100.

  The operation panel 6 includes an operation unit 6a, a display unit 6b, a microphone 6c, and a speaker 6d. The operation unit 6a is an operation switch for setting a cold storage temperature of each storage room and an operation mode (such as a thawing mode) of the refrigerator 100. The display unit 6 b is a liquid crystal display that displays various information such as the temperature of each storage room. In addition, the operation panel 6 may include a touch panel that doubles as the operation unit 6a and the display unit 6b.

  The microphone 6 c is a voice input device for inputting a voice around the refrigerator 100, in particular, a voice uttered by a refrigerator user standing in front of the refrigerator compartment door 7. The speaker 6 d is a voice output device for producing a voice to a refrigerator user who stands around the refrigerator 100, in particular, in front of the refrigerator compartment door 7.

  The refrigerator 100 includes a thermistor 91 and a door open / close detection switch 92. The thermistor 91 detects the temperature in each storage chamber. The thermistor 91 is installed in each storage room. The door open / close detection switch 92 is an open / close detection means for detecting the open / close of the door of the storage room. The door open / close detection switch 92 detects, for example, the opening / closing of the refrigerator compartment door 7 of the refrigerator compartment 10. The door open / close detection switch 92 is, for example, a switch of a general magnet system. That is, the door open / close detection switch 92 detects, for example, the proximity of the magnet embedded in the refrigerator compartment door 7 by a pair of reed switches installed on the refrigerator 100 main body side.

  A detection signal of the temperature inside each storage chamber is input from the thermistor 91 to the control device 8. Further, an operation signal from the operation unit 6 a of the operation panel 6 is also input to the control device 8. Further, a detection signal from the door open / close detection switch 92 is also input to the control device 8.

  The control device 8 executes a process of controlling the operation of the compressor 2 and the blower fan 4 so that the inside of each storage chamber is maintained at the set temperature based on the input signal. That is, the control device 8 controls the operation of the refrigerator 100 by controlling the above-described cooling means and the like.

  Next, the configuration of a refrigerator system provided with the refrigerator 100 configured as described above will be described with reference to FIGS. 5 to 7 in addition to FIG. 4. As described above, the refrigerator system includes the refrigerator 100, the terminal device 200, and the server device 300. As shown in FIG. 5, the control device 8 of the refrigerator 100 includes a reproduction sound determination unit 101, a refrigerator sound reproduction unit 102, a refrigerator recording unit 103, a sound identification unit 104, an imaging unit 105, and a refrigerator communication unit 110. Further, as shown in FIG. 6, the terminal device 200 includes a terminal recording unit 201, a reproduction order setting unit 202, a terminal voice reproduction unit 203, an image display unit 204, and a terminal communication unit 210. Then, as shown in FIG. 7, the server apparatus 300 includes a storage unit 301 and a server communication unit 310.

  The refrigerator communication unit 110 is for bidirectionally communicating data between the refrigerator 100 and the server device 300. The terminal communication unit 210 is also for bidirectionally communicating data between the terminal device 200 and the server device 300. The server communication unit 310 is for bidirectionally communicating data between each of the server device 300 and the refrigerator 100 and between the server device 300 and the terminal device 200.

  The terminal recording unit 201 is for recording a voice emitted by the watching user of the terminal device 200. The terminal device 200 is provided with or connected to an audio input device such as a microphone (not shown). The terminal recording unit 201 records the voice input to the voice input device of the terminal device 200, and stores the voice as voice data.

  The reproduction order setting unit 202 is for setting the reproduction order of the audio data recorded by the terminal recording unit 201. The reproduction order is an order to be reproduced by the refrigerator sound reproduction unit 102 of the refrigerator 100 for one or more pieces of audio data. The watching user can operate the terminal device 200 to set a reproduction order for each voice data recorded by the terminal recording unit 201. The reproduction order setting unit 202 sets the reproduction order for each audio data recorded by the terminal recording unit 201 according to the operation content of the watching user on the terminal device 200.

  Then, the reproduction order setting unit 202 generates reproduction order designation data based on the set reproduction order. The reproduction order specification data includes information for specifying the reproduction order set by the reproduction order setting unit 202 for one or more pieces of audio data.

  Note that the watching user may be able to set the playback order indirectly instead of directly. Specifically, the setting of the indirect reproduction order may be, for example, setting the importance to each audio data, and setting the reproduction order of each audio data based on the importance of each audio data. In this case, the higher the degree of importance of the audio data, the higher the reproduction order. If there is a plurality of audio data having the same importance, for example, the recording time is late, that is, the reproduction order of newer audio data is increased. If there is only one audio data, the reproduction order of the audio data is automatically set to the highest order.

  The terminal communication unit 210 transmits the audio data recorded by the terminal recording unit 201 and the reproduction order designation data generated by the reproduction order setting unit 202 to the server device 300.

  The server communication unit 310 of the server device 300 receives the audio data and the reproduction order designation data transmitted from the terminal device 200. The storage unit 301 of the server device 300 stores the audio data and the reproduction order designation data received by the server communication unit 310. That is, the storage unit 301 is storage means for storing voice data recorded by the terminal device 200 and reproduction order designation data generated by the terminal device 200. In addition, the server communication unit 310 transmits the received voice data and reproduction order designation data to the refrigerator 100.

  The refrigerator communication unit 110 receives the audio data and the reproduction order designation data transmitted from the server device 300. That is, the refrigerator communication unit 110 is a data acquisition unit that acquires voice data and reproduction order designation data of the voice data from the outside.

  The reproduction sound determination unit 101 determines next time reproduction sound data from among the sound data received by the refrigerator communication unit 110 based on the reproduction order designation data received by the refrigerator communication unit 110. That is, the reproduction sound determination unit 101 is reproduction sound determination means for determining the next reproduction sound data based on the reproduction order designation data. Here, the next reproduced voice data is voice data to be reproduced next when the door open / close detection switch 92 detects the open / close of the refrigerator compartment door 7.

  Specifically, among the audio data acquired by the refrigerator communication unit 110 which is the data acquisition means, the reproduction audio determination unit 101 sets, next time, audio data for which the highest reproduction order is specified by the reproduction order specification data. Determined as playback audio data. At this time, it is preferable that the reproduction sound determination unit 101 determine next time reproduction sound data from among the sound data not yet reproduced by the refrigerator sound reproduction unit 102 and the speaker 6 d. In addition, the reproduction sound determination unit 101 determines the reproduction sound data next time from among the sound data acquired by the refrigerator communication unit 110 after the door open / close detection switch 92 finally detects the opening / closing of the refrigerator compartment door 7 You may

  Further, the reproduction sound determination unit 101 may determine next reproduction sound data based on the reproduction order designation data and the current time. Specifically, for example, when the content of the voice data is to notify the expiration date and the like of the food in the cold storage room 10, when the current time passes the expiration date, the reproduction sound determination unit 101 specifies the reproduction order. Regardless of the data, the voice data is determined as the next playback voice data.

  When the door open / close detection switch 92 detects the opening / closing of the refrigerator compartment door 7, the refrigerator sound reproducing unit 102 causes the speaker 6d to reproduce the next reproduction sound data determined by the reproduction sound determination unit 101. Then, the speaker 6d sounds the next reproduction voice data to notify the refrigerator user of the contents of the next reproduction voice data by voice. That is, the speaker 6d and the refrigerator sound reproduction unit 102 constitute sound reproduction means for reproducing the next reproduction sound data determined by the reproduction sound determination means when the opening and closing detection means detects the opening and closing of the door.

  The refrigerator recording unit 103 records the voice input to the microphone 6 c of the refrigerator 100. The refrigerator recording unit 103 starts voice recording after reproduction of the next reproduction sound data by the speaker sound reproduction unit 102 by the speaker 6 d is completed. That is, the refrigerator recording unit 103 and the microphone 6c constitute a sound recording unit that records sound after the sound reproduction unit ends the reproduction of the reproduction sound data next time.

  The refrigerator communication unit 110 transmits the sound recorded by the refrigerator recording unit 103 to the server device 300 as target sound data. Therefore, the refrigerator communication unit 110 is also a data transmission unit that transmits the sound recorded by the recording unit as the target sound data to the outside.

  At this time, when the voice recorded by the refrigerator recording unit 103 includes a human voice, the refrigerator communication unit 110 may transmit the voice as target voice data. For this purpose, the control device 8 comprises a voice identification unit 104. The voice identification unit 104 analyzes the voice recorded by the refrigerator recording unit 103, and determines whether or not the voice includes a human voice. When it is determined by the voice identification unit 104 that the voice recorded by the refrigerator recording unit 103 includes human voice, the refrigerator communication unit 110 transmits the voice as target voice data.

  The analysis of the sound recorded by the refrigerator recording unit 103 by the sound recognition unit 104 can be performed using an existing known technique. Specifically, for example, Mel-frequency cepstral coefficients (MFCC), Hidden Markov Model (HMM), and the like can be given as such existing known techniques.

  The server communication unit 310 of the server device 300 receives the target voice data transmitted from the refrigerator 100. The storage unit 301 stores the target voice data received by the server communication unit 310. That is, the storage unit 301, which is a storage unit, further stores the target voice data transmitted by the data transmission unit. Then, the server communication unit 310 transmits the received target voice data to the terminal device 200.

  The terminal communication unit 210 of the terminal device 200 receives the target voice data transmitted from the server device 300. The terminal voice reproduction unit 203 is for reproducing the target voice data received by the terminal communication unit 210, watching the contents thereof, and notifying the user by voice. Therefore, the terminal device 200 is provided with or connected to an audio output device such as a speaker (not shown). The terminal audio reproduction unit 203 causes the audio output device of the terminal device 200 to reproduce the target audio data. Then, the voice output device of the terminal device 200 sounds the target voice data to notify the watching user of the content of the target voice data by voice. Thus, the terminal device 200 can acquire and reproduce the target voice data stored in the storage unit of the server device 300.

  In the refrigerator 100 configured as described above and the refrigerator system including the refrigerator 100, a message of voice data from the watching user is triggered by the daily operation of the refrigerator user to open and close the door of the refrigerator 100. Can be communicated to the refrigerator user. At this time, even if there are a plurality of audio data from the watching user, the reproduction audio determination unit 101 selects appropriate audio data based on the reproduction order designation data to determine the next reproduction audio data. For this reason, it is possible to reproduce the voice data according to the user's intention to the refrigerator user only by the daily operation of the refrigerator user to open and close the door of the refrigerator 100.

  Also, after the voice data from the watching user is reproduced to the refrigerator user, the voice of the refrigerator user can be automatically recorded, and the voice of the recorded refrigerator user can be watched by the terminal device 200 by watching the voice of the refrigerator user . Therefore, it is possible to support appropriate two-way communication between the refrigerator user and the watching user without requiring a special operation or the like of the refrigerator user.

  Furthermore, recording of voice data by the watching user and reproduction of the voice data to the refrigerator user, and recording of target voice data by the refrigerator user and reproduction of the target voice data by the watching user both need to be performed simultaneously in real time Absent. For this reason, communication can be achieved without being restricted by time.

  In the above, an example has been described in which the refrigerator recording unit 103 and the microphone 6c, which are recording means, start sound recording after reproduction of the next reproduction sound data by the refrigerator sound reproduction unit 102 is finished. However, the timing at which the recording means starts recording voice is not limited to this. If recording is performed when the door open / close detection switch 92 detects the opening / closing of the refrigerator compartment door 7, the refrigerator recording unit 103 may start voice recording before the reproduction of the next reproduction voice data ends.

  When recording is started during reproduction of the next reproduction voice data, the voice of the next reproduction voice data to be reproduced by the speaker 6d is inputted to the microphone 6c, and the voice of the next reproduction voice data is included in the voice to be recorded by the refrigerator recording unit 103. There is a possibility of overlapping. Therefore, even if the refrigerator recording unit 103 acquires next reproduction sound data being reproduced from the refrigerator sound reproduction unit 102, and records the sound signal input to the microphone 6c after removing the next reproduction sound data. Good.

  Next, with reference to FIGS. 8 and 9, an example of screen display on the terminal device 200 configured as described above will be described. As described above, the terminal device 200 is a personal computer, a smartphone, a tablet terminal, or the like. The terminal device 200 includes a display device capable of displaying various screens. The display device is, for example, a liquid crystal display or the like.

  FIG. 8 shows an example of a refrigerator user list screen 400 displayed on the terminal device 200. In the example of the refrigerator user list screen 400, various information such as the refrigerator user serial number 401, the refrigerator user name 402, the refrigerator user age 403, the refrigerator user living information 404, the refrigerator user address 405 and the final confirmation date 406 It is displayed in a list for each refrigerator user who is to be watched over 200. The final confirmation date 406 is the date when the terminal device 200 finally confirmed the detailed information of the refrigerator user.

  Further, the refrigerator user list screen 400 displays a detailed information display button 407 for each refrigerator user. When the detailed information display button 407 is operated, a transition is made to a refrigerator user detail screen that displays detailed information of the refrigerator user corresponding to the operated button.

  What is shown in FIG. 9 is an example of the refrigerator user detail screen (only sound) displayed on the terminal device 200. When the detailed information display button 407 of the refrigerator user list screen 400 is operated, the refrigerator user detailed screen (only voice) 500 of the corresponding refrigerator user is displayed on the terminal device 200.

  In the example of the refrigerator user detail screen (voice only) 500 shown in FIG. 9, a back button 501, refrigerator user basic information 510, recording start button 520, refrigerator information list 530, watching voice information list 540, watching voice An information list 550 and a reproduction order setting / selection unit 560 are displayed.

  When the back button 501 is operated, the refrigerator user list screen 400 can be returned. When the recording start button 520 is operated, recording by the terminal recording unit 201 is started, and the above-mentioned voice data can be created. During recording by the terminal recording unit 201, the recording start button 520 changes to a recording stop button.

  The refrigerator information list 530 includes door open / close timing information 531 and door open time information 532. These pieces of information are created based on the detection result of the door open / close detection switch 92 of the refrigerator 100.

  In the audio information list 540 on the watching side, a list of information of audio data recorded by the terminal device 200 is displayed. The voice information list 540 on the watching side includes a creation date 541, an importance check box 542, a play button 543, and a next play check box 544. The creation date 541 is the date when the voice data was recorded. When the watching user operates the importance check box 542, the importance of the voice data can be set. In this example, when the importance degree check box 542 is checked, the importance degree of the audio data can be set to “high”. When the importance check box 542 is unchecked, the importance of the audio data can be set to “low”.

  When the watching user operates the play button 543, the voice data is played on the terminal device 200, and the watching user can confirm the contents. When the watching user operates the reproduction check box 544 next time, it is possible to set whether or not the voice data is to be reproduced on the refrigerator 100 side. In this example, when the next playback check box 544 is checked, the voice data can be set to be played back when the refrigerator door 7 is opened next time. In addition, when the check of the reproduction check box 544 is removed next time, the voice data can be set not to be reproduced at the next time the refrigerator door 7 is opened and closed.

  In the audio information list 540 on the watching side, information indicating whether the audio data has been reproduced by the refrigerator 100 is also displayed. In the example of FIG. 9, the “not reproduced” mark is displayed in the sound data not reproduced by the refrigerator 100.

  In the voice information list 550 on the side to be watched, a list of information on target voice data recorded by the refrigerator 100 is displayed. The voice information list 550 on the watched side includes a creation date 551 and a play button 552. The creation date 551 is the date when the target voice data was recorded. When the watching user operates the play button 552, the terminal voice reproducing unit 203 plays the target voice data, and the watching user can confirm the contents.

The reproduction order setting / selection unit 560 is for setting the operation content of the reproduction order setting unit 202 when the reproduction order setting unit 202 sets the reproduction order. By operating the reproduction order setting / selection unit 560, for example, the following settings can be made.
・ Whether or not to reflect the check status of the importance degree check box 542 in the reproduction order ・ As a default operation when there is no specification in particular, whether or not to make the reproduction order of the latest voice higher in order ・ There is no specification in special order The default behavior in this case is whether to play all unplayed audio

  Here, as described above, the refrigerator 100 is provided with the camera 9 for photographing the inside of the storage room (the cold storage room 10). Therefore, for example, with the configuration described below, the terminal device 200 may be able to confirm an image inside the refrigerator compartment 10 photographed by the camera 9. That is, as shown in FIG. 5, the control device 8 of the refrigerator 100 includes the imaging unit 105. The imaging unit 105 controls the photographing operation of the camera 9. Then, the imaging unit 105 acquires an image in the storage room taken by the camera 9.

  Specifically, for example, when the door open / close detection switch 92 detects the open / close of the refrigerator compartment door 7, the imaging unit 105 causes the camera 9 to image the inside of the refrigerator compartment 10. More specifically, the imaging unit 105, for example, detects that the cold storage door 7 that has been opened is closed, and then causes the camera 9 to perform imaging while the inside of the cold storage room 10 is illuminated. Then, the image in the refrigerator compartment 10 captured by the camera 9 is sent to the imaging unit 105. Thus, the camera 9 and the imaging unit 105 constitute an imaging means for capturing an image inside the storage room.

  The refrigerator communication unit 110 transmits the image in the refrigerator compartment 10 photographed by the camera 9 to the server device 300 as image data. That is, the refrigerator communication part 110 which is a data transmission means transmits outside the image which the imaging means recorded as image data.

  The server communication unit 310 of the server device 300 illustrated in FIG. 7 receives the image data transmitted from the refrigerator 100. The storage unit 301 stores the image data received by the server communication unit 310. That is, the storage unit 301, which is a storage unit, further stores the image data transmitted by the data transmission unit. Then, the server communication unit 310 transmits the received image data to the terminal device 200.

  The terminal communication unit 210 of the terminal device 200 illustrated in FIG. 6 receives the image data transmitted from the server device 300. As shown in FIG. 6, the terminal device 200 includes an image display unit 204. The image display unit 204 causes the display device of the terminal device 200 to display the image data received by the terminal communication unit 210. Then, the watching user can confirm the image data displayed on the display device of the terminal device 200. Thus, the terminal device 200 can acquire and display the image data stored in the storage unit of the server device 300.

  In the refrigerator 100 configured as described above and the refrigerator system including the refrigerator 100, the watching user can check the state of the storage room of the refrigerator 100 with the terminal device 200. In addition, the watching user can record voice data in accordance with the situation in the storage room of the refrigerator 100, and can easily support eating habits and the like of the refrigerator user.

  FIG. 10 shows an example of a refrigerator user detail screen (voice and image) 600 displayed on the terminal device 200 in a configuration in which the terminal device 200 can check the image inside the cold storage room 10 photographed by the camera 9. is there. When it is possible to confirm the image inside the refrigerator compartment 10 photographed by the camera 9 with the terminal device 200 by operating the detailed information display button 407 of the refrigerator user list screen 400 of FIG. 8, the refrigerator user detailed screen of the corresponding refrigerator user (Voice and image) 600 is displayed on the terminal device 200.

  In the example of the refrigerator user detail screen (voice and image) 600 shown in FIG. 10, a back button 601, refrigerator user basic information 610, shooting date and time 621, 622, 623, storage room image 631, 632, 633, voice reproduction of watching side Buttons 641 and 642, voice reproduction buttons 651, 652 and 653 on the watching side and a recording start button 660 are displayed.

  When the back button 601 is operated, the refrigerator user list screen 400 can be returned. Storage room images 631, 632, 633 are images of the inside of the refrigerator compartment 10 taken by the camera 9. In the example of FIG. 10, the storage room image 631 is of the shooting date 621. Also, the storage room image 632 is of the shooting date 622. The storage room image 633 is of the shooting date 623.

  In the example of FIG. 10, the audio reproduction button 641 on the watching side is displayed corresponding to the storage room image 631. Further, an audio playback button 642 on the side to be watched corresponding to the storage room image 632 is displayed. The latest storage room image 633 is displayed corresponding to the recording start button 660 instead of the audio playback button on the watching side.

  When the watching user operates the recording start button 660, recording by the terminal recording unit 201 is started, and audio data can be created. During recording by the terminal recording unit 201, the recording start button 660 changes to a recording stop button. The audio data created by this recording is associated with the latest storage room image 633. Thus, each audio data is associated with the latest storage room image at the time of creation of the audio data.

  When the watching user operates the voice playback buttons 641 and 642 on the watching side, voice data corresponding to each button is played on the terminal device 200, and the watching user can confirm the contents. In the example of FIG. 10, when the audio reproduction button 641 on the watching side is operated, audio data corresponding to the storage room image 631 is reproduced. Further, when the audio reproduction button 642 on the watching side is operated, audio data corresponding to the storage room image 632 is reproduced.

  When the watching user operates the voice playback buttons 651, 652, 653 on the watching side, the terminal voice playing unit 203 plays the target voice data corresponding to each button, and the watching user can confirm the contents. In the example of FIG. 10, the voice reproduction button 651 on the watched side corresponds to the storage room image 631. Therefore, the target audio data reproduced by the audio reproduction button 651 on the watched side corresponds to the storage room image 631.

  Also, the audio playback button 652 on the watched side corresponds to the storage room image 632. Therefore, the target audio data reproduced by the audio reproduction button 652 on the watched side corresponds to the storage room image 632. However, in the example of FIG. 10, the target audio data corresponding to the storage room image 632 does not exist. For this reason, the target audio data is not reproduced even if the audio reproduction button 652 on the watched side is operated. As described above, when there is no corresponding target voice data, it is preferable that the corresponding watched audio playback button can not be operated.

  The audio playback button 653 on the watched side corresponds to the storage room image 633. Therefore, the target audio data reproduced by the audio reproduction button 653 on the watched side corresponds to the storage room image 633. In this way, each target audio data is associated with image data captured at the time of recording of the target audio data.

  Next, with reference to FIG. 11, an example of sound reproduction and recording operations of the refrigerator 100 configured as described above will be described. When the refrigerator 100 starts the voice reproduction and recording operation (step S100), first, in step S101, the control device 8 confirms whether or not the door open / close detection switch 92 detects that the refrigerator compartment door 7 is opened. . When the door open / close detection switch 92 does not detect that the cold storage door 7 is opened, the process waits in step S101 until the door open / close detection switch 92 detects that the cold storage door 7 is opened. Then, when the door open / close detection switch 92 detects that the refrigerator compartment door 7 is opened, the process proceeds to step S102.

  In step S102, the refrigerator sound reproduction unit 102 causes the speaker 6d to reproduce the next reproduction sound data determined by the reproduction sound determination unit 101. After step S102, the process proceeds to step S103.

  In step S103, the refrigerator recording unit 103 starts sound recording after reproduction of the next reproduction sound data by the refrigerator sound reproduction unit 102 ends. After step S103, the process proceeds to step S104.

  In step S104, the refrigerator recording unit 103 checks whether or not a predetermined time set in advance has elapsed since the start of recording in step S103. If the predetermined time has not elapsed from the start of recording, the process proceeds to step S105.

  In step S105, the refrigerator recording unit 103 confirms whether or not the door open / close detection switch 92 detects that the refrigerator compartment door 7 is closed. If the door open / close detection switch 92 does not detect that the refrigerator compartment door 7 is closed, the process returns to step S104.

  On the other hand, when the door open / close detection switch 92 detects that the refrigerator compartment door 7 is closed in step S105, the process proceeds to step S106. The process also proceeds to step S106 when a predetermined time has elapsed from the start of recording in step S104. In this case, the process proceeds from step S104 directly to step S106 without passing through step S105. In step S106, the refrigerator recording unit 103 stops the sound recording. After step S106, the process proceeds to step S107.

  In step S107, the voice identification unit 104 checks whether the voice recorded by the refrigerator recording unit 103 in steps S103 to S106 includes human voice. If the voice recorded by the refrigerator recording unit 103 includes human voice, the process proceeds to step S108. In step S108, the voice identification unit 104 determines that the voice recorded by the refrigerator recording unit 103 includes human voice.

  On the other hand, when the voice recorded by the refrigerator recording unit 103 does not include human voice in step S107, the process proceeds to step S109. In step S109, the control device 8 deletes the voice data recorded by the refrigerator recording unit 103 in steps S103 to S106. After step S109, the process proceeds to step S110. In step S110, the voice identification unit 104 determines that the voice recorded by the refrigerator recording unit 103 does not include human voice. When the process of step S108 or step S110 is completed, the series of sound reproduction and recording operations of the refrigerator 100 end.

  Next, an example of the image transmission operation of the refrigerator 100 configured as described above will be described with reference to FIG. When the door open / close detection switch 92 detects that the refrigerator compartment door 7 is opened in step S101 in the flow of voice reproduction and recording operation of the refrigerator 100 shown in FIG. 11, in parallel with step S102 and subsequent steps in the flow of FIG. The image transmission operation shown in FIG. 12 is performed.

  When the refrigerator 100 starts an image transmission operation (step S200), first, in step S201, the imaging unit 105 checks whether the door open / close detection switch 92 detects that the cold storage door 7 is closed. When the door open / close detection switch 92 does not detect that the cold storage door 7 is closed, the process waits in step S201 until the door open / close detection switch 92 detects that the cold storage door 7 is closed. When the door open / close detection switch 92 detects that the cold storage door 7 is closed, the process proceeds to step S202.

  In step S202, after the door open / close detection switch 92 detects that the refrigerator compartment door 7 is closed in step S201, the imaging unit 105 checks whether a predetermined time set in advance has elapsed. The predetermined time of step S202 is set separately from the predetermined time of step S104 described above. If the fixed time has not elapsed since the door open / close detection switch 92 detects that the refrigerator compartment door 7 is closed, the process waits in step S202 until the fixed time elapses. Then, when the predetermined time has elapsed, the process proceeds to step S203.

  In step S203, the imaging unit 105 causes the camera 9 to capture an image of the inside of the storage room. Then, the imaging unit 105 acquires an image inside the storage room captured by the camera 9. After step S203, the process proceeds to step S204.

  In step S204, the refrigerator communication unit 110 transmits the image data of the image inside the storage room taken in step S203 to the server apparatus 300. In addition, in step S204, the refrigerator communication unit 110 includes the voice of the person when the voice determined as the voice of the person is included by the execution of the step S108 in FIG. 11 is recorded. The corresponding voice is transmitted to the server device 300 as target voice data. When the process of step S204 is completed, a series of image transmission operations of the refrigerator 100 end.

  Next, an example of the voice recording operation of the terminal device 200 configured as described above will be described with reference to FIG. When the terminal device 200 starts the voice recording operation (step S300), first, in step S301, the terminal recording unit 201 starts the recording start button 520 of the refrigerator user detail screen (voice only) 500 shown in FIG. It is confirmed whether the recording start button 660 of the refrigerator user detail screen (voice and image) 600 shown in FIG. When the recording start button is not operated, the voice recording operation of the terminal device 200 ends. On the other hand, when the recording start button is operated, the process proceeds to step S302.

  In step S302, the terminal recording unit 201 starts sound recording. After step S302, the process proceeds to step S303. In step S303, the terminal recording unit 201 confirms whether or not a predetermined time set in advance has elapsed since the start of recording in step S302. The predetermined time of step S303 is set separately from the predetermined time of steps S104 and S202 described above. If the predetermined time has not elapsed from the start of recording, the process proceeds to step S304.

  In step S304, the terminal recording unit 201 operates the recording stop button (not shown) of the refrigerator user detail screen (voice only) 500 or the recording stop button (not shown) of the refrigerator user detail screen (voice and image) 600. Make sure. If the recording stop button is not operated, the process returns to step S303.

  On the other hand, when the recording stop button is operated in step S304, the process proceeds to step S305. Also in the case where a predetermined time has elapsed from the start of recording in step S303, the process proceeds to step S305. In this case, the process proceeds from step S303 directly to step S305 without passing through step S304. In step S305, the terminal recording unit 201 stops the sound recording. After step S305, the process proceeds to step S306.

  In step S306, the terminal communication unit 210 transmits the audio data recorded in step S302 to step S305 to the server device 300. In addition, in step S306, the terminal communication unit 210 also transmits the reproduction order designation data generated by the reproduction order setting unit 202 to the server 300. When the process of step S306 is completed, the series of voice recording operations of the terminal device 200 end. When the process of step S110 is completed, a series of sound reproduction and recording operations of the refrigerator 100 are ended.

  When the target voice data recorded in one refrigerator 100 can be reproduced by a plurality of terminal devices 200 in the refrigerator system configured as described above, the refrigerator user can use which terminal device 200 to reproduce the audio data. It may be selectable. As a utilization scene of such a configuration, for example, it is considered that a refrigerator user is an elderly person, and a family member and facility staff of the elderly person watch and use different terminal devices 200 as the user, respectively. Be

  In this case, for example, the target voice data is transmitted to the terminal device 200 of the facility staff at normal times. Then, the target voice data can be transmitted to the family terminal device 200 only when the specific button is pressed. By doing this, it is possible to tell only the family that the refrigerator user does not want to tell the facility staff, and it is possible to secure privacy.

  In addition, when transmitting the image data in the storage room to the terminal device 200, the terminal device 200 may calculate the estimated consumption of the food by the refrigerator user from the time change of the image data. Furthermore, the terminal device 200 may determine the type of the consumed food to be able to evaluate the nutritional status of the refrigerator user. By doing this, the watching user can easily grasp the diet and nutritional condition of the refrigerator user.

  Further, the control device 8 of the refrigerator 100 may analyze the content of the audio data recorded by the terminal device 200 and control the operation of the refrigerator 100 in accordance with the content of the audio. When the content of the voice data relates to a specific food in the refrigerator compartment 10, for example, the position of the food may be illuminated with an LED or the like so that the position of the food may be known to the refrigerator user.

DESCRIPTION OF SYMBOLS 1 heat insulation box 2 compressor 3 cooler 4 air blower fan 5 air path 6 operation panel 6a operation part 6b display part 6c microphone 6d speaker 7 refrigerator compartment door 7a right door 7b left door 8 control device 8a processor (CPU)
8b memory 9 camera 10 cold storage room 20 switching room 30 ice room 40 freezing room 50 vegetable room 91 thermistor 92 door open / close detection switch 100 refrigerator 101 playback sound determination unit 102 refrigerator sound reproduction unit 103 refrigerator recording unit 104 sound identification unit 105 imaging unit 110 Refrigerator communication unit 200 terminal device 201 terminal recording unit 202 reproduction order setting unit 203 terminal voice reproduction unit 204 image display unit 210 terminal communication unit 300 server device 301 storage unit 310 server communication unit 400 refrigerator user list screen 500 refrigerator user detail screen (voice only)
600 Refrigerator user detail screen (voice and image)

Claims (9)

Opening / closing detection means for detecting opening / closing of the storage room door;
Data acquisition means for acquiring audio data and reproduction order designation data of the audio data from the outside;
Reproduction sound determination means for determining next reproduction sound data which is the sound data to be reproduced next when the open / close detection means detects the opening / closing of the door based on the reproduction order designation data;
Voice reproduction means for reproducing the next reproduction sound data determined by the reproduction sound determination means when the opening / closing detection means detects opening / closing of the door;
Recording means for recording a voice when the open / close detection means detects the opening / closing of the door;
A data transmission unit for transmitting the sound recorded by the recording unit to the outside as target sound data;   The refrigerator according to claim 1, wherein the reproduction sound determination unit determines the next reproduction sound data from the unreproduced sound data acquired by the data acquisition unit.   The said reproduction | regeneration audio | voice determination means determines the said next reproduction | regeneration audio | voice data out of the audio | voice data which the said data acquisition means acquired after the time of the said open / close detection means detecting opening / closing of the said door last The refrigerator according to claim 2.   The refrigerator according to any one of claims 1 to 3, wherein the reproduction sound determination means determines the next reproduction sound data based on the reproduction order designation data and the current time.   The refrigerator according to any one of claims 1 to 4, wherein the data transmission means transmits the voice as the target voice data when the voice recorded by the recording means includes human voice.   The refrigerator according to any one of claims 1 to 5, wherein the recording means starts recording after completion of reproduction of the next reproduction audio data by the audio reproduction means. It further comprises an imaging means for capturing an image inside the storage room,
The refrigerator according to any one of claims 1 to 6, wherein the data transmission unit further transmits the image captured by the imaging unit as image data to the outside. A refrigerator according to any one of claims 1 to 6,
A server device provided to be able to communicate with the refrigerator;
A terminal device provided to be able to communicate with the server device;
The server device includes storage means for storing the voice data recorded by the terminal device and the reproduction order designation data generated by the terminal device.
The data acquisition means acquires the audio data and the reproduction order designation data stored in the storage means from the server device.
The storage means further stores the target voice data transmitted by the data transmission means,
The refrigerator system wherein the terminal device is capable of acquiring and reproducing the target audio data stored in the storage means. A refrigerator according to claim 7, and
A server device provided to be able to communicate with the refrigerator;
A terminal device provided to be able to communicate with the server device;
The server device includes storage means for storing the voice data recorded by the terminal device and the reproduction order designation data generated by the terminal device.
The data acquisition means acquires the audio data and the reproduction order designation data stored in the storage means from the server device.
The storage unit further stores the target audio data and the image data transmitted by the data transmission unit.
The refrigerator system is capable of acquiring and reproducing the target audio data stored in the storage unit and capable of acquiring and displaying the image data stored in the storage unit.

Images