TWI906662B - Item management system and refrigerator - Google Patents

Abstract

A refrigerator and an item management system are provided to suppress the decrease in the accuracy of item management within the refrigerator. The item management system includes: a refrigerator with a main body; and a camera device installed on the top surface of the main body for photographing the refrigerator. The camera device includes: a first wireless communication module capable of transmitting management data for managing the items contained in the refrigerator. When the camera device is installed in the main body, the first wireless communication module is positioned further forward than the opening of the main body.

Description

Item management system and refrigerator

This disclosure relates to an inventory management system and a refrigerator.

Patent document 1 discloses the following technology: a camera is installed at the top of a refrigerator, which takes a picture when it detects an object passing through the refrigerator's opening, and manages the food stored in the refrigerator based on the picture results. (Prior art documents, patent documents)

Patent Document 1: Japanese Patent Application Publication No. 2019-070476

The invention aims to solve the following problem: it discloses an item management system and a refrigerator that can suppress the reduction of management accuracy of items contained in the refrigerator.

The means to solve the problem disclosed herein is an item management system comprising: a refrigerator having a main body; and a camera device installed on the top surface of the main body for photographing the refrigerator. The camera device comprises: a first wireless communication module capable of transmitting management data for managing the items contained in the refrigerator. When the camera device is installed in the main body, the first wireless communication module is located further forward than the opening of the main body.

Furthermore, the refrigerator disclosed herein is a refrigerator with a main body, equipped with: a camera device mounted on the top surface of the main body for photographing the refrigerator; the camera device also includes: a first wireless communication module capable of transmitting management data for managing the items contained in the refrigerator; the first wireless communication module is located further forward than the opening of the main body. Additionally, this specification includes the entire contents of Japanese Patent Application No. 2022-165227, filed on October 14, 2022.

The invention's effect: The item management system and refrigerator disclosed herein can suppress interference caused by the top surface of the main unit in the radio waves transmitted and received by the first wireless communication module. Accordingly, since the first wireless communication module can stably transmit management data, the reduction in the accuracy of item management can be suppressed.

In terms of the form of the invention (the knowledge and insights that form the basis of this disclosure), when the inventors conceived of this disclosure, there was a technique that involved taking a picture when an object was detected passing through the opening of a refrigerator, and managing the food stored in the refrigerator based on the picture results. In conventional technology, the camera device transmits management data for managing the items stored in the refrigerator to an external device via wireless communication. However, since the top surface of the main body of the refrigerator is generally made of metal, depending on the positional relationship between the wireless communication module of the camera device and the top surface of the main body, interference may occur in the radio waves transmitted and received by the wireless communication module due to the top surface. Therefore, the inventors discovered that the positional relationship between the wireless communication module of the camera device and the top surface of the main cabinet might prevent the wireless communication module from stably transmitting management data, thus reducing the accuracy of item management. This discovery, made to solve this problem, ultimately led to the subject of this disclosure. Therefore, this disclosure provides an item management system and a refrigerator that can suppress the reduction in the accuracy of item management within the refrigerator.

The embodiments are described in detail below with reference to the drawings. However, sometimes unnecessary details are omitted. For example, detailed descriptions of well-known matters or repetitive descriptions of substantially the same components are sometimes omitted. Furthermore, the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the patent application.

(Embodiment 1) [1-1. Composition] [1-1-1. Composition of the Item Management System] Figure 1 is a diagram showing the composition of the item management system 1 in embodiment 1. The item management system 1 is a system for managing the items contained in the refrigerator 2. As an example of item management, the item management system 1 of this embodiment manages images of the items contained in the refrigerator 2.

In this embodiment, the items contained in the refrigerator 2 include food or containers for containing food. Pre-defined markings may also be attached to the items.

The inventory management system 1 includes a refrigerator 2. The refrigerator 2 has a main body 20 with an opening on the front surface. Within the main body 20, there are a refrigerator compartment 21, an ice-making compartment 22, a fresh-freezing compartment 23, a freezer compartment 24, and a vegetable compartment 25. These compartments are cooled by air. A rotating door 21A is provided in the opening 201 on the front surface of the refrigerator compartment 21. The door 21A includes a rotating right door 21B and a rotating left door 21C. Each of the ice-making compartment 22, the fresh-freezing compartment 23, the freezer compartment 24, and the vegetable compartment 25 has drawers 22A, 23A, 24A, and 25A for storing items. Each of the drawers 22A to 25A has an opening on its top surface when in use. Additionally, a rotating partition 21D is provided in the left door 21C. The rotating partition 21D is a long, narrow component that closes the gap between the right door 21B and the left door 21C when they are closed. Hereinafter, without distinguishing between drawers 22A, 23A, 24A, and 25A, it will be referred to as "drawer 26A" using the symbol "26A". The right door 21B is equivalent to "door 1" in this disclosure. The left door 21C is equivalent to "door 2" in this disclosure.

The inventory management system 1 is equipped with a camera device 3. The camera device 3 is installed on the top surface 20A of the main body 20 of the refrigerator 2. The top surface 20A of the main body 20 is made of metal. Alternatively, the top surface 20A of the main body 20 may be made of resin. The camera device 3 photographs the refrigerator 2. Furthermore, the camera device 3 can also photograph items passing through the opening 201 of the refrigerator compartment 21.

The item management system 1 includes a terminal device 4. The terminal device 4 is a portable computer such as a smartphone. The terminal device 4 has an application installed related to the management of items contained in the refrigerator 2, and communicates with the item management server 5 through the functions of this application. In the following description, this application will be referred to as the "item management application" and marked with the symbol "411".

In Figure 1, solid lines represent users P at home, and dashed lines represent users P who have left home H. When terminal device 4 is used by user P at home, it communicates with the item management server 5 via communication device 6, or not via communication device 6. Conversely, when terminal device 4 is used by user P who has left home H and cannot establish a communication connection with communication device 6, it does not communicate with the item management server 5 via communication device 6.

Communication device 6 is connected to a network NW, such as a public switched telephone network, and communicates with the item management server 5 through the network NW. Communication device 6 functions as an interface device for connecting refrigerator 2, camera device 3, and terminal device 4 to the network NW. Communication device 6 establishes a local area network within its own H.

The item management system 1 includes an item management server 5. The item management server 5 is a device that manages the items contained in the refrigerator 2. As part of item management, the item management server 5 stores images of the items. The item management server 5 is connected to the network (NW). Furthermore, although the item management server 5 is represented by a single square in the diagrams, this does not necessarily mean that the item management server 5 is a single device.

[1-1-2. Refrigerator Composition] Figure 2 is a three-dimensional view of refrigerator 2 in its set state. In Figure 2, the X-axis, Y-axis, and Z-axis are shown. The X-axis, Y-axis, and Z-axis are orthogonal to each other. The Z-axis represents the vertical direction and corresponds to the height of refrigerator 2. The X-axis and Y-axis are parallel to the horizontal direction. The X-axis represents the horizontal direction. The horizontal direction corresponds to the width of refrigerator 2. The Y-axis represents the front-back direction. The positive direction of the X-axis represents the right direction. The positive direction of the Y-axis represents the front direction. The positive direction of the Z-axis represents the upward direction.

Refrigerator 2 includes a main body 20, a door 21A, and drawers 26A. The main body 20 is a box-shaped component forming the main body of refrigerator 2. The main body 20 includes: an outer casing 202 with an opening on its front surface; an inner casing 203 housed inside the outer casing 202 and also with an opening on its front surface; and insulation material filling the space between the outer casing 202 and the inner casing 203. The outer casing 202 is formed of a metal such as iron or a hard resin such as ABS. The upper surface of the bottom 202, i.e., the top surface 20A of the main casing 20, is formed of metal. The inner casing 203 is formed of a hard resin such as ABS. The insulation material may be, for example, rigid urethane foam. In addition, the insulation material is not limited to this. Any material that has insulation properties and can fill the space between the outer casing 202 and the inner casing 203 can be used.

The interior space of the inner box 203 is divided into a plurality of spaces by partition components. In this embodiment, the interior space of the inner box 203 is divided into 5 spaces, and each of the 5 spaces functions as a refrigerator compartment 21, an ice-making compartment 22, a fresh-freezing compartment 23, a freezer compartment 24, and a vegetable compartment 25.

The right door 21B is pivotally supported by a hinge component 204, located on the top surface 20A of the main housing 20, making it rotatable. The left door 21C is pivotally supported by a hinge component 205, located on the top surface 20A of the main housing 20, making it rotatable. The right door hinge component 204 corresponds to the "first hinge component" of this disclosure. The left door hinge component 205 corresponds to the "second hinge component" of this disclosure.

A second wireless communication module 206 is provided in the right door hinge component 204. The second wireless communication module 206 is a module that receives control data for controlling the refrigerator 2 from an external device or sends predetermined data to an external device. Furthermore, the reason for providing the second wireless communication module 206 in the right door hinge component 204 rather than the left door hinge component 205 is as follows: The rotating partition 21D of the left door 21C has a heater for preventing condensation, and this heater is driven by AC (Alternating Current) power supplied from the main compartment 20 through the left door hinge component 205. Therefore, in order to ensure that the radio waves are not affected by the power supplied to the heater, the second wireless communication module 206 is installed on the right door hinge component 204 instead of the left door hinge component 205.

[1-1-3. Composition of the Camera Device] Referring to Figures 3-7, the camera device 3 will be described. Figures 3-7 show the same X, Y, and Z axes as those shown in Figure 2. The X-axis corresponds to the width and left-right direction of the camera device 3. The Y-axis corresponds to the front-back direction of the camera device 3. The Z-axis corresponds to the up-down direction of the camera device 3.

Figure 3 is a plan view of the camera device 3 installed in the refrigerator 2 from the right. Figure 4 is a perspective view of the limiting member 34 abutting the front edge 20B of the main body 20 from the left front.

As shown in Figures 3 and 4, the camera device 3 includes: a main body 30, disposed on the top surface 20A of the main housing 20; and a camera component 31 extending forward from the main body 30. The camera component 31 is disposed above the front part of the main body 30.

The main body 30 has a power button 32 on its front surface for turning the power of the camera device 3 on and off. The power button 32 is equipped with a first LED (light emitting diode) 33 that provides preset information.

The main body 30 has a limiting member 34. The limiting member 34 restricts the position of the camera component 31 in the front-rear direction of the refrigerator 2 by abutting against the front end edge 20B of the main body 20. The limiting member 34 is a piece extending from the bottom surface 30A of the main body 30. The limiting member 34 extends from the front surface of the main body 30 in the width direction of the main body 30 (the width direction of the camera device 3), and has a mark 35 at the center of the width direction of the main body 30. The mark 35 indicates the center of the width direction of the camera device 3. The limiting member 34 is formed to extend from the bottom surface 30A in a length that is shorter than the length from the top surface 20A of the main body 20 to the door liner, which is a member that is arranged approximately parallel to the top surface 20A in the upper part of the inner surface of the door 21A.

Figure 5 is a front view of refrigerator 2 and the camera device 3 installed on refrigerator 2. In Figure 5, refrigerator 2 shows door 21A open. Figure 6 is a right view of refrigerator 2 and the camera device 3 installed on refrigerator 2. In Figure 6, refrigerator 2 shows drawer 25A open, i.e., pulled out. Figure 7 is a top view of the camera device 3 installed on refrigerator 2.

The imaging device 3 includes a wide-angle camera 301 and a narrow-angle camera 302. The wide-angle camera 301 and the narrow-angle camera 302 are disposed at the front end of the bottom surface 31A of the imaging component 31. When the imaging device 3 is installed in the refrigerator 2 and the door 21A is closed, the wide-angle camera 301 and the narrow-angle camera 302 are positioned in front of the door 21A.

Wide-angle camera 301 is a camera with a wider angle than narrow-angle camera 302. With the imaging device 3 installed in the refrigerator 2, wide-angle camera 301 takes pictures of the refrigerator compartment 21 from the front and above of the refrigerator 2. For example, when viewed from the front of the refrigerator 2, the field of view of wide-angle camera 301 includes range A1 as shown in FIG. 5. Range A1 includes the door pockets 21E of the right door 21B and left door 21C and the opening 201 of the refrigerator compartment 21 when the right door 21B and left door 21C are opened to their maximum extent. Furthermore, when viewed from the right side of the refrigerator 2, the field of view of wide-angle camera 301 includes range A3 as shown in FIG. 6. The range A3 is the range from the front end of the right door 21B and the left door 21C in the open state to the front end of the shelf 21F in the front-back direction, and the range including the opening 201 of the refrigerator compartment 21 in the vertical direction.

The narrow-angle camera 302 has a narrower angle than the wide-angle camera 301. With the imaging device 3 installed in the refrigerator 2, the narrow-angle camera 302 takes pictures of the drawer 26A from the upper front of the refrigerator 2. For example, when viewed from the front of the refrigerator 2, the field of view of the narrow-angle camera 302 includes area A2 as shown in Figure 5. Area A2 includes the drawer 26A when viewed from the front of the refrigerator 2. Furthermore, for example, when viewed from the right side of the refrigerator 2, the field of view of the narrow-angle camera 302 includes area A4 as shown in Figure 6. Area A4 includes the drawer 26A in its fully extended position in the front-to-back direction.

The imaging device 3 includes a first light sensor 303 and a second light sensor 304. The first light sensor 303 and the second light sensor 304 are disposed on the main body 30. The first light sensor 303 is used to detect the opening and closing of the right door 21B. The first light sensor 303 is a reflective light sensor. When the imaging device 3 is installed in the refrigerator 2, the first light sensor 303 projects light onto a predetermined location on the right door 21B. Then, the first light sensor 303 outputs a signal displaying the amount of light received. The second light sensor 304 is used to detect the opening and closing of the left door 21C. The second light sensor 304 is a reflective light sensor. With the camera device 3 installed in the refrigerator 2, the second light sensor 304 projects light onto a predetermined location on the left door 21C. Then, the second light sensor 304 outputs a detection value showing the amount of light received.

The camera device 3 is equipped with a thermopile 305. The thermopile 305 is a sensor used to detect the opening and closing of the drawer 26A. The thermopile 305 is disposed at the front end of the bottom surface 31A of the camera component 31. The thermopile 305 outputs a display of the detected temperature value.

The imaging device 3 is equipped with a motion sensor 306. The motion sensor 306 can be an infrared sensor or an ultrasonic sensor. The motion sensor 306 outputs a detection value. The motion sensor 306 is disposed at the front end of the bottom surface 31A of the imaging component 31.

The imaging device 3 is equipped with an illuminance sensor 307. The illuminance sensor 307 is a sensor used to adjust the exposure of the wide-angle camera 301 and the narrow-angle camera 302. The illuminance sensor 307 outputs a detection value. The illuminance sensor 307 is disposed at the front end of the imaging component 31.

The camera device 3 includes a second LED 308 and a third LED 309. The second LED 308 is located at the front right end of the bottom surface 31A of the camera component 31. The third LED 309 is located at the front left end of the bottom surface 31A of the camera component 31. The second LED 308 and the third LED 309 illuminate the refrigerator 2 on which the camera device 3 is mounted from above.

As shown in Figure 7, the camera device 3 includes a first control board 311 and a second control board 312.

The first control board 311 is a board that controls the various sensors and power supply of the camera device 3. The first control board 311 is disposed inside the camera device 3. The first control board 311 is connected to a power button 32, a first LED 33, a first light sensor 303, a second light sensor 304, a thermopile 305, a motion sensor 306, and an illuminance sensor 307. Furthermore, the first control board 311 controls these devices. The first control board 311 is disposed behind the second control board 312 and the first wireless communication module 313.

The second control board 312 controls the wide-angle camera 301 and the narrow-angle camera 302, and also controls the first wireless communication module 313. The second control board 312 is disposed inside the imaging device 3. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control board 312. Furthermore, the second control board 312 controls these devices. The second control board 312 is arranged in front of and alongside the first control board 311.

The first wireless communication module 313 is a device equipped with hardware such as an antenna or wireless communication circuit. The first wireless communication module 313 is disposed inside the camera assembly 31, to the left of the second control board 312. In the camera device 3, the first wireless communication module 313 is positioned further forward than the opening 201 of the main body 20 when the camera device 3 is installed in the refrigerator 2. More specifically, the first wireless communication module 313 is positioned further forward than the limiting member 34 in the camera device 3.

Assuming the first wireless communication module 313 is positioned behind the limiting member 34, and the camera device 3 is installed in the refrigerator 2, the first wireless communication module 313 will overlap with the top surface 20A from a top-down view. Since the top surface 20A of the main body 20 is generally made of metal, when the first wireless communication module 313 overlaps with the top surface 20A from a top-down view, interference may occur in the radio waves transmitted and received by the first wireless communication module 313 caused by the top surface 20A. However, in this embodiment, since the first wireless communication module 313 does not overlap with the top surface 20A in a top-view perspective, interference caused by the top surface 20A in the radio waves transmitted and received by the first wireless communication module 313 can be suppressed. Furthermore, in this embodiment, the first wireless communication module 313 overlaps with the gate 21A in a top-view perspective. However, the frame of the gate 21A is generally made of resin. Therefore, the possibility of the gate 21A interfering with the radio waves transmitted and received by the first wireless communication module 313 is relatively low.

Furthermore, the first wireless communication module 313 can also be positioned above the top surface of the door 21A when the camera device 3 is installed in the refrigerator 2. For example, the first wireless communication module 313 can also be positioned above the surface of the door 21A. Alternatively, for example, the first wireless communication module 313 can also be positioned above the top surface 20A of the refrigerator 2. Furthermore, for example, the first wireless communication module 313 can also be positioned in front of and above the door 21A when it is closed. In this way, even if the frame of the door 21A is made of metal, interference with the radio waves of the first wireless communication module 313 can be suppressed.

With the camera device 3 installed in the main housing 20, the first wireless communication module 313 is located on the opposite side of the position where the second wireless communication module 206 is located. In this embodiment, as shown in FIG7, with the camera device 3 installed in the refrigerator 2, the first wireless communication module 313 is located to the left of the center ST of the camera device 3 in the left-right direction of the refrigerator 2 (the left-right direction of the camera device 3). That is, with the camera device 3 installed in the refrigerator 2, the first wireless communication module 313 is located to the left of the door hinge component 205, which is further to the left than the center ST of the camera device 3. In this way, since the distance between the first wireless communication module 313 and the second wireless communication module 206 can be ensured, interference of radio waves between the first wireless communication module 313 and the second wireless communication module 206 can be suppressed.

Furthermore, the location opposite to the position where the second wireless communication module 206 is installed is not limited to the position with the center ST of the camera device 3 in the left-right direction of the refrigerator 2 as the reference. This reference can also be a predetermined portion of the camera device 3. For example, the reference could be the right side of the camera device 3, or it could be the substrate of the camera device 3. Alternatively, the reference could be a predetermined portion of the refrigerator 2.

The camera device 3 includes a metal part 314. When the camera device 3 is installed in the main housing 20, the metal part 314 is located on the opposite side of the position where the first wireless communication module 313 is located. In this embodiment, as shown in FIG. 7, when the camera device 3 is installed in the refrigerator 2, a metal plate 314 is provided between the first wireless communication module 313 and the second wireless communication module 206 in the left-right direction. The metal plate 314 may also be provided to cover the entire area of the right inner side 315 of the camera component 31. Alternatively, the metal plate 314 may be provided in the right inner side 315 of the camera component 31, within the area including the first wireless communication module 313 from a side view. Furthermore, the metal plate 314 can also be configured to cover the entire area of the right inner side of the imaging device 3. By providing the metal plate 314, heat dissipation within the imaging device 3 can be improved, and interference of radio waves generated between the first wireless communication module 313 and the second wireless communication module 206 can be further suppressed. Additionally, safety can be enhanced through a non-flammable design. The metal plate 314 is equivalent to the "metal part" disclosed herein.

Furthermore, regarding the metal part 314, the position opposite to the location of the first wireless communication module 313 is not limited to the right inner side 315 of the imaging component 31 or the right inner side of the imaging device 3. The position opposite to the location of the first wireless communication module 313 can be any position with a predetermined component within the imaging device 3 as a reference. For example, this reference can also be set as the boundary between the first wireless communication module 313 and other components.

Figure 8 is a block diagram showing the configuration of the imaging device 3. As shown in Figure 8, the first control board 311 includes a first processor 320 such as a CPU (Central Processing Unit), a first memory 330, a first communication module 340, and interface circuits for connecting other devices or sensors.

The first memory 330 is a storage device that stores programs or data executed by the first processor 320. The first memory 330 is constructed using a non-volatile storage device such as ROM (Read Only Memory). The first memory 330 may also include a volatile storage device such as RAM (Random Access Memory) that constitutes the working area of the first processor 320. The first memory 330 stores data processed by the first processor 320 or control programs 331 executed by the first processor 320.

The first communication module 340 has hardware such as a communication circuit or connector according to a predetermined wired communication specification, and communicates with the second control board 312 under the control of the first processor 320.

The first processor 320 reads and executes the control program 331, thereby turning the power of the camera device 3 on and off in response to the operation of the power button 32. The first processor 320 also reads and executes the control program 331, thereby turning the first LED 33, the second LED 308, and the third LED 309 on and off. Furthermore, the first processor 320 reads and executes the control program 331, thereby sending the detection values output by each sensor connected to the first control board 311 to the second control board 312 through the first communication module 340.

The second control board 312 includes a second processor 350 such as a CPU, a second memory 360, a second communication module 370, and interface circuits for connecting other devices or sensors.

The second memory 360 is a storage device that stores programs or data executed by the second processor 350. The second memory 360 is constructed using a non-volatile storage device such as ROM. Alternatively, the second memory 360 may also include a volatile storage device such as RAM that constitutes the working area of the second processor 350. The second memory 360 stores data processed by the second processor 350, or a control program 361 executed by the second processor 350, or a camera device ID (identification) 362. The camera device ID 362 uniquely identifies the camera device 3.

The second communication module 370 has hardware such as a communication circuit or connector according to a predetermined wired communication specification, and communicates with the first control board 311 under the control of the second processor 350.

The second processor 350 reads and executes the control program 361 stored in the second memory 360, thereby performing its functions as the first communication control unit 351, the second communication control unit 352, the on/off determination unit 353, the camera control unit 354, the video recording control unit 355, and the generation unit 356.

The first communication control unit 351 communicates with the item management server 5 through the first wireless communication module 313.

The second communication control unit 352 communicates with the first control board 311 through the second communication module 370. The second communication control unit 352 receives detection values from various sensors connected to the first control board 311.

The opening/closing determination unit 353 determines whether the right door 21B is open or closed based on the detection value received by the first photodetector 303 from the second communication control unit 352. If the second communication control unit 352 receives a detection value different from the value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is open. Conversely, if the second communication control unit 352 receives a detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is closed.

Similar to the determination for the right door 21B, the opening/closing determination unit 353 determines whether the left door 21C is open or closed based on the detection value received by the second photodetector 304 from the second communication control unit 352.

The opening/closing determination unit 353 determines whether the drawer 26A is open or closed based on the detection value of the thermopile 305 received by the second communication control unit 352. If the second communication control unit 352 receives a detection value from the thermopile 305 indicating a temperature below a predetermined temperature, the opening/closing determination unit 353 determines that the drawer 26A is open. Conversely, if the second communication control unit 352 receives a detection value from the thermopile 305 indicating a temperature exceeding a predetermined temperature, the opening/closing determination unit 353 determines that the drawer 26A is closed.

When the opening/closing determination unit 353 determines that door 21A or drawer 26A is open, the camera control unit 354 starts shooting with the wide-angle camera 301 and the narrow-angle camera 302. Additionally, when the camera control unit 354 starts shooting with the wide-angle camera 301 and the narrow-angle camera 302, the second communication control unit 352 sends a lighting request message to the first control board 311. The lighting request message requests the lighting of the second LED 308 and the third LED 309. Upon receiving the lighting request message, the first processor 320 lights up the second LED 308 and the third LED 309 and notifies the user P that shooting has begun.

When the opening/closing determination unit 353 determines that door 21A and drawer 26A are closed, the camera control unit 354 terminates photography by the wide-angle camera 301 and the narrow-angle camera 302. Additionally, when the camera control unit 354 terminates photography by the wide-angle camera 301 and the narrow-angle camera 302, the second communication control unit 352 sends a light-off request message to the first control board 311. The light-off request message requests that the second LED 308 and the third LED 309 be turned off. Upon receiving the light-off request message, the first processor 320 turns off the second LED 308 and the third LED 309 and notifies the user P that photography has ended.

When the opening/closing determination unit 353 determines that door 21A or drawer 26A is open, the recording control unit 355 begins recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302. That is, the recording control unit 355 begins recording video data 363 into the second memory 360. The video data 363 includes first video data showing the shooting results of the wide-angle camera 301 and second video data showing the shooting results of the narrow-angle camera 302.

When the opening/closing determination unit 353 determines that the door 21A and drawer 26A are closed, the recording control unit 355 terminates the recording of the shooting results of the wide-angle camera 301 and the narrow-angle camera 302.

The generation unit 356 generates management data D1. Management data D1 is data uploaded to the item management server 5 and is used to manage the items contained in the refrigerator 2. The first communication control unit 351 sends the management data D1 generated by the generation unit 356 to the item management server 5.

[1-1-4. Composition of the Item Management Server] Figure 9 is a block diagram showing the composition of the terminal device 4 and the item management server 5. The item management server 5 has a processor such as a CPU, namely a server processor 50, server memory 51, and a server communication module 52. The server memory 51 and the server communication module 52 are connected to the server processor 50.

Server memory 51 is a storage device for storing programs or data executed by server processor 50. Server memory 51 is composed of non-volatile storage devices. Furthermore, server memory 51 may also include non-volatile storage devices that constitute the working area of server processor 50. Server memory 51 stores data processed by server processor 50 or control programs 511 and item management DB (database) 512 executed by server processor 50. Item management DB 512 has one record per camera device 3. This record contains camera device ID362, image data of the first camera image, and image data of the second camera image. The first and second camera images will be described later.

The server communication module 52 is equipped with a communication circuit or connector according to a predetermined communication specification, and communicates with the refrigerator 2, the camera device 3 and the terminal device 4 through the network NW.

The server processor 50 reads and executes the control program 511 stored in the server memory 51, thereby functioning as the server communication control unit 501 and the server processing unit 502. The server communication control unit 501 communicates with the refrigerator 2, the camera device 3, and the terminal device 4 through the server communication module 52. The server processing unit 502 uses the camera device 3 and the terminal device 4 as clients to perform various information processing tasks. The server processing unit 502 can also use the refrigerator 2 as a client.

[1-1-5. Composition of the terminal device] Next, the composition of the terminal device 4 will be explained. The terminal device 4 has a processor such as a CPU, namely a terminal processor 40, a terminal memory 41, a terminal communication module 42, and a touch panel 43.

Terminal memory 41 is a non-volatile storage device that stores programs or data executed by terminal processor 40. Terminal memory 41 is constructed using non-volatile storage devices. Furthermore, terminal memory 41 may also include RAM that constitutes the working area of terminal processor 40. Terminal memory 41 stores data processed by terminal processor 40 or control programs and item management applications 411 executed by terminal processor 40.

The terminal communication module 42 has a communication circuit or connector according to a predetermined communication specification, and communicates with the item management server 5 under the control of the terminal processor 40. The touch panel 43 has a display panel and a touch sensor, which are overlapped or integrated on the display panel.

The terminal processor 40 functions as an application execution unit 400. The application execution unit 400 communicates with the item management server 5 via the terminal communication module 42. The application execution unit 400 displays a user interface related to the management of items contained in the refrigerator 2 on the touch panel 43. In the following description, this user interface will be referred to as the "application UI". The display control unit 402 of this embodiment displays the application UI, thereby displaying images of items contained in the refrigerator 2. The application execution unit 400 receives various inputs from the user P through the application UI.

[1-2. Actions] Next, the actions of each part of the item management system 1 will be explained.

[1-2-1. Actions related to recording] First, the actions of the camera device 3 related to recording will be explained. Figure 10 is a flowchart FA showing the actions of the camera device 3.

The opening/closing determination unit 353 determines whether the door 21A or drawer 26A is in the open state (step SA1).

If the opening/closing determination unit 353 determines that the door 21A or drawer 26A is not in an open state (step SA1: no), the second processor 350 returns the processing to step SA1.

When the opening/closing determination unit 353 determines that the door 21A or drawer 26A is in an open state (step SA1: Yes), the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start taking pictures (step SA2).

Next, the recording control unit 355 begins recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 (step SA3).

Next, the opening/closing determination unit 353 determines whether the door 21A and drawer 26A are closed (step SA4). If the opening/closing determination unit 353 determines that the door 21A and drawer 26A are not closed (step SA4: no), the determination in step SA4 is performed again.

On the other hand, if it is determined that door 21A and drawer 26A are closed (step SA4: Yes), the recording control unit 355 ends the recording of the video recording results (step SA5). Thereby, video data 363 is generated in the second memory 360.

Next, the camera control unit 354 terminates the photography of the wide-angle camera 301 and the narrow-angle camera 302 (step SA6).

[1-2-2. Actions Related to the Sending of Management Data] Next, the actions related to the sending of management data D1 will be explained. Figure 11 is a flowchart showing the actions of the camera device 3 and the item management server 5. In Figure 11, flowchart FB shows the actions of the camera device 3, and flowchart FC shows the actions of the item management server 5.

The generation unit 356 determines whether the second memory 360 stores video data 363 (step SB1). If the generation unit 356 determines that no video data 363 is stored (step SB1: no), it performs the determination in step SB1 again.

On the other hand, when the generation unit 356 determines that video data 363 is stored (step SB1: Yes), it selects the first camera image from the dynamic image shown by the first video data of the video data 363 stored in the second memory 360 (step SB2).

Step SB2 is described in detail. The first photographic image is taken with the right door 21B and left door 21C open at a rotation angle of α°≦θ≦β°, and is the photographic image with the least amount of user P in the frame. α° is, for example, 60°. β° is, for example, 100°.

The generation unit 356 selects a photographic image from the dynamic images shown in the first video recording that captures a predetermined mark attached to door 21A. This predetermined mark is pre-attached to the right door 21B and the left door 21C so that it will be captured in the photographic image. Next, the generation unit 356 selects the photographic image with the least amount of user P in the frame from the selected photographic images as the first photographic image. The generation unit 356 identifies user P in the photographic images using methods such as pattern matching, and selects the photographic image with the smallest area of the identified user P as the photographic image with the least amount of user P in the frame.

Returning to the description of the flowchart FB, the generation unit 356 selects the second camera image from the dynamic image shown in the second video data of the stored video data 363 (step SB3).

Step SB3 is described in detail. The second image is taken with the drawer 26A pulled out to a degree of γ% or more, with the pull-out degree set to 0% when closed and 100% when fully open. The pull-out degree refers to the extent to which the drawer is pulled out. Furthermore, the second image is the one with the least amount of user P in the frame. γ% is, for example, 70%.

The generation unit 356 selects a photographic image from the dynamic images shown in the second video data that captures a predetermined marker attached to the drawer 26A. This predetermined marker is pre-attached so that it will be captured in the photographic image when the drawer 26A is pulled out by more than γ%. Next, the generation unit 356 selects the photographic image with the least amount of user P in the frame from the selected photographic images as the second photographic image.

The generation unit 356 determines whether at least one of the first and second photographic images has been selected (step SB4).

If the generation unit 356 determines that it is unable to select the first and second photographic images (step SB4: no), it will proceed to step SB7.

On the other hand, if the generation unit 356 determines that at least one of the first and second photographic images has been selected (step SB4: yes), it generates management data D1 (step SB5).

If both the first and second images have been selected, the generation unit 356 generates image data containing the first image, image data containing the second image, and management data D1 for the camera device ID 362. If the first image has been selected, but the second image cannot be selected, the generation unit 356 generates image data containing the first image and management data D1 for the camera device ID 362. If the second image has been selected, but the first image cannot be selected, the generation unit 356 generates image data containing the second image and management data D1 for the camera device ID 362.

Next, the first communication control unit 351 sends the management data D1 generated by the generation unit 356 to the item management server 5 (step SB6).

Next, the generation unit 356 deletes the video data 363, which was processed after step SB2, from the second memory 360 (step SB7).

As shown in flowchart FC, the server communication control unit 501 determines whether it has received management data D1 from the camera device 3 (step SC1). If the server communication control unit 501 determines that it has received management data D1 (step SC1: yes), the server processing unit 502 updates the item management DB512 based on the management data D1 received by the server communication control unit 501 (step SC2).

Step SC2 is described in detail. The server processing unit 502 identifies the record containing the camera device ID 362 included in the management data D1 from the item management DB512, and updates the image data of the first camera image in the identified record to the image data of the first camera image included in the management data D1. The server processing unit 502 identifies the record containing the camera device ID 362 included in the management data D1 from the item management DB512, and updates the image data of the second camera image in the identified record to the image data of the second camera image included in the management data D1.

[1-2-3. Actions Related to the Application UI] Next, the actions of the item management system 1 related to the application UI will be explained. Figure 12 is a flowchart showing the actions of the terminal device 4 and the item management server 5. In Figure 12, flowchart FD shows the actions of the terminal device 4, and flowchart FE shows the actions of the item management server 5.

As shown in flowchart FD, the application execution unit 400 determines whether to request image data saved in the item management DB512 from the item management server 5 (step SD1). For example, if the item management application 411 has been started, the terminal communication control unit 401 makes an affirmative determination in step SD1.

When the application execution unit 400 determines that it requests image data stored in the item management DB512 (step SD1: Yes), it sends the request data to the item management server 5 (step SD2). The request data includes the camera device ID 362 stored in the terminal memory 41. Furthermore, the terminal device 4 retrieves the camera device ID 362 at a predetermined time.

As shown in flowchart FE, the server communication control unit 501 receives the request data (step SE1). Next, the server processing unit 502 obtains image data from the item management DB 512 that corresponds to the camera device ID 362 contained in the request data received by the server communication control unit 501 (step SE2). Then, the server communication control unit 501 sends the image data obtained by the server processing unit 502 as a response to the request data to the terminal device 4 (step SE3).

Referring to flowchart FD, application execution unit 400 receives image data from item management server 5 (step SD3). Then, application execution unit 400 displays application UI on touch panel 43 (step SD4), the aforementioned application UI containing the image shown in the image data received in step SD3.

[1-3. Effects, etc.] As described above, the item management system 1 includes: a refrigerator 2 with a main body 20; and a camera device 3 installed on the top surface 20A of the main body 20 to photograph the refrigerator 2. The camera device 3 includes: a first wireless communication module 313, which can transmit management data D1 for managing the items contained in the refrigerator 2. When the camera device 3 is installed in the main body 20, the first wireless communication module 313 is located further forward than the opening 201 of the main body 20.

This can suppress interference caused by the top surface 20A of the main housing 20 in the radio waves transmitted and received by the first wireless communication module 313. Accordingly, since the first wireless communication module 313 can stably transmit management data D1, the reduction in the management accuracy of the item can be suppressed.

The refrigerator 2 includes: a right door 21B and a left door 21C, an opening 201; and a second wireless communication module 206, which is disposed on the right door hinge component 204 that supports the right door 21B. When the camera device 3 is installed in the main body 20, the first wireless communication module 313 is located on the left door hinge component 205 that supports the left door 21C, which is closer to the center ST of the camera device 3 in the left-right direction of the refrigerator 2.

In this way, by ensuring the distance between the first wireless communication module 313 and the second wireless communication module 206, interference of radio waves between the first wireless communication module 313 and the second wireless communication module 206 can be suppressed. Accordingly, since the first wireless communication module 313 can transmit management data D1 more stably, the reduction in the management accuracy of the item can be further suppressed.

The refrigerator 2 includes a right door 21B and a left door 21C, an opening 201, and a second wireless communication module 206, which is mounted on the right door hinge component 204 that supports the right door 21B. The camera device 3 includes a metal plate 314. When the camera device 3 is installed in the main body 20, the metal plate 314 is positioned between the first wireless communication module 313 and the second wireless communication module 206 in the left-right direction of the refrigerator 2.

This further suppresses interference of radio waves between the first wireless communication module 313 and the second wireless communication module 206. Consequently, since the first wireless communication module 313 can transmit management data D1 more stably, the reduction in the management accuracy of the item can be further suppressed.

The refrigerator 2 is equipped with: a door 21A and an opening 201. The first wireless communication module 313 is located further forward than the door 21A when the camera device 3 is installed in the main body 20.

This further suppresses interference between the radio waves transmitted and received by the first wireless communication module 313. Consequently, since the first wireless communication module 313 can stably transmit management data D1, the reduction in the management accuracy of the item can be further suppressed. Furthermore, even assuming that the frame of door 21A is made of metal, interference between the radio waves transmitted and received by the first wireless communication module 313 can be suppressed, thus reducing the reduction in the management accuracy of the item.

(Implementation Form 2) Next, an explanation will be given regarding Implementation Form 2. In the explanation of Implementation Form 2, for the constituent elements that are the same as the constituent elements of the item management system 1 of Implementation Form 1, the same symbols will be attached and detailed explanations will be appropriately omitted.

[2-1. Composition] Figure 13 is a diagram showing the composition of the item management system 1A in Embodiment 2. The item management system 1A in this embodiment includes a refrigerator 2A, an item management server 5, and a terminal device 4. Although the refrigerator 2 and the camera device 3 in Embodiment 1 are separate entities, the refrigerator 2A in Embodiment 2 includes the camera device 3A. That is, the refrigerator 2A and the camera device 3A in this embodiment are not separate entities.

Figure 14 is a block diagram showing the configuration of the refrigerator 2A in this embodiment. The refrigerator 2A has a third control board 220. The third control board 220 has a third processor 230 such as a CPU, a third memory 240, a third communication module 250, and interface circuits for connecting other devices or sensors.

The third memory 240 is a storage device that stores programs or data executed by the third processor 230. The third memory 240 is constructed using non-volatile storage. Alternatively, the third memory 240 may also include volatile storage devices that constitute the working area of the third processor 230. The third memory 240 stores data processed by the third processor 230 or control programs 241 executed by the third processor 230.

The third communication module 250 has communication hardware such as a communication circuit or connector according to a predetermined wired communication specification, and communicates with the first control board 311A under the control of the third processor 230.

The third control board 220 is connected to a second wireless communication module 206, a cooling unit 207, a first on/off sensor 208, a second on/off sensor 209, a third on/off sensor 210, a fourth on/off sensor 211, a fifth on/off sensor 212, and a sixth on/off sensor 213.

The cooling unit 207 includes a compressor or condenser, capillary tubes, a cooler, a cooling fan that delivers cold air generated by the cooler to each compartment, and a damper that diverts the cold air delivered by the cooling fan, thus cooling each compartment of the refrigerator 2A. The cooling unit 207 cools each compartment of the refrigerator 2A under the control of the third control board 220.

The first opening/closing sensor 208 to the sixth opening/closing sensor 213 are constructed using, for example, Hall integrated circuits (Hall ICs). The first opening/closing sensor 208 is used to detect the opening/closing of the right door 21B. The second opening/closing sensor 209 is used to detect the opening/closing of the left door 21C. The third opening/closing sensor 210 is used to detect the opening/closing of drawer 22A. The fourth opening/closing sensor 211 is used to detect the opening/closing of drawer 23A. The fifth opening/closing sensor 212 is used to detect the opening/closing of drawer 24A. The sixth opening/closing sensor 213 is a sensor used to detect the opening/closing of the drawer 25A. Each opening/closing sensor outputs the detected value to the third control board 220.

The third processor 230 reads and executes the control program 241, thereby communicating with external machines through the second wireless communication module 206. Furthermore, the third processor 230 reads and executes the control program 241, thereby cooling each compartment of the refrigerator 2A using the cooling section 207. Also, the third processor 230 reads and executes the control program 241, thereby sending the detection values output by each of the first opening/closing sensors 208 to the sixth opening/closing sensor 213 to the first control board 311A.

Refrigerator 2A is equipped with camera device 3A. Compared with camera device 3, camera device 3A does not have a first light sensor 303, a second light sensor 304, a thermopile 305, a motion sensor 306, a second LED 308, and a third LED 309.

The camera device 3A in Embodiment 2 replaces the first control board 311 and instead has a first control board 311A. The first control board 311A is connected to a power button 32, a first LED 33, and an illuminance sensor 307. Furthermore, the camera device 3A in Embodiment 2 replaces the second control board 312 and instead has a second control board 312A. The second control board 312A is connected to a wide-angle camera 301, a narrow-angle camera 302, and a first wireless communication module 313.

The first control board 311A is disposed in the imaging device 3A at the same position as the first control board 311 in the imaging device 3. Similarly, the second control board 312A is disposed in the imaging device 3A at the same position as the second control board 312 in the imaging device 3. Furthermore, in this embodiment, the wide-angle camera 301, the narrow-angle camera 302, the power button 32, the first LED 33, the illuminance sensor 307, and the first wireless communication module 313 are disposed at the same positions as in Embodiment 1.

The first control board 311A includes a first processor 320A such as a CPU, a first memory 330A, a first communication module 340A, and interface circuits for connecting other devices or sensors.

The first memory 330A is the same memory as the first memory 330. The first memory 330A stores data processed by the first processor 320A or control program 331A executed by the first processor 320A.

The first communication module 340A has communication hardware such as a pre-defined wired communication circuit or connector, and communicates with the second control board 312A and the third control board 220 via a cable.

The first processor 320A reads and executes the control program 331A, thereby turning the power of the camera device 3A on and off in response to the operation of the power button 32. The first processor 320A reads and executes the control program 331A, thereby turning the first LED 33 on and off. The first processor 320A reads and executes the control program 331A, thereby sending the detection value output by the illuminance sensor 307 to the second control board 312A through the first communication module 340A. The first processor 320A reads and executes the control program 331A, thereby sending the detection values of the first on/off sensor 208 to the sixth on/off sensor 213 received from the third control board 220 to the second control board 312A through the first communication module 340A.

The second control board 312A includes a second processor 350A (such as a CPU), a second memory 360A, a second communication module 370A, and interface circuits for connecting other devices or sensors.

The second memory 360A is the same memory as the first memory 330. The second memory 360A stores data processed by the second processor 350A or control program 361A executed by the second processor 350A.

The second communication module 370A has communication hardware such as a communication circuit or connector according to a predetermined communication specification, and communicates with the first control board 311A under the control of the second processor 350A.

The second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby performing its functions as the first communication control unit 351, the second communication control unit 352A, the on/off determination unit 353A, the camera control unit 354, the recording control unit 355, and the generation unit 356.

The second communication control unit 352A communicates with the first control board 311A through the second communication module 370A. The second communication control unit 352A receives detection values from various sensors from the first control board 311A.

The opening/closing determination unit 353A determines whether the right door 21B is open or closed based on the detection value received by the first opening/closing sensor 208 from the second communication control unit 352A. If the second communication control unit 352A receives a detection value different from the value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is open. Conversely, if the second communication control unit 352 receives a detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is closed.

Similar to the determination for the right door 21B, the opening/closing determination unit 353A determines whether the left door 21C is open or closed based on the detection value received by the second opening/closing sensor 209 from the second communication control unit 352.

The opening/closing determination unit 353A determines whether the drawer 26A is open or closed based on the detection values received by the second communication control unit 352 from the third opening/closing sensor 210 to the sixth opening/closing sensor 213. The opening/closing determination unit 353A performs the determination in the same way as the determination of the right door 21B.

[2-2. Operation] The operations of each part of the item management system 1A in this embodiment are the same as the operations of each part of the item management system 1 in embodiment 1. That is, the operation of the camera device 3A related to recording is the same as the operation of the camera device 3 in embodiment 1. Also, the operation of the camera device 3A related to the management data D1 is the same as the operation of the camera device 3 in embodiment 1.

[2-3. Effects, etc.] As explained above, the refrigerator 2A with the main body 20 is equipped with: a camera device 3A, which is installed on the top surface 20A of the main body 20 and takes pictures of the refrigerator 2A. The camera device 3A is equipped with: a first wireless communication module 313, which can transmit management data D1 for managing the items contained in the refrigerator 2A. The first wireless communication module 313 is located further forward than the opening 201 of the main body 20.

In this way, the same effect as the item management system 1 in implementation form 1 can be achieved.

(Other Embodiments) As described above, embodiments 1 and 2 are illustrated as examples disclosed in this application. However, the technology disclosed herein is not limited to these embodiments and can be applied to embodiments that have been modified, substituted, added, or omitted. Furthermore, new embodiments can be created by combining the constituent elements described in embodiments 1 and 2. Therefore, other embodiments are illustrated below.

In embodiments 1 and 2 described above, a metal plate 314 is exemplified as the "metal part" of this disclosure. However, the "metal part" of this disclosure is not limited to the metal plate 314. For example, when the second control board 312 is shielded by metal, the "metal part" of this disclosure can also be this shield. The second control board 312 is disposed in the left-right direction of the refrigerator 2 between the first wireless communication module 313 and the second wireless communication module 206. Therefore, this shield has the same effect as the metal plate 314.

In embodiments 1 and 2 described above, the third LED 309, illuminance sensor 307, motion sensor 306, wide-angle camera 301, narrow-angle camera 302, thermopile 305, and second LED 308 are arranged sequentially at the front end of the imaging component 31. However, this arrangement is only one example, and the arrangement order of these devices is not limited to embodiments 1 and 2.

In the above-described embodiment 2, the first photodetector 303, the second photodetector 304, the thermopile 305, the motion sensor 306, the second LED 308, and the third LED 309 are not included. In other embodiments related to embodiment 2, any of these devices may be included.

In embodiments 1 and 2 described above, the cases in which the widths of the right door 21B and the left door 21C are different are illustrated. In other embodiments, the widths of the right door 21B and the left door 21C may be the same, and the width of the left door 21C may also be longer than the width of the right door 21B.

In embodiments 1 and 2 described above, the second wireless communication module 206 is configured to be installed on the right door hinge component 204. In other embodiments, the second wireless communication module 206 may also be installed on the left door hinge component 205. In this case, the left door 21 corresponds to the "first door" of this disclosure, the right door 21B corresponds to the "second door" of this disclosure, the left door hinge component 205 corresponds to the "first hinge component" of this disclosure, and the right hinge component 204 corresponds to the "second hinge component" of this disclosure.

In embodiments 1 and 2 described above, the camera devices 3 and 3A are configured to have a second LED 308 and a third LED 309. In other embodiments, the second LED 308 and the third LED 309 may also be composed of a single LED.

In Embodiment 1 described above, the sensor used to detect the opening and closing of drawer 26A is exemplified as being constructed of a thermopile 305. However, the sensor used to detect the opening and closing of drawer 26A can also be a range sensor. In this case, the opening/closing determination unit 353 determines whether drawer 26A is open or closed based on the detection value of the range sensor. When drawer 26A has been pulled out, an action is detected. Furthermore, the range sensor can be infrared or ultrasonic.

In Embodiment 2 described above, the first to sixth opening/closing sensors 208 are illustrated as being constructed using Hall effect integrated circuits. However, the first and second opening/closing sensors 208 and 209 can also be other sensors such as range sensors, angle sensors, cameras, or temperature sensors. Furthermore, the third to sixth opening/closing sensors 210 and 213 can also be other sensors such as accelerometers, mechanical switches, angle sensors, cameras, or temperature sensors.

In other embodiments, the camera may end recording or video recording after a predetermined time has elapsed since the door 21A or drawer 26A is opened. For example, the predetermined time could be 1 minute. Alternatively, the predetermined time may be extended if the door 21A or drawer 26A is detected to be open, if someone is present near the refrigerators 2 and 2A, or if user P is taking items out or putting them in. Furthermore, if the door 21A or drawer 26A is detected to be open, if someone is present near the refrigerators 2 and 2A, or if user P is taking items out or putting them in, the camera may end recording or video recording after a predetermined time has elapsed since the detected time.

In other embodiments, the second LED 308 and the third LED 309 may be turned on when the opening/closing determination unit 353 determines that the door 21A or drawer 26A is open. Alternatively, the second LED and the third LED 309 may be turned off when the opening/closing determination unit 353 determines that the door 21A or drawer 26A is closed. Furthermore, the lighting of these LEDs may be controlled according to the camera's recording status. For example, if the camera recording or photography ends after a predetermined time has elapsed since the door 21A or drawer 26A became open, these LEDs may be turned off when the camera recording ends. Similarly, these LEDs can be turned off when the camera finishes taking pictures.

In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A can also function as detection units for detecting items. During the recording period from the start of recording by the wide-angle camera 301 and the narrow-angle camera 302 by the recording control unit 355 until the end, the detection unit can also cut out images of items taken out and placed in the refrigerator compartment 21 from the moving images. Furthermore, if the items are marked, the detection unit can also detect the marked items.

In other embodiments, in addition to the functional units of the second processor 350 in embodiment 1, the second processor 350 can also function as a person presence determination unit. The person presence determination unit determines whether there is a person around the refrigerator 2 based on the detection value of the motion sensor 306. More specifically, the person presence determination unit determines that there is a person around the refrigerator 2 if the detection value of the motion sensor 306 corresponds to the presence of a person, and determines that there is no person around the refrigerator 2 if this is not the case. When the second processor 350 functions as a person presence detection unit, the camera control unit 354 starts recording with the wide-angle camera 301 and the narrow-angle camera 302 when the person presence detection unit determines that someone is present, and stops recording with the wide-angle camera 301 and the narrow-angle camera 302 when the person presence detection unit determines that no one is present. Furthermore, when the second processor 350 functions as a person presence detection unit, the recording control unit 355 starts recording when the person presence detection unit determines that someone is present, and stops recording when the person presence detection unit determines that no one is present. Therefore, even if the opening/closing determination unit 353 makes an error, the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 can be recorded appropriately.

In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A can also function as inventory entry/exit determination units. In this case, the inventory entry/exit determination unit changes the boundary line used for determining inventory entry/exit based on the opening and closing angles of the left door 21C and the right door 21B to determine the entry/exit of items. Furthermore, the inventory entry/exit determination unit changes the boundary line used for determining inventory entry/exit based on the degree to which the drawer 26A is pulled out to determine the entry/exit of items.

In other implementations, the first photographic image can also be modified and uploaded to the item management server 5. Image modification can be exemplified by viewpoint shifting.

In the above embodiments, the first and second photographic images are illustrated for display purposes on the terminal device 4. However, the first and second photographic images can also be used for other purposes, such as detecting items contained in refrigerators 2 and 2A.

In other embodiments, the compartments formed in the main body 20 of the refrigerator 2, 2A may have more than the five types of storage compartments mentioned above, or fewer compartments. Also, in other embodiments, the number of doors provided in the opening 201 of the refrigerator compartment 21 may be one.

The first processor 320, 320A, the second processor 350, 350A, the server processor 50, and the terminal processor 40 can be configured as a single processor or as a plurality of processors. These processors can also be hardware designed to implement corresponding functional units. That is, these processors can also be configured as, for example, ASICs (Application Specific Integrated Circuits) or FPGAs (Field Programmable Gate Arrays).

The configuration of the camera devices 3 and 3A, refrigerators 2 and 2A, terminal device 4, and item management server 5 shown in Figures 8, 9, and 14 is merely an example, and the specific installation method is not particularly limited. That is, it is not necessary to install individual corresponding hardware in each part; the configuration can also be made so that the functions of each part are implemented by a single processor executing a program. Furthermore, a part of the function implemented in software in the above embodiments can be made into hardware, or a part of the function implemented in hardware can be implemented in software.

The action units shown in Figures 10, 11, and 12 are divided according to the main processing content for ease of understanding. The action is not limited by the method of division or name of the processing unit. It can also be divided into more action units according to the processing content. Furthermore, it can be divided into one action unit containing more processing. Moreover, the order of these steps can be appropriately changed without affecting the main theme disclosed herein.

Furthermore, since the above embodiments are used to illustrate the embodiments of the technology disclosed herein, various changes, substitutions, additions, omissions, etc., may be made within the scope of the patent application or its equivalents.

(Note) The following technology is revealed through the above description of the implementation forms.

(Technology 1) An item management system comprising: a refrigerator having a main body; and a camera device installed on the top surface of the main body for photographing the refrigerator, the camera device comprising: a first wireless communication module capable of transmitting management data for managing items contained in the refrigerator, wherein the first wireless communication module, when the camera device is installed in the main body, is positioned further forward than the opening of the main body. With this configuration, interference caused by the top surface of the main body can be suppressed in the radio waves transmitted and received by the first wireless communication module. Accordingly, since the first wireless communication module can stably transmit management data, the reduction in item management accuracy can be suppressed.

(Technology 2) In the item management system of Technology 1, the refrigerator is equipped with a second wireless communication module, and the first wireless communication module is located on the opposite side of the location of the second wireless communication module when the camera device is installed in the main body. With this configuration, since the distance between the first and second wireless communication modules can be ensured, interference of radio waves between the first and second wireless communication modules can be suppressed. Accordingly, since the first wireless communication module can stably transmit management data, the reduction in item management accuracy can be further suppressed.

(Technology 3) The item management system of Technology 2, wherein the refrigerator has: a first door and a second door, the opening of which is opened and closed, and the second wireless communication module is disposed on the first hinge component that supports the first door. When the camera device is installed in the main body, the first wireless communication module is located closer to the second hinge component that supports the second door than the center of the camera device in the left-right direction of the refrigerator. According to this configuration, since the distance between the first wireless communication module and the second wireless communication module can be ensured, interference of radio waves between the first wireless communication module and the second wireless communication module can be suppressed. Accordingly, since the first wireless communication module can stably transmit management data, the reduction in the management accuracy of items can be further suppressed.

(Technology 4) An item management system according to any one of Technologies 1 to 3, wherein the aforementioned camera device has a metal part, which, when the aforementioned camera device is installed in the aforementioned main housing, is disposed on the opposite side of the location where the aforementioned first wireless communication module is disposed. With this configuration, interference with the radio waves transmitted and received by the first wireless communication module can be prevented, and heat dissipation within the camera device can be achieved. Therefore, since the impact of heat within the camera device on the processing of the camera device can be suppressed, the first wireless communication module can stably transmit management data, and the reduction in item management accuracy can be further suppressed. Furthermore, since safety can be improved through a non-flammable design, the reduction in item management accuracy can be further suppressed.

(Technology 5) An item management system according to any one of Technologies 1 to 4, wherein the refrigerator includes: a first door and a second door for opening and closing; and a second wireless communication module disposed on a first hinge member supporting the first door; the camera device having a metal part, wherein, when the camera device is installed in the main body, the metal part is disposed between the first and second wireless communication modules in the left-right direction of the refrigerator. With this configuration, interference of radio waves between the first and second wireless communication modules can be further suppressed. Accordingly, since the first wireless communication module can transmit management data more stably, the reduction in item management accuracy can be further suppressed.

(Technology 6) An item management system according to any one of Technologies 1 to 5, wherein the refrigerator includes: a door, the opening of which is opened and closed, and the first wireless communication module, when the camera device is installed in the main body, is positioned above the top surface of the door. With this configuration, interference between radio waves transmitted and received by the first wireless communication module can be further suppressed. Accordingly, since the first wireless communication module can properly transmit management data, the reduction in item management accuracy can be further suppressed. Furthermore, even if the door is made of metal, interference between radio waves transmitted and received by the first wireless communication module can be suppressed, thus reducing the reduction in item management accuracy.

(Technology 7) A refrigerator having a main compartment includes: a camera device disposed on the top surface of the main compartment for photographing the refrigerator; the camera device also includes: a first wireless communication module capable of transmitting management data for managing items contained in the refrigerator; the first wireless communication module is located further forward than the opening of the main compartment. With this configuration, the refrigerator achieves the same effect as the item management system described in Technology 1.

As described above, the industrial applicability of the present invention, including the item management system and the refrigerator, can be used to suppress the reduction in the management accuracy of the items contained in the refrigerator.

1,1A: Inventory Management System 2,2A: Refrigerator 3,3A: Camera Device 4: Terminal Device 5: Inventory Management Server 6: Communication Device 20: Main Cabinet 20A: Top Surface 20B: Front Edge 21: Refrigerator Compartment 21A: Door 21B: Right Door 21C: Left Door 21D: Rotating Divider 21E: Door Pocket 21F: Shelf 22: Ice Maker 2A, 23A, 24A, 25A, 26A: Drawer; 23: Fresh food freezer; 24: Freezer; 25: Vegetable compartment; 30: Main body; 30A, 31A: Bottom surface; 31: Camera component; 32: Power button; 33: First LED; 34: Restriction component; 35: Marker; 40: Terminal processor; 41: Terminal memory; 42: Terminal communication module; 43: Touch panel 50: Server processor 51: Server memory 52: Server communication module 201: Opening 202: Outer casing 203: Inner casing 204: Right door hinge component 205: Left door hinge component 206: Second wireless communication module 207: Cooling unit 208: First opening/closing sensor 209: Second opening/closing sensor 210: Third 211: 4th Opening/Closed Sensor; 212: 5th Opening/Closed Sensor; 213: 6th Opening/Closed Sensor; 220: 3rd Control Board; 230: 3rd Processor; 240: 3rd Memory; 241, 331, 331A, 361, 361A, 511: Control Program; 250: 3rd Communication Module; 301: Wide-Angle Camera; 302: Narrow-angle camera 303: First light sensor 304: Second light sensor 305: Thermopile 306: Motion sensor 307: Illuminance sensor 308: Second LED 309: Third LED 310: Control board (shielding part) 311, 311A: First control board 312, 312A: Second control board 313: First wireless communication module 314: Metal part 315: Right inner side 320, 320A: First processor 330, 330A: First memory 340, 340A: First communication module 350, 350A: Second processor 351: First communication control unit 352, 352A: Second communication control unit 353, 353A: On/off determination unit 354: Camera Control Unit 355: Recording Control Unit 356: Generation Unit 360, 360A: Second Memory 362: Camera Device ID 363: Recording Data 370, 370A: Second Communication Module 400: Application Execution Unit 411: Item Management Application 501: Server Communication Control Unit 502: Server Processing Unit 512: Item Management DBA1, A2, A3, A4: Scope D1: Management Data FA, FB, FC, FD, FE: Flowchart H: Home NW: Network P: User SA1~SA6, SB1~SB7, SC1, SC2, SD1~SD4, SE1~SE3: Step ST: Center X, Y, Z: Direction

Figure 1 is a diagram showing the composition of the item management system in Embodiment 1. Figure 2 is a perspective view of the refrigerator in Embodiment 1. Figure 3 is a plan view of the camera device in Embodiment 1 viewed from the right. Figure 4 is a perspective view of the limiting member abutting the front edge of the main body in Embodiment 1 viewed from the left front. Figure 5 is a plan view of the refrigerator and camera device in Embodiment 1 viewed from the front. Figure 6 is a plan view of the refrigerator and camera device in Embodiment 1 viewed from the right. Figure 7 is a plan view of the camera device installed in the refrigerator in Embodiment 1 viewed from above. Figure 8 is a block diagram showing the composition of the camera device in Embodiment 1. Figure 9 is a block diagram showing the composition of the terminal device and item management server in Embodiment 1. Figure 10 is a flowchart showing the operation of the camera device in Embodiment 1. Figure 11 is a flowchart showing the operation of the camera device and the item management server in Embodiment 1. Figure 12 is a flowchart showing the operation of the terminal device and the item management server in Embodiment 1. Figure 13 is a diagram showing the structure of the item management system in Embodiment 2. Figure 14 is a block diagram showing the structure of the refrigerator in Embodiment 2.

2: Refrigerator

3: Camera Device

20A: Top surface

21A: Door

21B: Right Gate

21C: Left door

21D: Rotating partition

201:Opening part

204: Right door hinge component

205: Left door hinge component

206: Second Wireless Communication Module

301: Wide-angle camera

302: Narrow-angle camera

305: Thermopile

306: Motion Sensor

307: Illuminance Sensor

308: 2nd LED

309: 3rd LED

311: First Control Board

312: Second Control Board

313: First Wireless Communication Module

314: Metal Section

315: Right inner side

ST: Central

X, Y, Z: Direction (Note: The last line appears to be a typo and should be removed.)

Claims (7)

An inventory management system includes: a refrigerator having a main body; and a camera device mounted on the top surface of the main body for photographing the refrigerator. The camera device comprises: a main body disposed on the top surface of the main body; and a camera component extending forward from the main body and positioned above the front portion of the main body. Inside the aforementioned camera component, there is a first wireless communication module. The first wireless communication module can transmit management data for managing the items contained in the aforementioned refrigerator. The aforementioned camera device has a limiting component that extends downward from the bottom surface of the aforementioned body and abuts against the front edge of the aforementioned main body, thereby limiting the position of the aforementioned camera component in the front-rear direction of the aforementioned refrigerator. When the aforementioned camera device is installed in the aforementioned main body, the aforementioned first wireless communication module is positioned further forward than the aforementioned limiting component and further forward than the opening of the aforementioned main body. As in the item management system of claim 1, the aforementioned refrigerator is equipped with a second wireless communication module, and the aforementioned first wireless communication module is located on the opposite side of the location where the aforementioned camera device is installed in the aforementioned main body. As in the item management system of claim 2, the aforementioned refrigerator has: a first door and a second door for opening and closing the aforementioned opening, the aforementioned second wireless communication module is disposed on the first hinge component that supports the aforementioned first door, and the aforementioned first wireless communication module is located on the side of the second hinge component that supports the aforementioned second door, further from the center of the aforementioned camera device in the left-right direction of the aforementioned refrigerator, when the aforementioned camera device is installed in the aforementioned main body. The item management system of any of the requests 1 to 3, wherein the aforementioned camera device has a metal part, and the aforementioned metal part is located on the opposite side of the location where the aforementioned first wireless communication module is located when the aforementioned camera device is installed in the aforementioned main body. As in the item management system of claim 4, the aforementioned refrigerator has: a first door and a second door for opening and closing the aforementioned opening; and a second wireless communication module disposed on the first hinge component that supports the aforementioned first door; the aforementioned camera device has a metal part; and the aforementioned metal part is disposed between the aforementioned first wireless communication module and the aforementioned second wireless communication module in the left-right direction of the aforementioned refrigerator when the aforementioned camera device is installed in the aforementioned main body. The item management system of any one of claims 1 to 3, wherein the refrigerator has: a door for opening and closing the aforementioned opening, and the aforementioned first wireless communication module being located above the top surface of the aforementioned door when the aforementioned camera device is installed in the aforementioned main body. A refrigerator having a main body includes: a camera device disposed on the top surface of the main body for photographing the refrigerator; the camera device comprising: a main body disposed on the top surface of the main body; and a camera component extending forward from the main body and disposed above the front portion of the main body. Inside the aforementioned camera component, there is a first wireless communication module. The first wireless communication module can transmit management data for managing the items contained in the aforementioned refrigerator. The aforementioned camera device has a limiting component that extends downward from the bottom surface of the aforementioned body and abuts against the front edge of the aforementioned main body, thereby limiting the position of the aforementioned camera component in the front-rear direction of the aforementioned refrigerator. When the aforementioned camera device is installed in the aforementioned main body, the aforementioned first wireless communication module is positioned further forward than the aforementioned limiting component and further forward than the opening of the aforementioned main body.