JP7458730B2 - Information processing device, control method for information processing device, and program thereof - Google Patents

Description

The present invention relates to an information processing device capable of communicating with an imaging device, a method of controlling the information processing device, and a program thereof.

Conventionally, in still-life photography such as product photography, an imaging device such as a digital camera is remotely operated from a computer, and a live view image generated in real time by the imaging processing of the imaging device is enlarged and displayed on the computer display. There is. Additionally, auxiliary lines and pre-prepared images can be superimposed on the live view image (overlay display) to reproduce the shooting situation of a previous shot or to make it easier to decide on the composition. There is an overlay display function that allows you to

For example, Patent Document 1 discloses a method of displaying an overlay image superimposed on a remote live view and hiding the overlay image depending on the type of operation.

Japanese Patent Application Publication No. 2011-155595

However, with the technology disclosed in Patent Document 1, when an overlay image is displayed at the start of remote live view, if the overlay image is a highly private or confidential image, it may be viewed by a third party. There is a problem.

The present invention has been made in view of such problems, and aims to protect privacy and improve convenience in an information processing apparatus having an overlay display function.

The information processing device of the present invention is an information processing device capable of communicating with an imaging device, and comprises: a storage means for storing overlay information in a storage unit, the overlay information including information on an overlay image and information on auxiliary lines based on an image previously stored in a storage medium; a control means for controlling the display of the overlay image and the auxiliary lines on a display unit by superimposing them on a captured image obtained from the imaging device based on the overlay information stored in the storage unit by the storage means; and a switching means for accepting an instruction to switch between displaying and not displaying each of the overlay image and the auxiliary lines in accordance with a first operation of a user, wherein the control means further controls the display of the overlay image or the auxiliary lines in accordance with the switching instruction, and further controls the display of the overlay image or the auxiliary lines in accordance with a second operation of the user that is different from the first operation, and controls the deletion of information on the overlay image from the overlay information stored in the storage unit by the storage means, and controls the display of the overlay image in accordance with a second operation of the user that is different from the first operation.

Information processing devices with an overlay display function can protect privacy and improve convenience.

1 is a diagram showing the overall configuration of a photographing system. FIG. 2 is a block diagram showing the configuration of a camera body. FIG. 1 is a block diagram showing the configuration of an information processing device. FIG. 3 is a diagram showing an example of a live view screen. It is a figure which shows an example of an overlay image. FIG. 3 is a diagram showing an example of a live view screen. FIG. 3 is a diagram showing an example of a live view screen. FIG. 3 is a diagram showing an example of a live view screen. 3 is a flowchart showing processing executed in the photographing system. 4 is a flowchart showing a process executed in the imaging system. It is a figure which shows an example of a retention period setting screen.

A preferred embodiment of the present invention will now be described with reference to the accompanying drawings.

FIG. 1 is a diagram showing the overall configuration of a photographing system 100 of this embodiment.
The imaging system 100 of this embodiment includes an information processing device 10 and an imaging device 20, and the information processing device 10 is connected to the imaging device 20 by wire or wirelessly so that they can communicate with each other.

In this embodiment, the information processing device 10 is a personal computer (PC). The information processing device 10 includes a PC main body 50 and a display 51.
In this embodiment, the imaging device 20 is an interchangeable lens digital camera. The imaging device 20 includes an exchangeable lens 1 and a camera body 2.

In this imaging system 100, various operations of the imaging device 20 are controlled by remote operation from the information processing device 10. Further, when the camera body 2 transmits a live view image to the information processing device 10, the live view image is displayed on the display 51. In this embodiment, the display 51 corresponds to a display section.

The interchangeable lens 1 includes a zoom lens 11 for varying the focal length, an aperture mechanism 12 for controlling brightness, and a focus lens 13 for focusing on a subject. Furthermore, the mount section 14 of the replaceable lens 1 can be mechanically joined to the mount section 27 of the camera body 2.

A contact group 15 is incorporated into the mount portion 14 . This contact group 15 is an electronic contact for receiving power supply from the camera body 2 and performing communication. The contact group 15 is divided into uses such as power supply, ground, transmission, reception, and clock. The contact group 15 of the mount section 14 is connected to the connection terminals in the contact group 28 on the mount section 27 of the camera body 2, respectively. As a result, power is supplied from the camera body 2 to the exchangeable lens 1 via the connection terminal, and communication between the exchangeable lens 1 and the camera body 2 becomes possible.

The camera body 2 includes a half mirror 21 that reflects the light that has passed through the replaceable lens 1, a focusing plate 24 that is placed at a position that focuses the reflected light, and a mirror that reflects the light that has passed through the replaceable lens 1. It has an image sensor 23 arranged at a position where an image is formed. The light focused on the focusing plate 24 is inverted by the prism 25 and observed as an erect image through the eyepiece 26. Furthermore, during photographing, the half mirror 21 flips up, the shutter 22 opens, and the light that has passed through the replaceable lens 1 forms an image on the image sensor 23.

FIG. 2 is a block diagram showing the configuration of the camera body 2. As shown in FIG.
The image sensor 23 is a sensor such as a CCD or CMOS, and the light imaged on the image sensor 23 is converted into an amount of charge corresponding to the amount of incident light in each pixel within the image sensor 23. A signal generated by the timing generator (TG) 32 drives the image sensor 23, transmits the charge accumulated in the sensor, and is sequentially converted into a voltage signal. This converted voltage signal is sampled by correlated double sampling (CDS/AGC) 30 and converted into a digital signal by an A/D converter 31. The image data converted into a digital signal is first input to the IC 33, enters the WB circuit 33a for calculating data necessary for white balance processing, and is temporarily stored in the first memory 35. The image data stored in the first memory 35 is then inputted to the IC 33 again and subjected to the following three types of image processing under the control of the CPU 36.

First, the image data converted into a digital signal is converted into losslessly compressed RAW data by a lossless compression circuit 33d which applies lossless compression (reversible compression) as it is, and is sent to the CPU bus 34.
Further, the image data converted into a digital signal is down-sampled by averaging the RAW data within a block or applying a low-pass filter to reduce the band, and then thinning out the band. Then, the RAW thumbnail circuit 33c performs thinning processing to convert the image into a thumbnail image smaller than the original image size, and the resulting image is sent to the CPU bus 34.
Finally, the image data converted into a digital signal is subjected to image processing in an image processing circuit 33b for JPEG compression. The YcbCr output as a result is converted into a raster block, subjected to JPEG compression by the JPEG compression circuit 33e, and sent to the CPU bus 34.

The CPU 36 controls the entire imaging device 20. The CPU 36 executes a program stored in the second memory 37, thereby controlling each component of the imaging device 20.
The CPU 36 initializes the camera body 2 and starts communication with the interchangeable lens 1 .
Furthermore, the CPU 36 stores the image data captured by the image sensor 23 and sent to the CPU bus 34 in the first memory 35. Then, the image data is written to the external memory 39 via the I/F circuit 38. The switch 40 connected to the CPU 36 is a release switch. By pressing the switch 40, the above-mentioned photographing operation is performed and the image data is written to the external memory 39.

During live view display, the CPU 36 flips up the half mirror 21, opens the shutter 22, and focuses the light that has passed through the interchangeable lens 1 on the image sensor 23, converts it into a digital signal, and acquires a live view image. The CPU 36 then stores the live view image in the second memory 37 and displays it on a rear display (not shown) installed on the camera body 2. The CPU 36 updates the live view image at regular intervals. The user can check the subject through the live view image on the rear display in the same way as checking the subject through an optical viewfinder.

FIG. 3 is a block diagram showing the configuration of the information processing device 10 (PC main body 50).
The information processing device 10 includes a CPU 301, a memory 302, a non-volatile memory 303, an image processing unit 304, a display I/F 305, an operation unit 306, a recording medium I/F 307, an external I/F 309, and a communication I/F 310. , each component is connected to an internal bus 312. Each component can exchange data with each other via an internal bus 312.

The CPU 301 controls the entire information processing device 10 . The CPU 301 controls each component of the information processing apparatus 10 by executing a program stored in the nonvolatile memory 303, and functions as a setting means and a control means in the present invention.
The memory 302 is composed of a RAM (volatile memory using a semiconductor element), and is used as a work memory for the CPU 301.
The non-volatile memory 303 is composed of a ROM, an HDD, etc., and stores various programs for controlling the CPU 301, image data, and various other data.

The image processing unit 304, under the control of the CPU 301, processes image data stored in the nonvolatile memory 303 and the recording medium 308, video signals acquired via the external I/F 309, and images acquired via the communication I/F 310. Perform various image processing on data etc. Image processing performed by the image processing unit 304 includes A/D conversion processing, D/A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement/reduction processing (resizing), noise reduction processing, and color conversion. This includes processing, etc. The image processing unit 304 may be configured with a dedicated circuit block for performing specific image processing. Further, depending on the type of image processing, the CPU 301 may perform image processing according to a program without using the image processing unit 304.

Display I/F 305 is an interface for outputting a video signal to be displayed on display 51. On the display 51, under the control of the CPU 301, a live view image, a GUI screen forming a GUI (Graphical User Interface), and the like are displayed. The CPU 301 generates a display control signal according to a program, converts it into a video signal, and controls the signal to be output to the display 51. The display 51 displays a video based on the output video signal.

The operation unit 306 is an input device for receiving user operations, such as a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, or a touch panel. Note that the touch panel is an input device that is configured in a planar manner so as to be superimposed on the display 51, and outputs coordinate information according to a touched position.

The storage medium I/F 307 allows a recording medium 308 such as a memory card, CD, or DVD to be attached, and performs operations such as reading data from the attached recording medium 308 and writing data to the recording medium 308 under the control of the CPU 301.
The external I/F 309 is an interface for connecting to an external device via a wired cable or wirelessly, and for inputting and outputting video signals and audio signals.
The communication I/F 310 is an interface for communicating with external devices, the Internet 311, etc., and transmitting and receiving various data such as files and commands.

In this embodiment, the information processing device 10 is connected to the camera body 2 via an external I/F 309. The CPU 301 executes a remote live view function and receives live view images from the camera body 2 at regular intervals. The CPU 301 displays the received live view image on the display 51 via the display I/F 305. Further, the CPU 301 controls the live view image to display auxiliary lines, images, etc. in a superimposed manner (overlay display). Note that the live view image is an example of a photographed image in the present invention. The captured image is not limited to a live view image as long as it is an image captured by the image sensor 23.

A method of controlling overlay display will be described below with reference to FIGS. 4 to 8.
FIG. 4 is a diagram showing an example of a live view screen 410 for operating and displaying live view images.

The live view screen 410 displays a live view image 411 received from the camera body 2. An overlay image operation area 420 is also provided for performing display operations related to the overlay image, which is an image displayed as an overlay. A grid line operation area 430 is also provided for performing display operations of grid lines, which are lattice-shaped auxiliary lines that can be set to any interval by user operation. A guide line operation area 440 is also provided for performing display operations of guide lines, which are auxiliary lines that can be moved to any position or angle by user operation.

The overlay image operation area 420 includes a reference information display area 421, an overlay image setting clear button 422, an overlay image retention period setting button 423, an image display ON button 424 for switching between displaying and non-displaying the overlay image, and an image display OFF button. A button 425 and the like are provided.
In the reference destination information display area 421, a character string (image path) indicating the reference destination image of the overlay image is displayed. When the reference destination image of the overlay image is specified, the CPU 301 sets an image path indicating the reference destination image of the overlay image as reference destination information. The non-volatile memory 303 holds information on deformations applied to the overlay image (display position, display magnification, rotation angle, background transmittance, etc.) in addition to set reference destination information. In this embodiment, the nonvolatile memory 303 corresponds to a storage section.

The grid line operation area 430 is provided with a grid line display ON button 431 and a grid line display OFF button 432 for switching between displaying and non-displaying the grid lines. The guide line operation area 440 is provided with a guide line display ON button 441 and a guide line display OFF button 442 for switching between displaying and non-displaying the grid lines.

FIG. 5 is a diagram illustrating an example of an overlay image 450 that is displayed superimposed on the live view image 411. The file format of overlay image 450 is not limited to an image file in a general image file format such as JPEG format. Lossless compression (reversible compression) format image files and RAW data generated by the camera body 2 can also be read and used as images for overlay display.

FIG. 6 is a diagram showing a state in which the live view screen 410 shown in FIG. 4 is displayed and the overlay image 450 shown in FIG. 5 is displayed in a superimposed manner. When the image display ON button 424 is enabled with the character string displayed in the reference destination information display area 421 of the overlay image operation area 420, the overlay image 450 is displayed as an overlay on the live view image 411.

FIG. 7 is a diagram showing a state in which the overlay image 450 is displayed superimposed on the live view image 411 as shown in FIG. 6, and the grid lines 433 and guide lines 443 are displayed superimposed.
When the grid line display ON button 431 is enabled, grid lines 433 are displayed as an overlay on the live view image 411. Further, when the guide line display ON button 441 is enabled, a guide line 443 is displayed as an overlay on the live view image 411.

The non-volatile memory 303 stores information about the display settings of auxiliary lines, such as the spacing between grid lines and the positions and angles of guide lines. In this embodiment, the above-mentioned reference information of the overlay image, the transformation information of the overlay image, and the information about the display settings of auxiliary lines correspond to the overlay information as referred to in the present invention.

FIG. 8 is a diagram showing a state in which the overlay image 450 is displayed superimposed on the live view image 411 as shown in FIG. 6, and the state after the setting of the reference destination information of the overlay image 450 is canceled. When the overlay image setting clear button 422 is pressed by operating the operation unit 306, the CPU 301 cancels the image path that was set as reference destination information for the overlay image 450. In this way, the overlay image setting clear button 422 functions as an instruction means for canceling the setting of the reference destination information of the overlay image 450. Further, the CPU 301 erases the character string displayed in the reference destination information display area 421. As a result, overlay image 450 is no longer displayed. On the other hand, even if the overlay image setting clear button 422 is pressed by operating the operation unit 306, the information regarding the display setting of the auxiliary line is not cleared and is retained.

Next, the flow of processing executed by the camera body 2 and the information processing device 10 in the photographing system 100 of this embodiment will be described with reference to the flowcharts of FIGS. 9 and 10.

The processes in the flowcharts of Figures 9 and 10 are realized by the CPU 301 of the information processing device 10 expanding a program stored in the non-volatile memory 303 into the memory 302, executing it, and controlling each functional block. Also, the processes are realized by the CPU 36 of the camera body 2 executing a program stored in the second memory 37, and controlling each functional block.

9 is a flowchart showing a process for canceling the setting of the reference destination information of an overlay image by a user operation. First, the camera body 2 is connected to the information processing device 10 via the I/F circuit 38 so as to be able to communicate with the information processing device 10.
In step S101, the camera body 2 and the information processing device 10 are powered on by a user operation, and become ready to operate. Then, the process proceeds to step S102.

In step S102, the CPU 36 of the camera body 2 notifies the information processing device 10 of current camera setting information as initial communication. The CPU 301 of the information processing device 10 receives the camera setting information. Thereby, the state of the camera body 2 is grasped on the information processing device 10 side, and preparations for starting remote live view are completed. After that, the process advances to step S103.
Note that the camera setting information includes various types of information such as a shooting mode setting state, white balance setting information, and a live view state. When the settings of the camera body 2 are changed by a user operation or when the settings of the camera body 2 are changed from the information processing device 10, an update notification of camera setting information is sent from the camera body 2 to the information processing device 10. It will be done.

In step S103, the CPU 301 of the information processing device 10 starts an application for executing the remote live view function, and causes the display 51 to display the live view screen 410. Then, the CPU 301 notifies the camera body 2 of the start of remote live view.
The CPU 36 of the camera body 2 starts live view display, and transmits live view images to the information processing device 10 at regular intervals.
The CPU 301 of the information processing device 10 receives a live view image from the camera body 2 at regular intervals, and updates the display of the live view image 411 on the live view screen 410 at regular intervals.

Note that the overlay information that was set when the previous remote live view ended is read when the remote live view starts in this step. This allows the previous shooting situation to be automatically reproduced when starting remote live view. After that, the process advances to step S104.

In step S104, the CPU 301 of the information processing apparatus 10 determines whether there is a user operation. If the CPU 301 determines that there is a user operation, the process advances to step S105. If the CPU 301 determines that there is no user operation, the process advances to step S108.

In step S105, the CPU 301 of the information processing apparatus 10 determines whether the overlay image setting clear button 422 has been pressed by a user operation. If the CPU 301 determines that the overlay image setting clear button 422 has been pressed, the process advances to step S106. If the CPU 301 determines that the overlay image setting clear button 422 has not been pressed, the process advances to step S108.

In step S106, the CPU 301 of the information processing device 10 determines whether reference destination information for the currently displayed overlay image has been set. If the CPU 301 determines that reference destination information has been set, the process advances to step S107. If the CPU 301 determines that reference destination information is not set, the process advances to step S108.

In step S107, the CPU 301 of the information processing device 10 deletes the reference destination information of the overlay image and also erases the character string displayed in the reference destination information display area 421. After that, the process advances to step S108.

In step S108, the CPU 301 of the information processing device 10 determines whether the display conditions for the overlay image are valid, and whether the image display ON button 424 is valid and the reference destination information for the overlay image is set. Determine whether or not. If the CPU 301 determines that the overlay image display conditions are invalid, the process advances to step S109. If the CPU 301 determines that the overlay image display conditions are valid, the process advances to step S110.

In step S109, the CPU 301 of the information processing apparatus 10 hides the currently displayed overlay image. The process advances to step S111.
In step S110, the CPU 301 of the information processing device 10 reads an image from the storage medium 308 according to the reference destination set as an overlay image, and processes the read image according to the transformation information to generate an overlay image. . Then, the generated overlay image is displayed as an overlay on the live view image 411. The process advances to step S111.

In step S111, the CPU 301 of the information processing device 10 determines whether an operation to end live view from the camera body 2 or an operation to end live view from the information processing device 10 has been performed. If the CPU 301 determines that a termination operation has been performed, the process advances to step S112. If the CPU 301 determines that the termination operation has not been performed, the process returns to step S104.

In step S112, the CPU 301 of the information processing device 10 hides the live view screen 410, ends the application, and notifies the camera body 2 of the end of the live view. After that, the process proceeds to step S113.
In step S113, the CPU 301 of the information processing device 10 disconnects the communication connection with the camera body 2. After that, the process proceeds to step S114.
In step S114, the power supply to the camera body 2 and the information processing device 10 is turned off by a user operation, and the series of processes ends.

According to the process shown in FIG. 9, the user can cancel the setting of the reference information for the overlay image with a simple operation. This makes it possible to avoid a situation where an image with high privacy or confidentiality that was previously set is unintentionally displayed the next time remote live view is started.

FIG. 10 is a flowchart showing a process of automatically canceling the setting of reference destination information of an overlay image by setting a retention period for the overlay image.
First, in S201 to S203, processing similar to the processing in S101 to S103 in FIG. 9 is executed.

In step S204, the CPU 301 of the information processing apparatus 10 determines whether there is a user operation. If the CPU 301 determines that there is a user operation, the process advances to step S205. If the CPU 301 determines that there is no user operation, the process advances to step S207.

In step S205, the CPU 301 of the information processing apparatus 10 determines whether the overlay image retention period setting button 423 has been pressed. If the CPU 301 determines that the overlay image retention period setting button 423 has been pressed, the process advances to step S206. If the CPU 301 determines that the overlay image retention period setting button 423 has not been pressed, the process advances to step S207.
In step S206, the CPU 301 of the information processing apparatus 10 causes the display 51 to display the retention period setting screen shown in FIG. 11, and sets the retention period for the reference destination information of the overlay image by user operation. After that, the process advances to step S207.

FIG. 11 is an example of a retention period setting screen 460 for setting a retention period of reference destination information of an overlay image. By selecting the check box 461, the retention period can be selectively set to either "always retain" or "retain until application ends." Note that in this embodiment, the settable retention period is either "always retained" or "retained until the application ends," but is not limited to these. The retention period may be freely set by user operation.

In steps S207 to S212, processing similar to S108 to S113 in FIG. 9 is executed.
In step S213, the CPU 301 of the information processing apparatus 10 determines whether the retention period of the reference destination information of the overlay image is set to "retain until application ends." If the CPU 301 determines that the setting is "retain until application ends", the process advances to step S214. If the CPU 301 determines that "retain until application ends" is not set, the process advances to step S215.

In step S214, the CPU 301 of the information processing apparatus 10 deletes the reference destination information of the overlay image, and then the process proceeds to step S215.
In step S215, the power supply to the camera body 2 and the information processing device 10 is turned off by a user operation, and the series of processes ends.

According to the process shown in FIG. 10 as described above, it becomes possible to automatically cancel the setting of the reference destination information of the overlay image after the retention period has elapsed. This prevents the situation where even if the user forgets to clear the reference information, the previously set highly private or confidential image will be displayed without them realizing it when starting the next live view. can be avoided.

Although the present invention has been described above in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and the present invention may take various forms without departing from the gist of the present invention. included. Some of the embodiments described above may be combined as appropriate.
Further, a case where a software program that implements the functions of the above-described embodiments is supplied directly from a recording medium or using wired/wireless communication to a system or device having a computer capable of executing the program, and the program is executed. Also included in the present invention.
Therefore, in order to realize the functional processing of the present invention on a computer, the program code itself that is supplied and installed in the computer also realizes the present invention. In other words, the present invention also includes a computer program itself for realizing the functional processing of the present invention. In this case, the form of the program does not matter, such as an object code, a program executed by an interpreter, or script data supplied to the OS, as long as it has the function of a program.

The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical/magnetic optical storage medium, or a nonvolatile semiconductor memory.
Further, as a method of supplying the program, a method may be considered in which a computer program forming the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

1: Interchangeable lens 2: Camera body 10: Information processing device 20: Imaging device 36: CPU of imaging device
37: Second memory of the imaging device 51: Display 100: Photographing system 301: CPU of the information processing device
303: Non-volatile memory of information processing device 410: Live view screen 411: Live view image 420: Overlay image operation area 421: Reference information display area 422: Overlay image setting clear button 423: Overlay image retention period setting button 430: Grid Line operation area 433: Grid line 440: Guide line operation area 443: Guide line 450: Overlay image 460: Retention period setting screen

Claims (6)

An information processing device capable of communicating with an imaging device,
holding means for holding in a storage unit overlay information including information regarding an overlay image and information regarding auxiliary lines based on an image previously stored in a storage medium;
control means for controlling the overlay image and the auxiliary line to be displayed on the display unit in a superimposed manner on the photographed image acquired from the imaging device, based on overlay information held in the storage unit by the holding unit; ,
a switching unit that receives an instruction to display or hide each of the overlay image and the auxiliary line according to a first operation by a user;
Equipped with
The control means further controls to hide the overlay image or the auxiliary line according to the switching instruction,
Furthermore, in accordance with a second operation different from the first operation by the user, the holding means controls to delete information regarding the overlay image from among the overlay information held in the storage unit, and An information processing device characterized by controlling an image to be hidden. The information processing apparatus according to claim 1, wherein the information regarding the overlay image is reference destination information indicating a reference destination of the overlay image in the storage medium. an image path indicating a reference image of the overlay image is displayed on the display section;
3. The information processing apparatus according to claim 2, wherein the control means controls the image path to be hidden in accordance with the second operation. 4. The information processing apparatus according to claim 1, wherein the information regarding the auxiliary line indicates at least one of an interval, a position, and an angle at which the auxiliary line is displayed. A method for controlling an information processing device capable of communicating with an imaging device, the method comprising:
a retaining step of retaining in the storage unit overlay information including information regarding the overlay image and information regarding the auxiliary line based on the image previously stored in the storage medium;
a control step of controlling the overlay image and the auxiliary line to be displayed on the display unit in a superimposed manner on the photographed image acquired from the imaging device, based on the overlay information held in the storage unit in the holding step; ,
a switching step of receiving an instruction to display or hide each of the overlay image and the auxiliary line according to a first operation by a user;
including;
The control step further controls to hide the overlay image or the auxiliary line according to the switching instruction,
Furthermore, in accordance with a second operation different from the first operation by the user, control is performed to delete information regarding the overlay image from among the overlay information held in the storage unit in the holding step , and A method for controlling an information processing device, comprising controlling an image to be hidden. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 4 .

Images