JP2013169013A - Image processor, control method, and program - Google Patents

Abstract

An image captured by a plurality of cameras from different angles is well synchronized.
An image processing apparatus includes: an input device for selecting an image frame; and a plurality of groups of asynchronous images obtained by shooting a subject and a golf ball asynchronously from each other in a plurality of shooting directions. The image is converted into a plurality of groups of still images, and a swing target extraction screen composed of the “front still image” and the image feed gauge 30 is displayed on the display device 16 and selected via the input device 15 based on the swing target extraction screen. The swing image synchronization screen composed of “impact moment” of the “front still image”, “rear still image”, “ball still image” and image feed gauges 40 and 50 is displayed on the display means, and “front still image” is displayed. ”And the“ backward still image ”and“ ball still image ”selected on the swing image synchronization screen are associated as synchronized still images. Includes a U11, the.
[Selection] Figure 12

Description

  The present invention relates to an image processing apparatus and a program.

  A person performing a motion of swinging a golf club to hit a golf ball (hereinafter referred to as a golf swing) is captured as an image such as a still image or a moving image, and the posture during the golf swing, the position of the golf club head, and the like are confirmed. A golf swing analysis system that enables this is known. As a conventional golf swing analysis system, for example, a configuration in which a golf swing is analyzed by digitizing a plurality of points operating on a moving image and comparing them with predetermined numerical data (for example, Patent Document 1), and a golf club provided with a microphone The configuration in which the golf swing is analyzed by judging the timing at which the microphone acquires the sound generated at the moment when the head hits the golf ball (impact moment) as the impact moment (for example, Patent Document 2), and the golf swing image and the analysis result There is a configuration (for example, Patent Document 3) that displays in contrast on one screen. Some of these golf swing analysis systems image a person (subject) performing a golf swing from different angles using a plurality of cameras.

Japanese Patent No. 2794018 JP 2006-263169 A Japanese Patent No. 2003-088604

  By the way, in a golf swing analysis system that images a subject using a plurality of cameras, there is a demand for synchronizing images captured by the respective cameras. This is because by confirming images from a plurality of different viewpoints at the same timing, the posture of the subject during the golf swing can be grasped more accurately. There is also a golf swing analysis system that has the function of grasping the positional relationship between the golf ball and the golf club head at the moment of impact by enlarging the vicinity of the pin on which the golf ball is placed in addition to the subject during the golf swing. However, in such a system, there is a demand for synchronizing an image obtained by photographing the posture of a subject at the moment of impact with an image obtained by enlarging the positional relationship between the golf ball and the head of the golf club at the moment of impact. This is because the state of the golf swing can be grasped more accurately by linking the posture of the subject during the golf swing and the positional relationship between the golf ball and the head of the golf club.

However, in the conventional golf swing analysis system, in order to synchronize a plurality of images captured by a plurality of cameras, a complicated configuration is required, so that it cannot be easily introduced.
For example, when acquiring the impact moment by sound and synchronizing the image, each camera needs to be equipped with a microphone, and since it is necessary to continuously record the image and sound in time series, A high-performance camera and recording apparatus are required, resulting in high costs. In addition, depending on the positional relationship of the cameras, the arrival timing of the sound accompanying the impact moment may be shifted and synchronization may be incomplete.
On the other hand, there is a method to analyze images taken by each camera with a dedicated algorithm and synchronize the images of multiple cameras based on the analysis results. In such a system, the point in the image for analysis is Since the angle of each camera is almost fixed, the degree of freedom is low. In addition, depending on the height of the subject, the motion of the golf swing, and other conditions, synchronization based on the analysis result may be inaccurate, and reliability is lacking. In addition, the development of software for processing with a dedicated algorithm increases the cost of the system.

  A problem of the prior art in a system that synchronizes a plurality of images is not limited to a golf swing analysis system, but is a universal problem that occurs in a configuration in which a moving locus of a moving object is imaged from different angles by a plurality of cameras.

  An object of the present invention is to satisfactorily synchronize images taken from different angles by a plurality of cameras.

In order to solve the above-described problem, an image processing apparatus according to the invention described in claim 1 includes operation input means for selecting an image frame,
Obtaining a still image of each of the plurality of groups from a plurality of groups of asynchronous moving images obtained by photographing a subject to be photographed from a plurality of photographing directions; and at least one still image operation scene and an adjustment frame for image frame feeding; A first operation screen comprising: a reference operation scene displayed on the display means, and a reference operation scene selected via the operation input means based on the first operation screen; and an operation scene of another still image corresponding to the reference operation scene; A control means for displaying on the display means a second operation screen comprising an adjustment gauge for image frame feeding, and associating the reference motion scene and the one still image selected on the second operation screen as a synchronized still image; It is characterized by providing.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the control unit reproduces a plurality of groups of images in synchronization based on the synchronized still images.

The invention according to claim 3 is the image processing apparatus according to claim 1 or 2, wherein the first operation screen can select one still image for each of a plurality of operation scenes,
The reference motion scene is any one of a plurality of still images selected on the first operation screen.

  According to a fourth aspect of the present invention, in the image processing device according to the third aspect, the latest motion scene is obtained from a still image obtained by capturing the most recent motion scene among the plurality of still images selected on the first operation screen. An extraction means for extracting a plurality of still images up to the captured still image is provided.

  According to a fifth aspect of the present invention, in the image processing device according to the fourth aspect, the still images of each group are assigned consecutive numbers according to the time series of imaging, and the control means is selected on the second operation screen. The number of the one still image, the number of the still image of the reference operation scene, the number of the still image obtained by capturing the most recent operation scene among the plurality of still images selected on the first operation screen, and the latest operation scene A plurality of images of the same operation scene as the plurality of still images extracted by the extraction unit among the still images selected in the second operation screen and the same group as the still images selected on the second operation screen. The still image is extracted.

The program according to the invention described in claim 6 is a computer,
At least one still image is obtained by obtaining an operation input means for selecting an image frame and a plurality of groups of asynchronous moving images obtained by photographing a subject to be photographed from a plurality of photographing directions. A first operation screen composed of an operation scene and an adjustment gauge for image frame sending is displayed on the display means, and a reference operation scene selected via the operation input means based on the first operation screen and the reference operation A second operation screen including an operation scene of another still image corresponding to the scene and an adjustment gauge for image frame feeding is displayed on the display means, and one still image selected on the reference operation scene and the second operation screen is displayed. It is characterized by functioning as control means for associating an image with a synchronized still image.

  According to the present invention, it is possible to satisfactorily synchronize images taken from different angles by a plurality of cameras.

It is a figure which shows the golf swing analysis system using the image processing apparatus which is one Embodiment of this invention. It is a block diagram which shows the structure of an image processing apparatus. It is a flowchart which shows the processing content of an image processing apparatus. An example of the reproduction screen D1 of the moving image file created by the synchronous moving image creation is shown. It is a block diagram which shows the memory content of a memory | storage device. It is a figure which shows an example of a reference | standard position information table. FIG. 6A is a diagram illustrating an example of the initialized reference position information table, and FIG. 6B is a diagram illustrating an example of the reference position information table after extraction of the swing target, and FIG. FIG. 5 is a diagram showing an example of a reference position information table after swing image synchronization. It is a figure which shows an example of 1st coordinate data and 2nd coordinate data. FIG. 7A is a diagram illustrating an example of the first coordinate data, and FIG. 7B is a diagram illustrating an example of the second coordinate data. It is a figure which shows an example of the screen which drawn the movement locus | trajectory of the golf club head and the golf ball in the “front still image”. It is a figure which shows an example of the screen which drew the movement locus | trajectory of the golf club head and the golf ball in the “rear still image”. It is a figure which shows an example of the screen which drew the movement locus | trajectory of the golf club head and the golf ball in the “ball still image”. It is a figure which shows an example of the display content of a swing object extraction screen, and its transition. FIG. 11A is a diagram showing the initial display contents, FIG. 11B is a diagram showing the selection of the “takeback start” still image, and FIG. 11C is the “takeback start” and “down”. It is a figure which shows after still image selection of "start", "impact moment", and "follow-through end". It is a figure which shows an example of a swing image synchronous screen. It is a flowchart which shows the processing content from step S21 to step S31 among the processing content of swing object extraction. It is a flowchart which shows the processing content from step S32 to step S44 among the processing content of swing object extraction. It is a flowchart which shows the processing content from step S51 to step S60 among the processing content of swing image synchronization. It is a flowchart which shows the processing content from step S61 to step S73 among the processing content of swing image synchronization.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a golf swing analysis system 1 using an image processing apparatus according to an embodiment of the present invention. The golf swing analysis system 1 includes three cameras 2, 3, 4, a HUB 5, an image processing device 10, a printer 6, and a disc writer 7.

The cameras 2, 3, and 4 capture images of the subject H performing a golf swing or the pin P on which a golf ball is placed and the vicinity thereof from different angles. In the following description, the direction in which the golf ball flies is the front, the opposite side is the rear, and the side on which the golf ball is placed as viewed from the subject H is the front. In this embodiment, the cameras 2, 3, and 4 are high-speed digital cameras capable of shooting a high-speed moving image of 1200 [fps].
The camera 2 images the subject H from the front side of the subject H who is a person performing a golf swing, and the camera 3 images the subject H from behind the subject H. The camera 4 images the pin P and its vicinity from the front side of the subject H, thereby imaging the golf ball and the head of the golf club hitting the golf ball. The captured image displayed during image editing is a still image, but the cameras 2, 3, and 4 output the captured image as a moving image.
The cameras 2, 3, and 4 have a buffer memory (not shown), and each camera stores a captured image in the buffer memory and later transfers and outputs it to the outside.

  The cameras 2, 3, and 4 can set the number of frames captured per second (frame per second, hereinafter referred to as fps) at the time of moving image capturing to 300 to 1200 [fps]. In this embodiment, the cameras 2 and 3 perform imaging at 300 [fps], and the camera 4 performs imaging at 1200 [fps].

  The HUB 5 is a line concentrator interposed between the three cameras 2, 3, 4 and the image processing apparatus 10, and connects the three cameras 2, 3, 4 and the image processing apparatus 10.

  The image processing apparatus 10 performs swing analysis. Swing analysis refers to storage of captured moving images by the three cameras 2, 3, and 4 and various processes based on the captured moving images. Details of the swing analysis will be described later.

The printer 6 is a device that performs printing on a print medium (for example, paper or the like) by printing processing performed by the image processing apparatus 10. The printer 6 is configured by a printing device such as a page printer.
The disc writer 7 is a device that writes digital data to a DVD medium under the control of the image processing apparatus 10.

Next, the configuration of the image processing apparatus 10 will be described. FIG. 2 is a block diagram illustrating a configuration of the image processing apparatus 10.
The image processing apparatus 10 includes a CPU 11, a RAM 12, a ROM 13, a storage device 14, an input device 15, a display device 16, interfaces 17, 18, and 19, and a bus 20 that interconnects these components, Have
In the present embodiment, the image printing apparatus 10 is configured by an information processing apparatus such as a personal computer or a desktop computer.

  The CPU 11 reads a program, data, and the like from the ROM 13 and the storage device 14 and executes them, and controls the operation of each unit of the image processing apparatus 10 and each component connected to the image processing apparatus 10. A program, data, and the like read by the CPU 11 and parameters temporarily generated in the processing of the CPU 11 are stored in the RAM 12.

The ROM 13 is, for example, a ROM chip or the like, and stores programs, data, and the like so as not to be rewritten.
The storage device 14 is, for example, a hard disk drive or a flash memory, and can store a program, data, and the like in a rewritable manner.

  The input device 15 is, for example, a keyboard or a mouse, and enables input processing to the image processing device 10 by an operator manually.

  The display device 16 is configured by, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, or the like, and displays contents according to the screen output processing of the image processing device 10.

  The interfaces 17, 18, and 19 are bus interfaces such as USB (Universal Serial Bus), for example, and enable connection between the image processing apparatus 10 and an external device. The cameras 2, 3, 4 are connected to the interface 17 via the HUB 5, the printer 6 is connected to the interface 18, and the discwriter 7 is connected to the interface 19.

  Next, the content of the swing analysis process will be described. FIG. 3 is a flowchart showing the contents of the swing analysis process.

  First, photographing is performed from three directions by the three cameras 2, 3, 4 (step S1). Shooting is a process in which the image input apparatus 10 operates the three cameras 2, 3, and 4 to image the subject H by a predetermined input operation by an operator. The moving images captured by the three cameras 2, 3, and 4 are transferred to the image processing device 10, and the image processing device 10 stores the transferred captured moving images in the storage device 14.

  Next, the image processing apparatus 10 generates a still image (step S2). Still image generation is processing for generating a still image from each captured moving image stored in the storage device 14. The generated still image is stored in the storage device 14.

  Next, the image processing apparatus 10 performs swing target extraction based on the input contents of the operator (step S3). Swing target extraction is a process of determining a range (swing target) of an image captured during a golf swing for an image captured by the camera 2 among images divided into still images. This is a process that is performed because a captured image that is not in a golf swing may be included during moving image capturing by the cameras 2, 3, and 4. Examples of the captured image that is not during the golf swing include a captured image in which the subject H is performing an operation of adjusting the posture, and a subject H is performing other operations (for example, smoking a cigarette). . Swing target extraction is not performed on images captured by the cameras 3 and 4, but the swing target is determined by swing image synchronization described later.

  Next, the image processing apparatus 10 performs swing image synchronization based on the input contents of the operator (step S4). Swing image synchronization is processing for synchronizing still images from three directions, and associates still images captured at the same timing with still images captured by the cameras 2, 3, and 4.

  Next, the image processing apparatus 10 performs a movement state plot based on the input contents of the operator (steps S5 to S7). The movement state plot is a process of storing, as coordinate data, the position of the head of the golf club in each still image picked up based on a predetermined thinning number for each of the images from the three directions. Among the still images after the impact position, the position of the golf ball is stored as coordinate data for the still image in which the golf ball is reflected. The golf club head position and the golf ball position are input manually by the operator. The movement state plot is individually performed for each of the still images captured by the cameras 2, 3, and 4.

  Next, the image processing apparatus 10 performs swing speed analysis (step S8). The swing speed analysis is a calculation process of the moving speed of the head of the golf club during the golf swing. The swing speed analysis is performed based on the golf club head position input by the moving state plot.

  Next, the image processing apparatus 10 performs swing form printing (step S9). Swing form printing is a process of picking up and printing an image during a golf swing based on a predetermined rule.

In parallel with the operation state plot, swing analysis, and swing form printing, or in a separate process, the image processing apparatus 10 can create a synchronized moving image (step S10). Synchronized video creation is a combination of still images for which the swing image synchronization has been completed, converted into a single video, and the video captured by the three cameras 2, 3, and 4 is played back synchronously by playing back that single video file. This is a process for creating a video file. CPU11 reproduces | regenerates the moving image file produced by synchronous animation production, and synchronizes and reproduces the image of a several group based on the still image synchronized through the swing image synchronization of step S4.
FIG. 4 shows an example of a playback screen D1 of a moving image file created by creating a synchronized moving image. As shown in FIG. 4, on the playback screen D1 of the moving image file created by the synchronous moving image creation, three images (front moving image, side moving image and ball moving image) taken by the cameras 2, 3, 4 are displayed. Image) is reproduced in a synchronized state on one screen.

  Processing by the image processing apparatus 10 is performed by the CPU 11 executing the swing analysis application 21. The swing analysis application 21 is stored in the storage device 14, read out by the CPU 11, and interpreted.

FIG. 5 is a block diagram showing the contents stored in the storage device 14. The storage device 14 stores at least the swing analysis application 21. The swing analysis application includes various program components such as a shooting instruction program 21a, a swing target extraction program 21b, a swing image synchronization program 21c, a movement state plot program 21d, a swing speed analysis program 21e, a swing form printing program 21f, and a synchronized video creation program 21g. including.
The processing content of the shooting in step S1 is included in the imaging support program 21a. The processing contents of the still image generation in step S2 and the swing target extraction in step S3 are included in the swing target extraction program. The processing content of the swing image synchronization in step S4 is included in the swing image synchronization program. The processing contents of the movement state plot in steps S5 to S7 are included in the movement state plot program 21d. The processing content of the swing speed analysis in step S8 is included in the swing speed analysis program 21e. The processing content of the swing form printing in step S9 is included in the swing form printing program 21f. The processing content of the synchronized moving image creation in step S10 is included in the synchronized moving image creating program 21g.

  Although not shown, the storage device 14 includes three moving image files (file FR, file BK, and file BL), a plurality of still images to be generated, a reference position information table to be described later, first coordinate data, and second coordinate data. And a storage area for storing the fps of each moving image.

  Here, data (file) to be processed in each process performed by the image processing apparatus 10, data (file) to be processed, or a medium such as a printed material will be described.

  Three moving image files are stored in the storage device 14 by photographing in step S1. In the following description, a movie file captured by the camera 2 is referred to as a file FR, a movie file captured by the camera 3 as a file BK, and a movie file captured by the camera 4 as a file BL.

In step S2, a still image is generated from each moving image file. That is, three groups of still images based on each moving image are generated, and each group includes a plurality of still images generated from the same moving image. At this time, each still image file is given a file name with a number added to the end of the name of the original moving image file. The number is a serial number with an initial value of 1, and 1 is assigned to the first frame of each moving image, and the number of frames that continue thereafter increases sequentially. Taking a still image generated from the file FR as an example, a file name with a 7-digit serial number appended to the end of the original movie file name, such as “FR0000001”, “FR0000002”, etc., is given. The
The same applies to still images generated based on the file BK and the file BL. In the following description, the number part following the end of the file name of the original moving image file assigned to each still image is described as “still image number”, and the original moving image is “FR”. Is described as “front still image”, the original video of “BK” is described as “rear still image”, and the original video of “BL” is referred to as “ball still image”. It describes.

The number of still images generated depends on the fps of each moving image. For example, when a moving image of 300 [fps] is used as a still image, 300 [still images] corresponding to a captured moving image for one second are generated. That is, one still image corresponds to one image frame in the original moving image.
The image processing apparatus 10 manages and stores the fps of each camera as a variable. Specifically, the fps of the camera 2 is set as the value of the variable FRfps, the fps of the camera 3 is set as the value of the variable BKfps, and the fps of the camera 4 is stored in the storage device 14 as the value of the variable BLfps. The CPU 11 generates a number of still images corresponding to the FRfps value from the file FR, generates a number of still images corresponding to the BKfps value from the file BK, and generates a number of still images corresponding to the BLfps value to the file BL. Generate from.

The reference position information table information is generated and updated by the swing target extraction in step S3.
FIG. 6 shows an example of the reference position information table. 6A shows the initialized reference position information table, FIG. 6B shows the reference position information table after extraction of the swing object, and FIG. 6C shows the reference position information table after synchronization of the swing image. Indicates.

The reference position information table is a table that stores still image numbers corresponding to “start takeback”, “start swinging down”, “impact moment”, and “end follow-through” among the postures of the subject H during the golf swing. is there.
“Takeback start” is an operation scene in which an operation (takeback) of swinging up the golf club via the rear as an operation before the subject H hits the golf ball is started. “Start swinging down” is an operation scene in which the subject H starts to swing down the golf club after takeback because the subject H hits the golf ball. The “impact moment” is an operation scene in which the golf club head hits a golf ball. “End follow-through” is an operation scene where the subject H finishes swinging the golf club forward after hitting the golf ball.

  As shown in FIGS. 6A, 6 </ b> B, and 6 </ b> C, the reference position information table includes a “takeback start” position for each of the “front still image”, “rear still image”, and “ball still image”, The number of the still image corresponding to the “starting to swing down” position, the “impact moment” position, and the “follow-through end” position is stored. The reference position information table is stored in the storage device 14.

  In the swing target extraction in step S3, the still image corresponding to “takeback start”, “downward start”, “impact moment”, and “follow-through end” is manually designated for the “front still image”. At this time, a still image corresponding to “takeback start”, a still image corresponding to “follow-through end”, and a still image therebetween are subject to swing.

  As shown in FIG. 6C, the “takeback start” position and the “swing down start” for each of the “front still image”, “rear still image”, and “ball still image” are obtained by the swing image synchronization in step S4. The number of the still image corresponding to the position, the “impact instant” position, and the “follow-through end” position is stored. Accordingly, the swing target of the “rear still image” and the “ball still image” is also determined.

The golf club head position and the first coordinate data of the golf ball shown in each still image are stored by the movement state plots in steps S5 to S7.
FIG. 7 shows an example of coordinate data. FIG. 7A shows the coordinate data (first coordinate data) of the head position and the ball position, and FIG. 7B shows the coordinate data of the line (second coordinate data).

  As shown in FIG. 7A, the first coordinate data is table data having a file name and parameters indicating the X and Y coordinates of the head position and the ball position. The head position indicates the position of the head of the golf club, and the ball position indicates the position of the golf ball. The first coordinate data is represented by an X coordinate and a Y coordinate. The origin for specifying the head position and the ball position by the X coordinate and the Y coordinate is determined in advance at a predetermined position of the still image (for example, the coordinates of the lower left corner of the rectangular image data). The X coordinate and Y coordinate of the head position and the ball position of the first coordinate data are not stored for all still images, but for each still image picked up based on a predetermined thinning number as described above. Remembered.

  The first coordinate data shown in FIG. 7A is the first coordinate data of the “front still image”, but the “rear still image” and the “ball still image” also have the same coordinates for the first coordinate data. Data is stored.

Further, by displaying the coordinate position data of each still image on one screen, it is possible to draw the movement locus of the golf club head and the movement locus of the golf ball.
FIG. 8 to FIG. 10 show an example of a screen on which the golf club head and the movement trajectory of the golf ball are drawn during the golf swing. 8 shows the movement locus in the “front still image”, FIG. 9 shows the movement locus in the “rear still image”, and FIG. 10 shows the movement locus in the “ball still image”. In the following description and drawings, the reference numeral W is attached to the head of the golf club included in the display content of each still image.

The golf club head W and the first coordinate data of the golf ball in the “front still image” are stored by the movement state plot (front) in step S5. As shown in FIG. 8, the coordinates of the golf club head W (plot Q) and the coordinates of the golf ball (plot S) in the “front still image” can be drawn as shown in FIG. Thus, the movement trajectory R of the golf club head and the movement trajectory T of the golf ball can be obtained from the plot S. Similarly, the first coordinate data and movement trajectory of the “rear still image” are obtained by the movement state plot (backward) in step S6, and the first “ball still image” of the “ball still image” is obtained by the movement state plot (ball) in step S7. Coordinate data and movement trajectory are obtained.
8 to 10, the plot Q is indicated by a white circle, the movement locus R is indicated by a broken line, the plot S is indicated by a black circle, and the movement locus T is indicated by a solid line. The plot Q is attached to only one white circle on the movement locus R, but all the white circles on the movement locus R are the plot Q. Although the plot S is attached to only one black circle on the movement locus T, the black circles on the movement locus T are all plots S.

  A line L can be drawn on the screen on which the golf club head and the golf ball moving locus during the golf swing shown in FIGS. 8 and 9 are drawn. The line L is a straight line, and the second coordinate data shown in FIG. 7B is shown with the one end side of the line as the start position, the other end side as the end position, and the start position and end position as the X coordinate and Y coordinate for the still image. To remember.

  As shown in FIG. 7B, the second coordinate data is data in a table format that associates the line type with the X and Y coordinates of the start position and the end position. The line type is a parameter indicating whether the line L is a vertical line L or an oblique line L. The vertical line L is applied to the front still image as shown in FIG. L is applied to the rear still image as shown in FIG. As shown in FIG. 8, when a plurality of lines L are applied to one type of still image, the coordinates of the start position and the end position are individually stored in the second coordinate data.

  The X coordinate and Y coordinate of each coordinate data shown in FIGS. 7 (a) and 7 (b) describe parameters represented by a single letter of X or Y and a three-digit numerical value, respectively. Stores values indicating X and Y coordinates.

  The moving speed of the golf club head is obtained by the swing speed analysis in step S8. The moving speed of the golf club head obtained by the swing speed analysis is an average of the moving speed of the golf club head throughout the golf swing.

  By the swing form printing in step S9, a still image picked up based on a predetermined rule can be printed. Printing can be performed individually for each of the “front still image”, “rear still image”, and “ball still image”.

  By creating a synchronized moving image in step S10, one moving image file is generated in which the captured moving images from the three cameras 2, 3, and 4 are reproduced in synchronization. One moving image file obtained by the creation of the synchronized moving image can be used for external output such as recording on a DVD medium as data of a DVD video disk using the disc writer 7 or for other utilization.

Next, details of the swing target extraction will be described.
At the start of swing target extraction, the CPU 11 of the image processing apparatus 10 generates a reference position information table in an initialized state. As shown in FIG. 6A, the reference position information table in the initialization state includes all “takeback start” and “downward start” of the “front still image”, “rear still image”, and “ball still image”. The smallest number (0000001) is stored for the still image numbers of “impact moment” and “follow-through end”.

Then, the CPU 11 causes the display device 16 to display a swing target extraction screen as a “first operation screen”.
FIG. 11 shows an example of the display contents of the swing target extraction screen and its transition. FIG. 11A shows the initial display contents, FIG. 11B shows the time of selecting a “takeback start” still image, and FIG. 11C shows “takeback start”, “down swing start”, and “impact moment”. ”And“ Follow-through end ”after still image selection.
As shown in FIGS. 11A to 11C, the swing target extraction screen displays four still images and an image feed gauge 30. FIG. The four still images are displayed side by side in the horizontal direction (one line), and correspond to still images of “takeback start”, “downward start”, “impact moment”, and “follow-through end” in order from the left side. The image feed gauge 30 is an interface for changing the displayed still image.

  The still image displayed on the swing target extraction screen is a “front still image”, and the number of the still image corresponds to the reference position information table. That is, when the reference position information table is in the initialized state, the number of still images displayed on the swing target extraction screen is “0000001” for all four images. When the swing target extraction screen display process is performed, the reference position information table is in an initialized state.

Thereafter, the operator performs an operation of selecting another still image for each of the four still images. Specifically, the operator clicks one of four still images. The CPU 11 performs a process of focusing on the clicked still image. In FIG. 11B, the focused still image is shown by surrounding it with a broken line.
Thereafter, the operator operates the image feed gauge 30 to change the focused still image. The operation of the image feed gauge 30 can be performed by either the operation of the arrow buttons 32 and 32 or the click operation on the blank portion 33 of the gauge.

  When the operator clicks the arrow buttons 32 and 32, the display content of the focused still image is updated to a still image having a still image number one larger or one smaller than the still image number displayed so far. The At this time, when the arrow button 32 positioned at the right end of the image feed gauge 30 is clicked, the still image number is updated to a larger still image, and when the arrow button 31 positioned at the left end of the image feed gauge 30 is clicked, The number is updated to a still image that is one smaller. If the number of the still image displayed so far is already the maximum, the display content of the still image is not updated even if the arrow button 31 at the right end is clicked. If the number of still images displayed so far is already the smallest, even if the leftmost arrow button 32 is clicked, the display content of the still image is not updated.

  When the operator clicks on the blank portion 33 of the gauge, the displayed content of the focused still image is set to a still image number that is larger by a predetermined skip amount or smaller than the previously displayed still image number. The still image is updated. At this time, when the blank portion 33 on the right side of the scroll gauge 34 of the image feed gauge 30 is clicked, the still image number is updated to a still image having a predetermined skip amount, and when the blank portion 33 on the left side of the scroll gauge 34 is clicked, the still image is clicked. Is updated to a still image with a predetermined skip amount smaller. When the blank portion 33 on the right side of the scroll gauge 34 is clicked, the difference in the still image number between the still image displayed on the scroll gauge 34 and the still image with the largest still image number becomes a predetermined skip amount. Otherwise, the display content of the still image that is in focus is not updated. When the left blank portion 33 of the scroll gauge 34 is clicked, the difference in the still image number between the still image displayed on the scroll gauge 34 and the still image with the smallest still image number becomes a predetermined skip amount. Otherwise, the display content of the still image that is in focus is not updated.

  The predetermined skip amount can be arbitrarily determined. In this embodiment, a variable (for example, variable SKIP) for determining the skip amount is stored and managed in the storage device 14, and the CPU 11 reads the value set in the variable and controls the skip amount.

  In skip target extraction, the operator selects a still image to be focused on and clicks on the image feed gauge 30 for the four still images, thereby performing “takeback start”, “swing down start”, and “impact moment”, respectively. ”And“ Follow-through end ”are selected.

  In response to a click operation on the arrow button of the image feed gauge 30 or the blank portion 33 of the gauge by the operator, the CPU 11 updates the display content of the still image focused on the swing target extraction screen as described above, and the reference position Of the still image numbers stored in the information table, the still image number corresponding to the still image whose display contents are updated is updated. For example, when the display of the still image “start takeback” is updated, the CPU 11 sets the still image number of “takeback position” of “front still image” in the reference information position table to the updated still image number. replace. With the same mechanism, in the skip target extraction, the still image numbers “starting to swing down”, “impact moment”, and “ending follow-through” of the “front still image” are updated.

  On the swing target extraction screen, a key for inputting an instruction to end the swing target extraction is also displayed (for example, a x button on the upper right of the screen). When the key for instructing to end the swing target extraction is clicked, the CPU 11 ends the display of the swing target extraction screen. At this time, the number of the still image corresponding to the still image of “takeback start”, “downward start”, “impact moment”, and “follow-through end” displayed on the swing target extraction screen immediately before the display ends is a reference. In the position information table, “take back position”, “start swinging down”, “impact moment”, and “follow-through end” are stored respectively.

The CPU 11 performs a process of updating the still image numbers of the “rear still image” and the “ball still image” in the reference position information table together with the process of ending the display of the swing target extraction screen.
Specifically, the CPU 11 assigns the numbers of the still images of “take back position”, “start to swing down”, “impact of impact”, and “end of follow-through” of the “rear still image” of the reference position information table to the “front still image”. , “Takeback position”, “starting to swing down”, “impact instant”, and “follow-through end” are set to the same number as the still image number.

Then, the CPU 11 determines the “take back position” of the “ball still image” based on the still image numbers of “take back position”, “start swinging down”, “impact moment”, and “follow-through end” of the “front still image”. , Processing to calculate the still image numbers of “starting to swing down”, “impact moment”, and “ending follow-through”. Calculation is based on the following formulas (1) to (4). In the following formulas, Y1 is the number of the still image of “Takeback position” of “Ball still image”, Y2 is the number of the still image of “Start swinging down” of “Ball still image”, Y3 is “Ball The number of the still image of “Impact instant” of “Still image”, and the number of the still image of “Follow-through end” of “Ball still image” of Y4. Furthermore, 70% of the largest still image number of the “ball still image” (either one of rounded down or rounded up) is X, and the still image number of the “take back position” of the “front still image” is X1, The number of the still image of “Starting to swing down” of “Front still image” is X2, the number of the still image of “Impact instant” of “Front still image” is X3, and “Follow-through end” of “Front still image” The number of the still image is X4.
Y1 = X- (X3-X1) × BLfps / FRfps (1)
Y2 = X- (X3-X2) × BLfps / FRfps (2)
Y3 = X (3)
Y4 = X + (X4-X3) × BLfps / FRfps (4)

  When obtaining the number of the “takeback start” still image number of the “ball still image”, as shown in Expression (1), the “impact moment” still image number of the “front still image” and the “takeback start” stillness The value obtained by multiplying the difference from the image number by the result of dividing BLfps by FRfps is subtracted from X. At this time, the difference between the still image number of “impact moment” in “front still image” and the still image number of “start takeback” is the difference between “impact moment” and “takeback start” in “front still image”. Corresponds to the number of still images in between. The result of dividing BLfps by FRfps corresponds to the ratio of the number of “ball still images” to the number of “front still images” captured during the same time period. In other words, the “takeback start” still image number of the “ball still image” obtained by the equation (1) is the number of still images between the “impact moment” and the “takeback start” of the “front still image”. And the ratio of the number of “ball still images” to the number of “front still images” taken during the same time. The number of the still image of “starting to swing down” of the “ball still image” obtained by the equation (2) and the number of the still image of “follow-through end” of the “ball still image” obtained by the equation (4) are also respectively shown. The relationship between the number of the still image of “Starting to swing down” or “End of follow-through” and the number of “Front still images” taken during the same time with the number of the still image of “Impact instant” of “Front still image” This corresponds to the ratio of the number of “ball still images”.

  The CPU 11 uses the calculated Y1 to Y4 as the still image numbers of “takeback position”, “starting to swing down”, “impact moment”, and “followthrough end” of the “ball still image” in the reference position information table, respectively. Remember.

Next, details of swing image synchronization will be described.
In the swing image synchronization, the CPU 11 causes the display device 16 to display a swing image synchronization screen as a “second operation screen”.
FIG. 12 shows an example of the swing image synchronization screen. In FIG. 12, the golf ball is described with a reference symbol G.
The swing image synchronization screen includes display contents of a total of three lines: two lines in which four still images are arranged in the horizontal direction (one line) and one line in which six still images are arranged in the horizontal direction (one line). The still images displayed in each row are “front still image”, “rear still image”, and “ball still image” from the upper row. The row displaying “front still image” and “rear still image” displays still images corresponding to the moments of “start takeback”, “start swinging down”, “impact moment” and “end follow-through” from the left side. . That is, in the reference position information table, the still image corresponding to the still image number of “take back position”, “starting to swing down”, “impact moment”, and “follow-through end” of “front still image” and “rear still image” Is displayed in the first and second lines.

  The third line is obtained by adding -3, -2, -1, 0, +1, +2 to the still image number of "impact moment" of "ball still image" in the reference position information table in order from the left 6 A “ball still image” having the number of still images is displayed. That is, the fourth still image from the left corresponds to the still image number of “impact instant” of “ball still image” in the reference position information table.

The operator performs an operation of selecting a still image corresponding to the “impact moment” for the “rear still image” and “ball still image” displayed on the swing image synchronization screen.
As shown in FIG. 12, the third still image from the right of the second row (hereinafter referred to as the second row and the third), that is, the “impact moment” still image of the “rear still image” and the third row −4 Image feed gauges 40 and 50 are displayed below the still image of the “impact moment” of the second still image, that is, the “ball still image”. The operator selects the still image corresponding to the “impact moment” of the “rear still image” by operating the image feed gauge 40 displayed below the second row—the third still image, and the third row— A still image corresponding to the “impact moment” of the “ball still image” is selected by operating the image feed gauge 50 displayed below the fourth still image. The operation method of the image feed gauges 40 and 50 displayed on the swing image synchronization screen is the same as that of the image feed gauge 30 on the swing target extraction screen.

  When the operator operates the second-third image feed gauge 40 by the operator, the CPU 11 updates the display content of each still image displayed on the second line. Specifically, the CPU 11 displays the second and third still images by the same mechanism as the update of the display content on the swing target extraction screen in response to an operation on the second and third image feed gauges 40. Update the contents.

  The CPU 11 updates the second line and the third image feed gauge 40 by operating the second line and the third image feed gauge 40 together with the reference position according to the operation content of the second line and the third image feed gauge 40. Replace the numbers of the still images of “Takeback position”, “Start swinging down”, “Impact of impact” and “End of follow-through” of “Backward still image” in the information table. Specifically, the CPU 11 performs addition / subtraction according to the transition amount of the still image number in accordance with the operation of the image feed gauge 40 below the second row to the third still image in the “rear still image” of the reference position information table. "Take back position", "start swinging down", "impact moment" and "follow-through end" of the still image number, and update the reference position information table based on the calculation result. For example, when the arrow button at the right end of the second line to the third image feed gauge 40 is clicked, the CPU 11 has the still image number which is 1 greater than the second line to the third still image number. In addition to updating to a still image, 1 is added to each of the still image numbers of “take back position”, “starting to swing down”, “impact instant”, and “follow-through end” of “rear still image” in the reference position information table. .

  Further, the CPU 11 associates the display content of the other still image in the second row with the number of the still image in the updated reference position information table. That is, the image feed is simultaneously performed on the four still images displayed on the second line by the operation of the second-third image feed gauge 40. For example, when the arrow button at the right end of the second line to the third image feed gauge 40 is clicked, the display contents of the four still images in the second line are respectively determined from the still image numbers displayed so far. Is also updated to a still image having a still image number one larger. That is, the CPU 11 determines the number of the still image included in the four still images displayed on the second line of the swing image synchronization screen, the “take back position” of the “rear still image” in the reference position information table, and “starts to swing down”. ”,“ Impact moment ”, and“ follow-through end ”still image numbers are controlled to be the same.

  When the operator operates the third-fourth image feed gauge 50, the CPU 11 displays the third-fourth still image by the same mechanism as the update of the four still images in the second row. Update the contents.

The CPU 11 updates the third line to the fourth still image, along with the “takeback position” of the “ball still image” in the reference position information table according to the operation content of the third to fourth image feed gauge 50, Calculate and update the number of still images of “Start swinging down”, “Impact moment” and “End follow-through”. The CPU 11 calculates the number of still images of “takeback position”, “starting to swing down”, “impact moment”, and “end of follow-through” by the following formulas (5) to (8). In Expressions (5) to (8), Z indicates the number of a still image included in the third row-fourth still image displayed on the swing image synchronization screen.
Y1 = Z- (X3-X1) × BLfps / FRfps (5)
Y2 = Z- (X3-X2) × BLfps / FRfps (6)
Y3 = Z (7)
Y4 = Z + (X4-X3) × BLfps / FRfps (8)

  That is, the CPU 11 sets the still image number displayed in the third line to the fourth line as the still image number corresponding to the “impact moment” of the “ball still image”, and the “ball still image” “ The numbers of still images of “takeback position”, “starting to swing down” and “ending follow-through” are calculated. The calculation results of the still image numbers of “take back position”, “starting to swing down”, and “end of follow-through” of the “ball still image” according to the equations (5), (6), and (8) are the above-described equations (1) ) To (4), the “ball stillness” with respect to the number of still images between the “impact moment” of the “front still image” and “start takeback” and the number of “front still images” taken during the same time period. This corresponds to the ratio of the number of “images”.

  Further, the CPU 11 updates the display content of another still image in the third row. At this time, the six still images in the third row are added with -3, -2, -1, 0, +1, +2 with reference to the still image number of the third row-the fourth still image. Control is performed so that “ball still images” having still image numbers are arranged in order from the left. That is, the numbers of the other still images in the third row are also changed by an amount corresponding to the transition amount of the numbers in the third row to the fourth still image.

  As described above, when the operator performs an operation on the swing image synchronization screen so that the second line-third “rear still image” becomes a still image obtained by capturing a still image of “impact moment”, four images in the second line From the left, the still images are still images corresponding to “takeback position”, “starting to swing down”, “impact moment”, and “follow-through end” moments of “backward still image”. That is, the four still images in the first row and the four still images in the second row are in a synchronized relationship. In the reference position information table, “take back position”, “starting to swing down”, “impact moment”, and “follow-through end” still image numbers are displayed on the second line of the swing image synchronization screen. By replacing the corresponding still image with the corresponding still image, the still image corresponding to the still image number of the “front still image” stored in the reference position information table and the still image of the “rear still image” are stored. The still image corresponding to the number is synchronized with the still image.

  Similarly, on the swing image synchronization screen, when the operator performs an operation so that the third-fourth “ball still image” becomes a still image obtained by capturing a still image of “impact moment”, the third-fourth row The still image is synchronized with the “impact instant” of the first-third still image, that is, the “front still image”. Then, the numbers of the still images of “take back position”, “starting to swing down”, “impact moment”, and “end of follow-through” of the “ball still image” in the reference position information table are calculated according to equations (5) to (8). The number of the still image corresponding to the number of each still image of the “front still image” and the number of each still image of the “ball still image” stored in the reference position information table by being updated to the number of the still image according to the result It is in a synchronized relationship with a still image corresponding to.

  The “front still image” and the “back still image” are synchronized, and the “front still image” and the “ball still image” are synchronized, so that the “rear still image” and the “ball still image” are also synchronized.

  On the swing image synchronization screen, it is possible to focus only on the second-third still image or the third-fourth still image, and the still image can be changed. It is not possible to focus on other still images individually and change the still images individually. In other words, the synchronization processing of the three still images is performed by synchronizing the “front still image”, the “rear still image”, and the “ball still image” with each other by associating the still image of “impact moment” with the “front still image” “ The difference between the number of the still image of “Impact moment” and the number of the still image of “Start takeback”, “Start swing down” and “End of follow-through”, and the fps of the video file FR that is the origin of the “Front still image” This is performed according to the ratio of the fps of the moving image file that is the basis of each still image and the number of the still image included in the “impact instant” still image in each still image.

  On the swing image synchronization screen, a key for instructing to end the swing image synchronization is also displayed (for example, an x button on the upper right of the screen). When the key for instructing to end the swing image synchronization is clicked, the CPU 11 ends the display of the swing image synchronization screen. At this time, the numbers of the four still images in the second row displayed on the swing image synchronization screen immediately before the display ends are “take back position” and “start to swing down” of “rear still image” in the reference position information table. , “Impact moment” and “follow-through end” are stored respectively, and the still image number calculated by the formulas (5) to (8) based on the number of the still image in the third line to the fourth line is the reference position information. Stored in “Takeback position”, “Start swinging down”, “Impact moment”, and “Follow-through end” of the “Ball still image” of the table.

Next, the flow of processing of the image processing apparatus 10 will be described using the flow of FIG. 13 to FIG.
First, the process flow of the image processing apparatus 10 in the swing target extraction will be described.
FIG. 13 and FIG. 14 are flowcharts showing the processing contents of the swing target extraction. FIG. 13 is a flowchart showing the processing contents from step S21 to step S31 among the processing contents of the swing target extraction. FIG. 14 is a flowchart showing the processing contents from step S32 to step S44 among the processing contents of the swing target extraction.

  The CPU 11 reads any one of the moving image files captured by the cameras 2, 3, and 4 from the storage device 14 (step S21), generates a still image from the moving image file (step S22), and generates the still image. Are stored in the storage device 14 with serial numbers attached to the names of the moving image files as individual files (step S23). In the process of step S2, the CPU 11 acquires fps of a moving image file that is a source for generating a still image, and generates still images for the number of seconds corresponding to the fps. And CPU11 memorize | stores the fps of the moving image file acquired in the process of step S2 in the memory | storage device 14 (step S24).

  After step S24, the CPU 11 determines whether still images are generated and fps are stored for all three moving images captured by the cameras 2, 3, 4 (step S25). When there is a moving image file in which a still image has not yet been generated among the three moving image files (step S25: NO), the CPU 11 designates a moving image file in which a still image has not been generated (step S26). Process. When still images are generated and fps are stored for all three moving image files (step S25: YES), the CPU 11 reads the “front still image” with the smallest still image number (step S27), and the reference position information. A table is generated in an initialized state and stored in the storage device 14 (step S28), and a swing target extraction screen is displayed on the display device 16 (step S29). On the swing target extraction screen displayed in step S29, four “front still images” read in step S27 are displayed side by side.

  Thereafter, the CPU waits until a click operation is performed on any one of the still images on the swing target extraction screen (step S30: NO). When a click operation is performed on any one of the still images (step S30: YES), the CPU 11 performs a display indicating that the image is focused (step S31). Then, the CPU 11 performs either a click operation on the image feed gauge 30 (Step S32, Step S37), a click operation on another still image (Step S39), or an input instruction (Step S40) to end the swing target extraction. Wait until.

  When either one of the arrow buttons 32 and 32 of the image feed gauge 30 is clicked (step S32: YES), the CPU 11 displays the still image displayed after the update from the still image number of the still image focused at that time. An image number is calculated (step S33), and it is determined whether or not a “front still image” having the still image number calculated in step S33 exists (step S34). If there is a “front still image” having the still image number calculated in step S33 (step S34: YES), the CPU 11 displays the display content of the focused still image in the still image calculated in step S33. The still image having the number is replaced and updated (step S35), and the still image number corresponding to the focused still image in the reference position information table is replaced with the still image number calculated in step S33. (Step S36). If there is no “front still image” having the still image number calculated in step S33 in step S34 (step S34: NO), the process returns to the determination in step S32.

  When either one of the arrow buttons 32 and 32 of the image feed gauge 30 is not clicked (step S32: NO) and the blank portion 33 of the image feed gauge 30 is clicked (step S37: YES), the CPU 11 pre- The value of the variable SKIP stored in the storage device 14, that is, a predetermined thinning amount is read (step S38). Thereafter, the CPU 11 performs the processing from step S33 onward based on the thinning amount obtained in step S38.

  If the blank portion 33 of the image feed gauge 30 is not clicked (step S37: NO), and if a still image different from the focused still image is clicked (step S39: YES), the process proceeds to step S31. Return to processing.

  At that time, if a still image different from the focused still image is not clicked (step S39: NO) and an input instruction to end the swing target extraction is issued (step S40: YES), the CPU 11 displays the display device 16. The display of the swing target extraction screen is terminated (step S41).

After step S41, the CPU 11 assigns the numbers of the still images of “take back position”, “start to swing down”, “impact moment”, and “follow-through end” of “rear still image” in the reference position information table to “front stationary image”. The reference position information table is updated so as to be the same as the still image numbers of “takeback position”, “starting to swing down”, “impact moment”, and “followthrough end” of “image” (step S42).
Then, based on the still image numbers of “take back position”, “starting to swing down”, “impact moment”, and “end of follow-through” of the “front still image”, the CPU 11 calculates “ The numbers of the still images of “Takeback position”, “Start swinging down”, “Impact of impact”, “End of follow-through” of “Ball still image” are calculated (step S43), and the calculation result of step S44 is stored in the reference position information table. Reflect and update (step S44). This completes the swing target extraction.

Next, a processing flow of the image processing apparatus 10 in swing image synchronization will be described.
15 and 16 are flowcharts showing the processing contents of the swing image synchronization. FIG. 15 is a flowchart showing the processing contents from step S51 to step S60 among the processing contents of the swing image synchronization. FIG. 16 is a flowchart showing the processing contents from step S61 to step S73 among the processing contents of the swing target extraction.
The CPU 11 reads the reference position information table (step S51). Next, the CPU 11 displays a swing image synchronization screen (step S52). In step S52, only the framework is displayed on the swing image synchronization screen, and the still images in the first to third rows are not yet displayed.

  After step S52, the CPU 11 reads still images having the still image numbers of “take back position”, “start to swing down”, “impact moment”, and “follow-through end” of the “front still image” read in step S51. The data are sequentially read and displayed (step S53). The process of step S53 is repeated until the display of the four “front still images” is completed (step S54: NO).

  When the display of the four “front still images” is completed (step S54: YES), the CPU 11 “take back position”, “start swinging down”, “impact moment”, “impact moment”, “ Still images having still image numbers of “follow-through end” are sequentially read and displayed (step S55). The process in step S55 is repeated until the display of the four “rear still images” is completed (step S56: NO).

  When the display of the four “rear still images” is completed (step S56: YES), the CPU 11 has the still image number “impact moment” of the “ball still image” read in step S51 and the front and rear Still images (still images having still image numbers of −3, −2, −1, ± 0, +1, +2 are read sequentially with reference to the still image number of “impact moment” of “ball still image”. (Step S57) The process of step S57 is repeated until the display of the six “ball still images” is completed (step S58: NO).

  After step S57, the CPU 11 performs a click operation on one of the second row and the third still image or the third row and the fourth still image, that is, the “impact moment” of the “back still image” or the “ball still image”. It waits until a click operation is performed on the still image corresponding to "" (step S59: NO). When a click operation is performed on either the second or third still image or the third or fourth still image (step S59: YES), the CPU 11 has focused on the clicked still image. Is displayed (step S60). Then, the CPU 11 performs a click operation (Step S61, Step S69) on one of the image feed gauges 40 and 50 displayed below the focused still image, a click operation (Step S71) on the other still image, or It waits until any input instruction (step S72) for ending the swing image synchronization is performed.

  When either one of the arrow buttons of the image feed gauges 40 and 50 is clicked (step S61: YES), the CPU 11 updates the number from the still image of the still image that is in focus at that time. The number of the still image to be displayed is calculated (step S62), and it is determined whether there is a “front still image” having the still image number calculated in step S62 (step S63). When there is a “front still image” having the still image number calculated in step S62 (step S63: YES), the CPU 11 displays the display content of the focused still image in the still image calculated in step S62. The still image having the number is replaced and updated (step S64). Then, the CPU 11 calculates the number of the still image of “impact moment” corresponding to the type of still image in focus (“backward still image” or “ball still image”) in the reference position information table in step S62. The reference position information is calculated by replacing the still image number with the updated still image number and calculating the still image number of the “takeback start”, “downward start” and “follow-through end” of the type of the still image that is in focus. The table is updated (step S65).

  After step S65, the CPU 11 determines whether or not the focused still image is a “rear still image” (step S66). In the case of “rear still image” (step S66: YES), “take back” of “rear still image” displayed on the second line of the swing image synchronization screen according to the updated still image number calculated in step S62. The display contents of the three still images corresponding to “Start”, “Start swinging down”, and “Follow-through end” are updated (step S67). In step S66, if the focused still image is a “ball still image” (step S66: NO), the third row of the swing image synchronization screen is displayed according to the updated still image number calculated in step S62. The display contents of the five still images other than the “impact moment” displayed on the screen are updated (step S68).

  In step S63, if there is no still image having the still image number calculated in step S62 (step S63: NO), the process returns to the determination in step S61.

  When either one of the arrow buttons of the image feed gauges 40 and 50 is not clicked (step S61: NO), and a blank portion of the image feed gauges 40 and 50 is clicked (step S69: YES). The CPU 11 reads the value of the variable SKIP stored in the storage device 14 in advance, that is, the predetermined thinning amount (step S70). Thereafter, the CPU 11 performs the processing after step S62 described above based on the thinning amount obtained in step S70.

  A blank portion of the image feed gauges 40 and 50 is not click-operated (step S69: NO), and a still image different from the still image of the “impact moment” that is focused at that time, that is, the still image of the other “impact moment” When the image is clicked (step S71: YES), the process returns to step S60.

  If the other “impact moment” still image is not clicked (step S71: NO) and an input instruction to end the swing target extraction is given (step S72: YES), the CPU 11 displays the swing target extraction screen on the display device 16. The display is terminated (step S73). This completes the swing image synchronization.

According to this embodiment, on the swing image synchronization screen, the still image of the “rear still image” and the “ball still image” captured at the same timing as the “impact moment” still image of the “front still image” The CPU 11 can manually select via the input to the gauge, and the CPU 11 sets the number of the still image of the “rear still image” and “ball still image” selected on the swing image synchronization screen of the still image of “impact moment”. The number is stored in the reference position information table in association with the still image number of “impact moment” of “front still image”. Thereby, it is possible to satisfactorily synchronize still images generated from a plurality of moving images taken from different angles by the plurality of cameras 2, 3, 4. This synchronization process does not require a configuration for the synchronization process by acquiring the sound, a dedicated algorithm for analyzing the image, and the like, and therefore the synchronization process can be performed at a low cost.
In addition, since it is a manual synchronization process, even if various conditions such as a camera angle change occur, it is possible to satisfactorily discriminate between still images captured at the same timing by the operator's judgment, An extremely flexible synchronization process can be realized.
Furthermore, the “impact moment” still image of the “front still image” can be selected on the swing target extraction screen. Thus, the operator can arbitrarily select a still image as a reference for synchronizing still images generated from the moving images.

  Further, the CPU 11 reproduces the moving image file created by creating the synchronized moving image, thereby reproducing the images of the plurality of groups in synchronization based on the still images synchronized through the swing image synchronization in step S4. As a result, it is possible to confirm a plurality of groups of golf swing images photographed from a plurality of photographing directions in a synchronized state, and the posture of the subject H during the golf swing, the movement locus of the golf club head position, the golf ball It is possible to satisfactorily grasp the movement trajectory and the like.

  Further, on the swing target extraction screen, it is possible to select “take back start”, “start swinging down”, and “follow-through end” still images of the “front still image”.

  Furthermore, by selecting the “takeback start” and “followthrough end” still images of the “front still image” on the swing target extraction screen, a plurality of still images captured during the golf swing can be extracted. . This makes it possible to remove other still images that are not subject to swing analysis from the processing target, greatly reducing the processing load, and allocating more machine power resources for the swing image synchronization processing. Therefore, a light synchronization process can be realized.

  Furthermore, the still image generated from each moving image is assigned a continuous number according to the time series, and the CPU 11 assigns the number of each still image corresponding to the “impact moment” of the still image generated from each moving image, and “ The difference between the still image number of “takeback start”, “downward start” and “follow-through end” with respect to the still image number of “impact moment” in the “front still image” and the video that is the source of each still image Based on the ratio of fps, the number of the still image of “takeback start”, “downward start” and “follow-through end” of “backward still image” and “ball still image” is obtained and stored in the reference position information table . That is, the CPU 11 performs a synchronization process between the “front still image” and the other two types of still images, and performs swing target extraction of the other two types of still images. As a result, as long as the swing target is extracted from one type of still image, a plurality of still images that are subject to swing analysis can be specified for other types of still images, and a plurality of other still images that are not subject to swing analysis. Can be removed from the processing target, so the processing load can be greatly reduced, and more machine power resources can be allocated for the swing image synchronization processing, thus realizing light synchronization processing. Can do.

When imaging the subject H during a golf swing, an image captured at a moment when the golf club head hits the golf ball (“impact moment”) is preferably obtained by imaging at a high fps setting of 300 [fps] or more. Can be obtained. That is, when the subject H during a golf swing is imaged at a high fps of 300 [fps] or more and the captured video is converted into a number of still images corresponding to fps, a still image obtained by capturing the “impact moment” It is included with extremely high probability.
If the subject H during a golf swing is imaged at less than 300 [fps], the probability that a “impact moment” still image cannot be obtained when the captured moving image is converted into a still image increases dramatically.

The description in the above embodiment shows an example of the present invention, and the present invention is not limited to this.
For example, the moving image handled in the present invention is not limited to a moving image obtained by shooting a golf swing, and any moving image that captures a moving locus of a moving object is applicable.
For example, baseball bat swing movement trajectory and ball movement trajectory, tennis racket swing movement trajectory and ball movement trajectory, table tennis racket swing movement trajectory and ball movement trajectory, etc. The present invention can be applied to an object that shoots or shoots a movement trajectory of an ice hockey racket swing and a movement trajectory of a pad (ball).

  The image processing apparatus 10 realizes each function of the image processing apparatus 10 by software processing in which the CPU 11 reads and executes software corresponding to the processing content, but a dedicated configuration may be provided for each function.

  The number of cameras is not limited to three, but may be two or four or more. A synchronized still image selection screen such as a swing image synchronization screen for synchronizing still images according to the number of moving images to be synchronized is displayed, and the correspondence information such as the reference position information table also has an information amount corresponding to the number of moving images. Expansion or reduction is performed.

  The predetermined timing that can be selected on the timing selection screen such as the swing target extraction screen is not limited to four, and an arbitrary number of timings can be provided. Corresponding information such as a reference position information table is also expanded or reduced in information amount according to the number of timings.

In addition, the numerical values, formulas, and the like described in the above-described embodiment are merely examples, and it is assumed that optimum conditions are set according to the number of moving images that are the source of the still image to be synchronized, fps, and other various conditions. .
For example, since the fps of the movie that is the source of the “front still image” and the “rear still image” in the above embodiment are the same, the “rear still image” based on the still image number of the “front still image” If the fps of the video that is the source of the two images are different, the fps ratio is set as in the relationship between the “front still image” and the “ball still image”. The still image number is calculated according to a formula that takes into account.
The reproduction of the synchronized image is not limited to the reproduction of the moving image file by creating the synchronized moving image. For example, a plurality of groups of still images may be synchronized and displayed continuously in parallel based on the number of still images associated as synchronized still images by swing image synchronization, or may be associated as synchronized still images. The moving images may be reproduced in parallel in a synchronized state based on the position of the image frame in the original moving image corresponding to the still image.

2, 3, 4 cameras 5 HUB
6 Printer 7 Discwriter 10 Image Processing Device 11 CPU
12 RAM
14 storage device 15 input device 16 display device 17, 18, 19 interface 20 bus H subject P pin

The present invention relates to an image processing apparatus , a control method, and a program.

In order to solve the above-described problem, an image processing apparatus according to the first aspect of the present invention includes an acquisition unit that acquires a plurality of moving images obtained by capturing a specific action in which the posture of a subject changes through a plurality of postures in a series of actions. Selecting a position of an image frame corresponding to the posture selected by the selection unit for each of a plurality of moving images acquired by the acquisition unit and a selection unit that selects one of a plurality of postures in the specific operation Based on the position information of the image frame specified by the specifying means and the specifying means, the timing for reproducing the frame corresponding to the posture selected by the selecting means is the same for the plurality of moving images. Synchronization means for synchronizing the reproduction timings of a plurality of moving images .

According to a second aspect of the present invention, the selecting unit further selects a plurality of postures in the specific action, and the specifying unit is configured to select the plurality of moving images acquired by the acquiring unit. The position of the image frame corresponding to each of the plurality of postures selected by the selection unit is specified, and the synchronization unit is configured to select a plurality of images selected by the selection unit based on the position information of the image frame specified by the specification unit. The timings for reproducing the plurality of moving images are synchronized so that the timings for reproducing the frames corresponding to the respective postures are the same for the plurality of moving images .

The invention according to claim 3 further includes an extracting unit that extracts a plurality of image frames from a start timing to an end timing of the specific operation for each of the plurality of moving images acquired by the acquiring unit, and the specifying The means is characterized in that an image frame corresponding to the posture selected by the selection means is specified from a plurality of image frames extracted by the extraction means .

According to a fourth aspect of the present invention, the plurality of moving images acquired by the acquiring unit are different from each other in the number of image frames captured during the specific operation period, and the extracting unit A different number of image frames are extracted from each moving image in accordance with a difference in the number of image frames photographed during the specific operation period.
According to a fifth aspect of the present invention, the specifying means uses the selecting means to determine the position of the image frame specified by the user for the first moving image of the plurality of moving images acquired by the acquiring means. The second moving image that is specified as the position of the image frame corresponding to the selected posture and is different from the first moving image is based on the position of the image frame specified by the user with respect to the first moving image. The position of the image frame corresponding to the posture selected by the selection means is automatically specified.
The invention according to claim 6 further includes that the plurality of moving images acquired by the acquiring unit differ in the number of image frames photographed per unit time, and the specifying unit includes the unit in each moving image. The posture selected by the selection means in the second moving image based on the difference in the number of image frames taken every time and the position of the image frame designated by the user with respect to the first moving image The position of the image frame corresponding to is automatically specified.
According to a seventh aspect of the present invention, the specifying means further includes the first moving image according to a ratio of the number of image frames taken per unit time in the first moving image and the second moving image. The position of the image frame of the second moving image corresponding to the position of the image frame designated by the user with respect to the moving image is calculated.
According to an eighth aspect of the present invention, the specifying unit further includes an image corresponding to each of a plurality of postures selected by the selecting unit with respect to a first moving image of the plurality of moving images acquired by the acquiring unit. After specifying the position of the frame, the position of the image frame corresponding to the first posture in the second moving image corresponding to the position of the image frame corresponding to the first posture in the first moving image. Is specified by the user, the second in the second moving image corresponding to the position of the image frame corresponding to the second posture different from the first posture in the first moving image. The image frame position corresponding to the posture is automatically specified.
The invention according to claim 9 is characterized in that the acquisition means acquires a plurality of moving images obtained by photographing the subject performing the specific operation from a plurality of different photographing directions.
The invention according to claim 10 further includes operation input means for selecting an image frame, and the specifying means has a first operation screen having an adjustment gauge for image frame feeding included in at least one moving image. The first designation means for designating an image frame via the operation input means based on the first operation screen, the image frame designated by the first designation means, and the image frame A second operation screen including an adjustment gauge for feeding an image frame included in another moving image to be displayed, and a second designation for designating an image frame via the operation input unit based on the second operation screen Means, an image frame specified by the first specifying means, and an image frame specified by the second specifying means in a posture selected by the selecting means. And identifies as an image frame of each of the moving image to be.

A control method according to an eleventh aspect of the present invention is a control method for controlling reproduction of a moving image by an image processing apparatus, wherein a specific motion in which the posture of a subject changes through a plurality of postures in a series of motions is photographed. An acquisition process for acquiring a plurality of moving images, a selection process for selecting one of a plurality of postures in the specific action, and a posture selected by the selection process for each of the plurality of moving images acquired by the acquisition process And a timing for reproducing a frame corresponding to the posture selected by the selection process based on the position information of the image frame specified by the specifying process and the position information of the image frame specified by the specifying process. And synchronization processing for synchronizing the reproduction timings of the plurality of moving images so that the images are the same.

According to a twelfth aspect of the present invention, there is provided a program for acquiring a plurality of moving images obtained by photographing a specific motion in which a subject's posture changes through a plurality of postures in a series of motions; Selecting means for selecting one of the postures; specifying means for specifying the position of an image frame corresponding to the posture selected by the selecting means for each of the plurality of moving images acquired by the acquiring means; Based on the position information of the image frame specified by the specifying unit, the playback of the plurality of moving images is performed so that the timing for playing back the frame corresponding to the posture selected by the selecting unit is the same for the plurality of moving images. It is characterized by functioning as synchronization means for synchronizing timing.

Claims (6)

Operation input means for selecting an image frame;
Obtaining a still image of each of the plurality of groups from a plurality of groups of asynchronous moving images obtained by photographing a subject to be photographed from a plurality of photographing directions; and at least one still image operation scene and an adjustment frame for image frame feeding; A first operation screen comprising: a reference operation scene displayed on the display means, and a reference operation scene selected via the operation input means based on the first operation screen; and an operation scene of another still image corresponding to the reference operation scene; A control means for displaying on the display means a second operation screen comprising an adjustment gauge for image frame feeding, and associating the reference motion scene and the one still image selected on the second operation screen as a synchronized still image; An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the control unit reproduces images of a plurality of groups in synchronization based on the synchronized still images. The first operation screen can select one still image for each of a plurality of operation scenes,
The image processing apparatus according to claim 1, wherein the reference motion scene is any one of a plurality of still images selected on the first operation screen.   An extraction unit configured to extract a plurality of still images from a still image obtained by capturing the most recent operation scene to a still image obtained by capturing a recent operation scene among the plurality of still images selected on the first operation screen; The image processing apparatus according to claim 3. The still images in each group are numbered consecutively according to the time series of imaging,
The control means includes the number of the one still image selected on the second operation screen, the number of the still image of the reference operation scene, and the past operation of the plurality of still images selected on the first operation screen. Based on the number of the still image capturing the scene and the number of the still image capturing the recent operation scene, the still image selected in the same group as the still image selected on the second operation screen is extracted by the extraction unit. The image processing apparatus according to claim 4, wherein a plurality of still images obtained by capturing the same operation scene as the plurality of still images are extracted. Computer
Operation input means for selecting an image frame;
Obtaining a still image of each of the plurality of groups from a plurality of groups of asynchronous moving images obtained by photographing a subject to be photographed from a plurality of photographing directions; and at least one still image operation scene and an adjustment frame for image frame feeding; A first operation screen comprising: a reference operation scene displayed on the display means, and a reference operation scene selected via the operation input means based on the first operation screen; and an operation scene of another still image corresponding to the reference operation scene; A control means for displaying on the display means a second operation screen comprising an adjustment gauge for image frame feeding, and associating the reference motion scene and the one still image selected on the second operation screen as a synchronized still image;
A program characterized by functioning as

Images