GB2361370A

GB2361370A - Analysing movement of objects

Info

Publication number: GB2361370A
Application number: GB0009212A
Authority: GB
Inventors: Allan Plaskett
Original assignee: Individual
Current assignee: Individual
Priority date: 2000-04-14
Filing date: 2000-04-14
Publication date: 2001-10-17
Also published as: GB0009212D0

Abstract

The movement of an object is captured using a television camera 2 and recorded as a television signal, the video part of the recorded television signal being stored as a set of data files. A user selects a set of frames of the video signal, and the relative position of the object is measured in a number of the frames. A polynomial equation, derived using the measured positions using control device 18, is used to interpolate the movement of the object up to the predetermined moment, and the movement plotted and superimposed onto the selected frames of the television signal for viewing simultaneously with those frames on monitor 16 or through video output 20. The polynomial equation may then be used to extrapolate the movement of the object, the extrapolated movement also being plotted and superimposed onto the selected frames. The method, and its associated system may have particular use in assisting cricket umpires in judging leg-before wicket (lbw) decisions.

Description

2361370 1 METHOD OR, AND SYSTEM FOR, ANALYSING EVENTS The present

invention relates to a method of, and a system for, analysing events. In particular, but not exclusively, the invention relates to a method of, and a system for, analysing the flight of the ball during a game of cricket.

The game of cricket is, of course, played by both men and women. For simplicity, in the following description the male gender is used throughout and is intended to include the female gender.

Slow motion television replays have for many years been used by commentators to analyse the decisions of cricket umpires, who have to make their decisions based on what they see while standing at the wicket. This has led to many disputes as to whether the umpires' decisions have been correct. However, the ability of the television companies to provide these action replays is of considerable commercial importance to them, as it greatly enhances the viewer's entertainment and interest in the cricket match.

Recently, television replays have also been used to help the umpires make their decisions, for example on appeals for run outs, the decision being made by a "third umpire" off the pitch, who views a video recording of either the broadcast television picture or the pictures from dedicated television cameras.

The ability of the umpires to make correct decisions is of considerable commercial as well as sporting importance, given the very large sums of income from sponsorship and prize money that may depend on the outcome of important first class games, and the investment made by television companies for the right to broadcast important matches.

One situation that can present difficulty for umpires is when there is an appeal for 1.b.w. (leg before wicket). Commentators frequently question the umpires' decisions in such cases but, even with slow motion action replays, it may be difficult for them tojudge whether the correct decision was reached.

As regards appeals for 1.b.w., a batsman cannot be given out 1.b.w. if the ball pitches outside leg stump, or would not have gone on to hit the stumps. The umpire must therefore watch carefully where the ball pitches and must also judge whether the ball would have hit the 2 stumps if it had not hit the batsman's pads. This requires the umpire to estimate the likely flight of the ball, based only on its path from the point of pitching to the moment of striking the pads. Even with slow motion action replays, it may be difficult to decide what would have happened.

It is an object of the present invention to provide method of, and a device for, analysing events that mitigates at least some of the aforementioned problems.

According to the present invention there is provided a method of analysing the movement of an object, in which the movement of the object is captured using a television camera and recorded as a television signal, the video part of the recorded television signal is stored as a set of data files representing in a graphical format selected frames of the video signal, the relative position of the object is measured in a predetermined number of the selected frames, a polynomial equation representing the apparent movement of the object relative to the frames up to a predetermined moment is derived from the measured positions and, using the polynomial equation, the apparent movement relative to the frames of the object up to the predetermined moment is interpolated, and the interpolated movement is plotted and superimposed onto the selected frames of the television for viewing simultaneously with those frames.

The method allows the path of a ball to be traced onto the screen, which makes it possible to see easily and accurately where the ball went and in which direction it was travelling. You can therefore see whether the ball pitched in line with the wicket and whether up to the moment of hitting the batsman's pads it was heading towards the stumps. The position of the ball only has to be measured in a few of the frames, its position in the intervening frames being calculated automatically by a process of interpolation.

Advantageously, the uninterrupted movement of the object subsequent to said predetermined moment is extrapolated using the polynomial equation and the extrapolated movement is plotted and superimposed onto selected frames of the television recorded subsequent to said predetermined moment for viewing simultaneously with those frames.

3 By extrapolating from the measured positions, the further uninterrupted movement of the ball can be extrapolated, making it possible to see whether the ball would have gone on to hit the stumps, had it not hit the batsman's pads.

Advantageously, the video part of the recorded signal is stored as an AV1 file.

Advantageously, the frames of the video signal are stored as a set of DIB files.

Advantageously, the selected frames of the video signal are presented in a graphical format display within a graphical user environment.

Advantageously, the graphical format display is converted into an A/V output signal.

According to a further aspect of the invention there is provided a system for analysing events, said system including a television camera, a recording device for recording a television signal of an event captured by the television camera, and a data processing device connected to the recording device for analysing the recorded television signal, the data processing device being arranged to store the video part of the recorded s ignal as a set of data flies representing in a graphical format selected frames of the video signal, and to present the selected frames of the video signal in a graphical format display, allowing the events recorded in the television signal to be analysed.

Advantageously, the video part of the recorded signal is stored as an AVI file.

Advantageously, the selected frames of the video signal are presented in the graphical format display within a graphical user environment.

Advantageously, the system includes means for converting the graphical format display into an A/V output signal.

Embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings, in which:

Fig. 1 is a block diagram of computer-based system for analysing events from a television signal; 4 Fig. 2 is a block diagram of a computer system (including hardware and software) that forms part of the system shown in Fig. 1; Fig. 3 is a flow diagram illustrating the steps of a process for analysing events using the system; Figs. 4a to 4e represent a sequence of selected video frames from a television broadcast, showing the successive positions of a ball as it travels towards the stumps, and Fig. 5 represents a video frame generated according to the method, showing a trace illustrating the interpolated path of the ball superimposed onto a still frame showing the positions of the stumps and the batsman.

Fig. 1 is a block diagram of computer-based system for analysing events at, for example, a cricket match. The system includes at least one television camera 2, although more cameras may also be provided. The television camera may form part of the television system used by a broadcaster for broadcasting coverage of the cricket match, or it may be a dedicated device that is used only within the analysis system.

The television camera 2 has an output 6 through which the video signal passes. This output is connected to a video recorder 10, which is preferably capable of super slow motion recording. The signal is recorded by the video recorder 10. A synchronised composite audiovisual (A/V) signal leaves the video recorder through an A/V output terminal 12.

The A/V output signal may be passed directly to the broadcaster for live transmission, or it 20 may be recorded by the video recorder 10 for retransmission later, for example in an action replay. The A/V output terminal 12 of the video recorder is connected to a data processing device 14, for example a personal computer (PC) having a video card. The video card allows audio-visual signals to be captured and stored as digital data in an audio-video interleaved (AVI) file. Numerous suitable video cards are commercially available, one such being the card sold under the brand ALL-IN-WONDER PRO by ATI. This card is capable of capturing a television video signal at a rate of up to 25 frames per second. The computer also includes a monitor 16 and a control device 18, for example a keyboard and/or a mouse, as well as the normal components of a personal computer, such as a central processor unit (CPU), random access memory (RAM), read only memory (ROM) and so on (not shown).

The computer 14 has an video output terminal 20, for providing a video output signal illustrating the result of the analysis performed by the computer. This output video signal may be passed to the broadcaster, for transmission to the public, or it may be transmitted for viewing by the "third umpire", to enable him to make a decision with regard to a contested appeal.

Fig. 2 is a block diagram of computer forming part of the system shown in Fig. 1. The computer 14 has a video capture card 24 that receives the composite audio/video signal through an input terminal 26. The video card 24 captures the video part of the television signal and outputs it as digital data in an audio-video interleaved (AVI) file. The video card 10 has an output terminal 30 for direct output of video and audio signals.

The computer includes a software AVI analysis program 32, which receives the captured AVI file and splits the file to extract the separate video frames making up the video clip as a sequence of device independent bitmap (DIB) files, DIB(1) to DIB(n). The AV1 analysis program 32 may for example be written as a VisuaIC++rm routine. Such a routine is available 15 from the Internet as freeware from Phade Software and is called AVI RIP.

The computer 14 also includes a software data analysis program 34 such as the spreadsheet program sold under the trade mark MICROSOFT EXCEL. Bespoke software 35 (including macros and other routines) is provided to control the data analysis program. This program receives the DIB files and displays the frames represented by those files. superimposed with a trace of the ball in a window of a graphical user interface display (GUI) 3) 6. The GUI display 36 including the video window is converted by the video card 24 into a video output signal, which exits through the video output terminal 20.

The process for analysing the flight of a ball using the system will now be described with reference to Fig. 3. The process has four main steps, these being:

1 Capture of the video signal and storing of the digital data representing that signal as an audio-video interleaved (AVI) file; 2. Analysis of the AVI file and separation of the video data into separate files, 3. Analysis of the movement of the ball, and 6 4. Graphical presentation of video signal as a sequence of selected frames, with the flight of the ball superimposed, to illustrate clearly the flight of the ball in relation to the other objects visible in the frames (e.g. the stumps, the batsman's legs etc.).

Each of these steps will now be described in more detail with reference to the flow diagram shown in Fig. 3. The video signal from the television camera is recorded on the video recorder. It is important that the camera does not move during the recording, so that stationary objects such as the stumps remain in the same position in each of the frames. To analyse the events that took place during the recording, the video recording of the event is replayed and 10 the A/V output signal from the recorder is passed to the computer through A/V input terminal 26. The video card allows the computer to display the television picture on the monitor, for example in a 320x240 bit window. The person operating the computer watches the picture and captures 40 the crucial part of the recording, which in a cricket match may be the period from when the bowler releases the ball to the moment when the ball hits the batsman's pads, plus five or six frames before and after those moments. The captured part of the AN recording can, if necessary, be edited down to the frames of greatest interest. This captured part of the AN signal is stored as a digital AVI file, using a full frame uncompressed format.

The AVI file is split 42 using a computer routine, for example a VisualC++ Fm routine, to extract the video data, which is saved 44 as an AVI (video only) file. The separate video frames making up the video clip are extracted 48 from the AVI (video only) file and saved as a sequence of device independent bitmap (DIB) files, DIBI-DIB, These files are passed to the data analysis program for graphical display in a GUI environment.

An operator views the video clip and using a pointing device such as a mouse 18 and a crosshair aiming symbol 50, marks precisely the position of the ball 52 on a number of selected frames. Normally, the position will be marked on five frames as shown in Figs. 4a-4e: at approximately the moment of release by the bowler 54 (Fig. 4a), midway through the first part of the ball's flight (Fig. 4b), when the ball pitches (Fig. 4c), midway through the first bounce (Fig. 4d) and at the point of striking the batsman's pads 56 (Fig. 4e). This provides three locations during the first part of the ball's flight (up to and including the moment of pitching) and three locations during the second part of the ball's flight (including and subsequent to the 7 moment of pitching). The X- and Y-coordinates for each of these positions are entered into a spreadsheet in the analysis programme.

The apparent path of the ball across the screen (not its true path through the air) during each uninterrupted part of its flight may be represented by a polynomial equation of the form y = a + bx + cx' where a, b and c are constants. The values of a, b and c for each part of the ball's flight are determined by entering the three sets of X- and Y-coordinates into the equations and solving simultaneously. This is carried out automatically in the computer, for example using matrix arithmetic, as follows:

For each part of the ball's flight, three sets of coordinates are taken:

xl:yl Xl:Y2 X3:Y3 Each of these sets of coordinates satisfies the equation y = a + bx + CX2. Therefore, three equations may be written, as follows:

Y1 = a + bxl + ex, 2 Y2 = a + bX2 + CX2 2 Y3 = a + bX3 + CX3 2 These three equations may be written in matrix notation as follows:

Y=AX Y1 a where Y Y2 A= b Y3 c Thus, Y1 Y2 - Y3 2 XI xl X2 X2 2 X3 X2 3 a b c and X = 2 xl xl X X 2 2 2 2 3 3, 8 The values of the constants a, b and c are obtained by solving the matrix equation: A=X-'Y This equation is solved twice to obtain two polynomial expressions representing the first and second parts of ball's flight (before and after the bounce). Then, by inserting values of the variable x into those expressions to obtain the corresponding values of the variable y, the intermediate positions of the ball can be interpolated. In practice, the number of values of the x coordinate entered into the equations is made equal to the number of frames selected from the video clip, so that the position of the ball in each of those frames can be interpolated.

After the x- and y-values have been calculated, they are plotted using the graph function of the spreadsheet program, and the resulting plot is superimposed on one or more frames of the video clip in the graphic user interface environment, producing a display of the type shown in Fig. 5. This shows the successive positions of the ball 52a - 52e and a trace 58 of the path of the ball. By running the video clip backwards and forwards and simultaneously drawing and deleting the plot of the ball's path, its flight can be accurately assessed. In particular, it is easy to see whether the ball pitched in line with the stumps and whether prior to hitting the batsman's pads, it was heading towards the stumps.

By alternately advancing the video one frame at a time and the graph one point at a time, a trace 58 of the ball's flight is made to appear behind the ball 52 as it progresses from the bowler's hand to the pitch and from there to the pad. Conversely, by reversing the video one frame at atime and the graph one point at a time, the trace 58 is made to disappear as the ball regresses to the bowler's hand.

The selected video frames and the superimposed trace of the ball are converted 60 into a combined graphical display within the graphical user interface (GUI) environment, for example one running under WINDOWS 95TM. This graphical display is converted 62 by the video card into a video output signal.

The probable flight of the ball, had it not hit the batsman's pads, can be extrapolated by inserting further values of the x-coordinate into the second polynomial expression and solving for the y-coordinate. The plot illustrating the path of the ball can then be extended to show its probable uninterrupted path. This makes it possible to assess whether the ball would have 9 gone on to hit the stumps. In order to assess this with accuracy, it is necessary to estimate how much further the ball had to travel before reaching the plane of the stumps, which can be done by observing how far down the pitch the batsman had advanced. This estimate can be used to decide how many more frames would have elapsed before the ball reached the stumps, thereby allowing the number of additional values the coordinate to be decided upon. this is obviously not an exact process but in most situations should provide a reasonable degree of accuracy.

Various modifications of the system are possible. For example, it is not essential to use a video recorder: the live signals from the camera can be fed directly into the computer and captured in real time as the ball is delivered. The graphical information can be presented in different formats: for example, the frames of the video signal may be shown consecutively. Various other modifications will be apparent to those skilled in the art. For example, the batsman may be electronically removed from the video image, allowing the position of the stumps to be seen. Markings may also be superimposed on the video image, for example to highlight the portion of the wicket lying directly between the two sets of stumps.

In order to improve the accuracy with which the ball's flight is traced, a higher order polynomial expression may be used to represent the apparent path of the ball across the screen. For example, the ball's path may be represented by a third order polynomial equation of the form y = a + bx + CX2 + dx' where a, b, c and d are constants. The values of a, b, c and d are determined by plotting four points along the path of the ball and entering the four sets of Xand Y-coordinates into the above equation then solving those equations simultaneously using matrix arithmetic, substantially as described above.

Claims

1. A method of analysing the movement of an object, in which the movement of the objectis captured using a television camera and recorded as a television signal, the video part of the recorded television signal is stored as a set of data files representing in a graphical format selected frames of the video signal, the relative position of the object is measured in a predetermined number of the selected frames, a polynomial equation representing the apparent movement of the object relative to the frames up to a predetermined moment is derived from the measured positions and, using the polynomial equation, the apparent movement relative to the frames of the object up to the predetermined moment is interpolated, and the interpolated movement is plotted and superimposed onto the selected frames of the television signal for viewing simultaneously with those frames.

2. A method according to claim 1, in which the uninterrupted movement of the object subsequent to said predetermined moment is extrapolated using the polynomial equation and the extrapolated movement is plotted and superimposed onto selected frames of the television recorded subsequent to said predetermined moment for viewing simultaneously with those frames.

3. A method according to claim 1 or claim 2, in which the video part of the recorded signal is stored as an AVI file.

4. A method according to any one of the preceding claims, in which the frames of the video signal are stored as a set of DIB files.

5. A method according to any one of the preceding claims, in which the selected frames of the video signal are presented in the graphical format display within a graphical user environment.

6. A method according to any one of the preceding claims, in which the graphical format display is converted into an A/V output signal.

7. A system for analysing the movement of an object, said system including a television camera for capturing the movement of the object, a recording device for recording a television signal from the camera, and a data processing device connected to the recording device for analysing the recorded television signal, the data processing device being arranged to store the video part of the recorded signal as a set of data files representing in a graphical format selected frames of the video signal, and to present the selected frames of the video signal in a graphical format display whereby the relative position of the object in a predetermined number of the selected frames may be measured, said data processing device including means for deriving a polynomial equation representing the apparent movement of the object relative to the frames up to a predetermined moment from the measured positions and, using the polynomial equation, for interpolating the apparent movement relative to the frames of the object up to the predetermined moment, and means for plotting the interpolated movement and superimposing the plotted movement onto the selected frames of the television signal for viewing simultaneously with those frames.

8. A system according to claim 15, in which the video part of the recorded signal is stored as an AVI file.

9.

A system according to claim 15 or claim 16, in which the frames of the video signal are stored as a set of DIB files.

10. A system according to any one of claims 15 to 20, in which the selected frames of the video signal are presented in the graphical format display within a graphical user environment.

11. A system according to any one of claims 15 to 23, including means for filtering the audio part of the recorded signal.

A system according to any one of claims 15 to 27, including means for converting the graphical fonnat display into an A/V output signal.