US20250256156A1

US20250256156A1 - Predicting Whether a User Executed a Golf Shot

Info

Publication number: US20250256156A1
Application number: US19/050,741
Authority: US
Inventors: Stephen Obsitnik; Ryan Johnson; David Thomas LeDonne; Michael Hutchinson; Owais Murad Hussain Syed; Faizaan Ali; Salman Hussain Syed
Original assignee: Arccos Golf LLC
Current assignee: Arccos Golf LLC
Priority date: 2024-02-13
Filing date: 2025-02-11
Publication date: 2025-08-14

Abstract

A method for predicting whether a user executed a golf shot includes (i) receiving a selection of a machine learning model from a plurality of machine learning models, each of the plurality of machine learning models being trained to predict whether the golf shot occurred, (ii) receiving inputs relevant to the machine learning model that was selected; and (iii) predicting, using the machine learning model that was selected, whether the user executed the golf shot using the inputs relevant to the machine learning model that was selected.

Description

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority benefit to a provisional patent application entitled “Predicting Whether a User Executed a Golf Shot,” which was filed on Feb. 13, 2024, and assigned Ser. No. 63/552,753. The entire content of the foregoing provisional patent application is incorporated herein by reference.

BACKGROUND

In recent years, there have been efforts to monitor, track, and/or analyze a golfer's performance during a round of golf using one or more sensors associated with the golfer or the golf clubs. While this technology continues to advance and enhance golfer performance, there remains a need for additional improvements to aid golfers and enhance performance.

SUMMARY

In an embodiment, a method for predicting whether a user executed a golf shot includes (i) receiving a selection of a machine learning model from a plurality of machine learning models, each of the plurality of machine learning models being trained to predict whether the golf shot occurred, (ii) receiving inputs relevant to the machine learning model that was selected; and (iii) predicting, using the machine learning model that was selected, whether the user executed the golf shot using the inputs relevant to the machine learning model that was selected.
Prior to receiving the selection of the machine learning model, the method may entail determining whether the user was stationary long enough to potentially hit the golf shot. The determination may be made by comparing a time the user was stationary to a threshold, and the threshold may be based on information about the user and/or information about a golf course.
In response to determining that the user was stationary long enough to potentially hit the golf shot, the method may entail receiving the selection of the machine learning model. In response to determining that the user was stationary long enough to potentially hit the golf shot, the method may entail determining that no golf shot occurred. Prior to determining whether the user was stationary long enough to potentially hit the golf shot, the method may entail receiving course information and/or user positional data.
In an embodiment, selection of the machine learning model may be based at least in part on a position of the user relative to a hole on a golf course. Inputs relevant to the machine learning model that was selected may include user profile information for the user, and the machine learning model that was selected may be trained to take as input the user profile information for the user and to predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include positional and hole history information, and the machine learning model that was selected may be trained to take as input the positional and hole history information and to predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include a swing detection from a hip movement swing model, and the machine learning model that was selected may be trained to take as input the swing detection from the hip movement swing model and to predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include a previous shot outcome model output, and the machine learning model that was selected may be trained to take as input the previous shot outcome model output and to predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include audio hit probability information, and the machine learning model that was selected may be trained to take as input the audio hit probability information and to predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include head movement data, and the machine learning model that was selected may be trained to take as input the head movement data and predict whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include a swing detection from a watch swing model, and the machine learning model that was selected may be trained to take as input the swing detection from the watch swing model whether the user executed the golf shot.
In an embodiment, inputs relevant to the machine learning model that was selected may include user profile information for the user, positional and hole history information, a swing detection from a hip movement swing model, a previous shot outcome model output, audio hit probability information, head movement data, and/or a swing detection from a watch swing model, and the machine learning model that was selected may be trained to take as input the inputs relevant to the machine learning model.
In an embodiment, predictions by the method may include generating a prediction value and comparing the prediction value to a threshold. It may be determined that the user executed the golf shot responsive to the prediction value satisfying the threshold. It may be determined that the user did not execute the golf shot responsive to the prediction value failing to satisfy the threshold.
In an embodiment, a system includes (i) a memory that includes computer readable instructions; and (ii) at least one processor for executing the computer readable instructions, the computer readable instructions controlling the at least one processor to perform operations for predicting whether a user executed a golf shot. The operations may include (i) receiving a selection of a machine learning model from a plurality of machine learning models, each of the plurality of machine learning models being trained to predict whether the golf shot occurred,) (ii) receiving inputs relevant to the machine learning model that was selected, and (iii) predicting, using the machine learning model that was selected, whether the user executed the golf shot using the inputs relevant to the machine learning model that was selected.
In an embodiment, a non-transitory computer-readable medium includes instructions, wherein execution of the instructions by at least one processor causes the at least one processor to perform operations for predicting whether a user executed a golf shot. The operations may include (i) receiving a selection of a machine learning model from a plurality of machine learning models, each of the plurality of machine learning models being trained to predict whether the golf shot occurred, (ii) receiving inputs relevant to the machine learning model that was selected, and (iii) predicting, using the machine learning model that was selected, whether the user executed the golf shot using the inputs relevant to the machine learning model that was selected.
Any combination and/or permutation of embodiments is envisioned. Other embodiments, objects, and features will become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed as an illustration only and not as a definition of the limits of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an autonomous golf recommendation and analysis environment according to one or more embodiments described herein.

FIG. 2 depicts a flow chart of a method for predicting whether a user executed a golf shot according to one or more embodiments described herein.

FIGS. 3A-3G depict various information, data, models, and the like that can be used as inputs to a machine learning model for predicting whether a user executed a golf shot according to one or more embodiments described herein.

FIG. 4 depicts a block diagram of components of a machine learning training and inference system according to one or more embodiments described herein.

FIG. 5 depicts a block diagram of an electronic device that can be used to predict whether a user executed a golf shot according to one or more embodiments described herein.

FIG. 6 depicts a block diagram of an exemplary embodiment of the performance monitoring and/or tracking environment that can be implemented in accordance with the present disclosure.

FIG. 7 depicts a block diagram of an exemplary server in accordance with embodiments of the present disclosure.

DETAILED DESCRIPTION

Exemplary embodiments of the present disclosure are directed to various components of systems, methods, and non-transitory computer-readable media for automatic performance tracking of a round of golf, which includes predicting whether a user executed a golf shot using one or more trained machine learning models.
Recommendations and/or analysis can be autonomously provided on a hole-by-hole and/or shot-by-shot basis before, during, and/or after a round of golf. The recommendations and/or analysis can utilize a user's personal performance history, golf shot data collected by the sensors associated with a plurality of users, weather conditions, elevation, golf course features, equipment selections, and the like.
It is useful to know when golf shots occur in order to provide such recommendations and/or analysis. Often, a user manually indicates, such as on a user interface of a user computing device (e.g., a smartphone), that a golf shot occurred. In other cases, sensors embedded in or connected to a user's golf clubs can be used to determine when a golf shot occurred. It is desirable to provide an automated approach to indicating when a golf shot occurred without sensors associated with a user's golf club and without manual intervention.
One or more embodiments described herein provide for predicting when a golf shot occurred without sensors associated with a user's golf club and without manual intervention. For example, a trained machine learning model is selected from multiple trained machine learning models. The selection can be based on the user's location on the golf course, for example. Inputs relevant to the selected trained machine learning model are then received and/or collected and used to predict, using the selected trained machine learning model, whether a golf shot occurred. The one or more embodiments described herein provide advantages over existing approaches by automating the approach to indicating when a golf shot occurred without sensors associated with a user's golf club and without manual intervention.
FIG. 1 is an autonomous golf recommendation and analysis environment 10 that provides for determining whether a golf shot occurred using a golf shot prediction engine 14 in accordance with embodiments of the present disclosure. The environment 10 can be configured to provide autonomous, dynamic, and real-time recommendations and analysis to users, including predicting whether a golf shot occurred, and can include a user interface 12, the golf shot prediction engine 14, and at least one of course data 16, environmental data 18, user data 20, and/or equipment data 22. The environment 10 can be responsive to user input received via the user interface to generate one or more predictions, recommendations, and/or analytical data based on the data 16, 18, 20, and 22.
The course data 16 can be received via one or more course data sources 30, can be measured by one or more environmental sensors 32 at one or more golf courses, can be user generated, and/or can be measured or captured by other sensors 33 (e.g., imaging sensors, such as RGB and/or Infrared/thermal imaging sensors). As an example, for embodiments in which the sensors 33 are imaging sensors, the imaging sensors can capture images that are processed using machine vision to extract course data 16. The course data 16 can include, but is not limited to, golf course data, such as course features, course ground conditions, type of grass (fairway/rough/green), length of rough, green speed, number of tee boxes, size of tee boxes, number of hazards, size of hazards, type of green, size of greens, location of hazards relative to fairway and green, type of sand in bunker, depth of bunkers, elevation changes on course, pace of play, pin positions, geographic location, altitude of course, number of trees, types of trees, other vegetation, topography (desert/park land/seaside, etc.), accessibility (e.g., public vs. private vs. resort), presence of caddies, course rating (golf advisor/yelp/etc.), course architect, course architectural style, frequency of mowing, schedule of mowing, mowing pattern, frequency of green rolling, schedule of green rolling, frequency of watering, schedule of watering, watering pattern, heat map of course usage, total yardage, order of pars (i.e., 4, 4, 5, 4, 3 . . . ), number of each par, length of holes, length of out of bounds (OB), topography of greens, initiation fees/dues, metal spikes (e.g., allowed, not allowed), practice range present at course, practice green present at course, practice bunker present, practice chipping area, number of rounds played per year, beer cart present/frequency, and/or the like including combinations and/or multiples thereof. In some cases, the course data 16 can be user generated, such as pace of play by shot/hole, typical locations, and/or the like including combinations and/or multiples thereof.
The environmental data 18 can be measured by one or more environmental sensors 32 at or in geographic proximity to one or more golf courses (e.g., one or more local or regional weather monitoring stations), by one or more of the sensors 33 (e.g., imaging sensors, such RGB or IR imaging sensors), and/or can be received via one or more course data sources 34. The environmental data 18 can include, but is not limited to, wind speed, wind direction, temperature, humidity, barometric pressure, previous weather conditions (e.g., the previous day, week, month) including for example precipitation influencing ground conditions, luminosity, length of day, sun angle, time of sunset, time of sunrise, shadows, solar reflection, time of year, and/or the like including combinations and/or multiples thereof.
The user data 20 can be received via one or more user data sources 36 and/or can be measured by one or more sensors 38 carried or worn by users during a round of golf and/or one or more sensors affixed to or embedded in one or more golf clubs and/or one or more sensors remote to the user. According to one or more embodiments described herein, the golf shot prediction engine 14 predicts whether a golf shot occurred without using a sensor affixed to or embedded in any golf clubs. As one example, the sensors 38 can include one or more sensors disposed in a user's shoes, in a wristband or watch worn by the user, in a user's pocket, on/in a user's belt, in a glove worn by the user, in a user's glasses, in a shaft of a golf club, in a grip of a golf club, in a head of a golf club, and the like. The sensors 38 can include inertial sensors, such as accelerometers and gyroscopes; force sensors, such as pressure sensors, strain gauges, piezoelectric sensors; imaging sensors, such as CMOS imaging sensors, CCD imaging sensors, RGB imaging sensors, and/or Infrared imaging sensors; blood pressure sensors; blood sugar sensors; pulse oximeter sensors; heart rate sensors; temperature sensors; moisture sensors; light sensors; acoustic transducers, such as microphones; chemical sensors; and/or the like including combinations and/or multiples thereof, including any suitable type of sensor. As another example, one or more sensors remote to the user can collect the user data 20. Such one or more sensors can be sensors positioned around a golf course, aerial sensors associated with an unmanned aerial vehicle, and/or the like including combinations and/or multiples thereof.
The user data 20 can include, but is not limited to golfer data, such as age, gender, weight, height, golf handicap, strokes gained by facet, years golfing, frequency with which a user plays a round of golf, nationality, race, religion, sexual orientation, hydration level, fitness level, resting heart rate, blood pressure, glucose levels, medications, stress level, flexibility, percentage body fat, hip rotation, body type, apparel, swing plane, putting grip, golf grip, other sports played, lowest score, variance of scores, relationship status, number of children, average sleep per night, alcohol consumption, illicit drug usage, occupation, industry of employment, whether the user has played in amateur tournaments, whether the user has played in professional tournaments, chronic diseases, injuries, eye sight, calorie intake, handedness (right/left), IQ, club distances, trend lines (trending longer/more accurate golf shots, etc.), social data (e.g., quantity of Facebook friends, quantity of twitter followers, frequency of social media usage), ability to handle pressure as measured by past performance in “pressure” situations, data from golf simulators (such as Trackman and Foresight), who the user's golf instructor is, and/or the like including combinations and/or multiples thereof.
The user data 20 can include golf round data and golf shot data, such as fatigue level during round, playing partners, type of golf game played (scramble, best ball etc.), start time of round, type of round (tournament/pleasure), score, swing tempo, swing speed, swing plane, club face angles of a golf club at impact with a golf ball, club paths during golf swing, locations at which golf clubs makes impact with balls (heel, sweet spot, toe, etc.) for golf shots or a practice shots, spin rates of golf balls after the golf balls are hit by golf clubs, spin axes and/or directions of rotation of golf balls after the golf balls are hit by golf clubs, type of club, type of shot (full/half), ball flight path (draw/fade/low/high, etc.), length of shot, length of intended shot, location of start of shot, location of end of shot, hole/course shot is on, type of stance, performance of previous shots, raw or processed sensor data during a golf shot (e.g., image data capture by one or more imaging sensor, gyroscope data, accelerometer data, magnetometer data, microphone data, GPS data, proximity sensor data, force sensor data) from sensors in or on wrist bands; watches; portable electronic devices; golf club shafts; golf club grips; golf club heads; articles of clothing such as belts, gloves, shoes, hats, glasses; and/or the like including combinations and/or multiples thereof.
The user data 20 can also include head movement and body movement of the user during a golf swing based on how the imaging sensor(s) move during a golf swing. As an example, for embodiments in which the imagining sensor(s) are embedded, secured, or affixed to smart glasses, a (smart) hat, or belt worn by the user, the images captured by the imaging sensor(s) can be used to capture the user's movement during a golf swing. In addition, or in the alternative, an accelerometer having a fixed position and orientation relative to the imaging sensor(s) can detect movement of the imaging sensor, which can be used to determine how a user moves during a golf swing. This information can be input to the environment 10, which can process the information to analyze a golf swing.
The user data 20 can also include historical player performance data, such as percent of Eagles, percent of Birdies, percent of Pars, percent of Bogeys, percent of Double Bogeys, percent of greater than Double Bogeys, drive distances, longest drives, percent of fairways hit, percent of left of fairway, percent of right of fairway, greens in regulation, distance to pin on greens hit in regulation distance to pin on all approaches, percent of greens missed short, percent of greens missed left, percent of greens missed right, percent of greens missed long, Chip & Down percentage, average distance to pin after chip, Sand & Down percentage, Sand Save percentage, average distance to pin after sand shot, putts per hole, putts per GIR, one putt percentage, two putt percentage, three putt percentage, strokes gained driving, strokes gained approach, strokes gained layup, strokes gained chipping, strokes gained sand, strokes gained putting, fairway bunker percentage, greenside bunker percentage, tee OB percentage, approach OB percentage, fairway water hazard percentage, green water hazard percentage, average putts segmented by approach distance, and/or the like including combinations and/or multiples thereof. According to one or more embodiments described herein, the historical player performance data can be specific to a particular user, can be relative to a standard user or benchmark, and/or can be for a group or population of users.
The equipment data 22 can be measured or captured by the one or more sensors 38 carried or worn by users during a round of golf and/or the one or more sensors 38 affixed to or embedded in one or more golf clubs and/or can be received via one or more user data sources 40. According to one or more embodiments described herein, the golf shot prediction engine 14 predicts whether a golf shot occurred without using a sensor affixed to or embedded in any golf clubs. The equipment data 22 can include, but is not limited to equipment attributes, such as brand of each club, brand of putter, brand of ball, brand of shoes, brand of glove, brand of hat, all apparel brands, shaft bend, shaft stiffness, shaft length per club, grip brand, grip size, club lie angle, club loft, driver/wood configuration (draw/slice biased, loft angle), GPS device used, driver head size, year of each club, number of shots take with each club, golf bag brand, golf bag type (carry/cart), golf bag weight, and/or the like including combinations and/or multiples thereof.
The equipment data 22 can also include historical per club performance data, such as, for example, percent of Eagles with club off of tee, percent of Birdies with club off of tee, percent of Pars with club off of tee, percent of Bogeys with club off of tee, percent of Double Bogeys with club off of tee, percent of Eagles with club on approach, percent of Birdies with club on approach, percent of Pars with club on approach, percent of Bogeys with club on approach, percent of Double Bogeys with club on approach, percent of greater than Double Bogeys with club on approach, drive distances, longest drive, percent of fairways hit, percent of left of fairway, percent of right of fairway, greens in regulation, distance to pin on greens hit in regulation, distance to pin on all approaches, percent of greens missed short, percent of greens missed left, percent of greens missed right, percent of greens missed long, Chip & Down percentage, average distance to pin after chip, Sand & Down percentage, Sand Save percentage, average distance to pin after sand shot, putts per hole after using club on approach, putts per GIR after using club on approach, one putt % after using club on approach, two putt percentage after using club on approach, three or more putt percentage after using club on approach, strokes gained driving with club, strokes gained approach with club, strokes gained layup with club, strokes gained chipping with club, strokes gained sand with club, strokes gained putting with club, fairway bunker percentage with club, greenside bunker percentage with club, tee OB percentage with club, approach OB percentage with club, fairway water hazard percentage with club, green water hazard percentage with club, total shots with club, last time club was used, longest distance achieved by the club, average distance for the club, smart distance for the club, variance for the club, percent of “bad” shots hit by the club, and/or the like including combinations and/or multiples thereof.
The user interface 12 can provide one or more graphical user interfaces and/or one or more application-program interfaces (APIs) that can be accessed and/or utilized by a user to interact with environment 10. The user interface 12 can receive text input, speech input, and/or image input. The user interface 12 can employ natural language processing, speech recognition, and/or machine vision techniques to process user input received via the user interface 12. In exemplary embodiments, the user can access the environment 10 via a user device configured to communicate with a remote device via a communications network. In some instance the user interface or portions thereof can be installed on the user device such that the environment 10 can be implemented in a distributed manner. In some instances, the user device can include an application, such as a web browser or specially configured application associated with the environment 10. The user device can interact with the user interface 12 to allow the user of the user device to receive dynamic, real-time golf recommendations and analyses before, during, and/or after a round of golf. The golf recommendations and analyses can be generated in response to data 16, 18, 20, and 22 that has previously been measured or captured and received and/or that is contemporaneously measured or captured and received and/or can be generated in response to receipt of user input. For example, in embodiments of the present disclosure, the user may interface with the environment via the user interface 12 to ask a question, such as “which club should I use for this shot?”, “what should my strategy be for playing this golf course?”, or “what can I do to improve my score the next time I play this golf course?”
The golf shot prediction engine 14 can receive requests from the user interface 12 and receive the data 16, 18, 20, and/or 22 to predict whether a golf shot was executed. Exemplary embodiments of the golf shot prediction engine 14 can include a machine learning engine 50 that can implement one or more trained machine learning models to generate a prediction of whether a golf shot was executed based on input data, such as the data 16, 18, 20, and/or 22. As one example, the golf shot prediction engine 14 can execute one or more of the trained machine learning models using the machine learning engine 50 to predict whether a user executed a golf shot. Predicting whether a golf shot was executed is useful for analyzing the user's round of golf, which in turn can be used to provide post-round analysis of a user's round of golf to generate one or more personalized recommendations or analyses identifying strategies and/or areas where the user can improve the user's golf game, to provide pre-round analysis for a golf course the user intends to play to generate one or more personalized recommendations or analyses identifying strategies for playing the golf course, to provide personalized hole-by-hole and/or shot-by-shot recommendations or analyses identifying strategies for playing a hole or shot, and/or the like including combinations and/or multiples thereof.
In exemplary embodiments, the trained machine learning models implemented by the machine learning engine 50 can be trained using one or more machine learning algorithms. Examples of such machine learning algorithms can include supervised learning algorithms, unsupervised learning algorithm, artificial neural network algorithms, artificial neural network algorithms, association rule learning algorithms, hierarchical clustering algorithms, cluster analysis algorithms, outlier detection algorithms, semi-supervised learning algorithms, reinforcement learning algorithms and/or deep learning algorithms. Examples of supervised learning algorithms can include, for example, AODE; Artificial neural network, such as Backpropagation, Autoencoders, Hopfield networks, Boltzmann machines, Restricted Boltzmann Machines, and/or Spiking neural networks; Bayesian statistics, such as Bayesian network and/or Bayesian knowledge base; Case-based reasoning; Gaussian process regression; Gene expression programming; Group method of data handling (GMDH); Inductive logic programming; Instance-based learning; Lazy learning; Learning Automata; Learning Vector Quantization; Logistic Model Tree; Minimum message length (decision trees, decision graphs, etc.), such as Nearest Neighbor algorithms and/or Analogical modeling; Probably approximately correct learning (PAC) learning; Ripple down rules, a knowledge acquisition methodology; Symbolic machine learning algorithms; Support vector machines; Random Forests; Ensembles of classifiers, such as Bootstrap aggregating (bagging) and/or Boosting (meta-algorithm); Ordinal classification; Information fuzzy networks (IFN); Conditional Random Field; ANOVA; Linear classifiers, such as Fisher's linear discriminant, Linear regression, Logistic regression, Multinomial logistic regression, Naive Bayes classifier, Perceptron, and/or Support vector machines; Quadratic classifiers; k-nearest neighbor; Boosting; Decision trees, such as C4.5, Random forests, ID3, CART, SLIQ, and/or SPRINT; Bayesian networks, such as Naive Bayes; and/or Hidden Markov models. Examples of unsupervised learning algorithms can include Expectation-maximization algorithm; Vector Quantization; Generative topographic map; and/or Information bottleneck method. Examples of artificial neural network can include Self-organizing maps. Examples of association rule learning algorithms can include Apriori algorithm; Eclat algorithm; and/or FP-growth algorithm. Examples of hierarchical clustering can include Single-linkage clustering and/or Conceptual clustering. Examples of cluster analysis can include K-means algorithm; Fuzzy clustering; DBSCAN; and/or OPTICS algorithm. Examples of outlier detection can include Local Outlier Factors. Examples of semi-supervised learning algorithms can include Generative models; Low-density separation; Graph-based methods; and/or Co-training. Examples of reinforcement learning algorithms can include Temporal difference learning; Q-learning; Learning Automata; and/or SARSA. Examples of deep learning algorithms can include Deep belief networks; Deep Boltzmann machines; Deep Convolutional neural networks; Deep Recurrent neural networks; and/or Hierarchical temporal memory.
In exemplary embodiments, the trained machine learning models can include machine visions algorithms and techniques to process image data captured by one or more imaging sensors. The machine vision algorithms and techniques can include, for example, Stitching/Registration, Filtering, Thresholding, Pixel counting, Segmentation, Inpainting, Edge detection, Color Analysis, Blob discovery and manipulation, Neural net processing, Pattern recognition, Optical character recognition, blurring, normalized lighting, greyscaling, OTSU, thresholding, erosion/dilation, convert correct hull, contour detection, blob/mass calculation normalization, and/or Gauging/Metrology to recognize and measure objects in a scene imaged by the imaging sensor(s), such as, for example, a type of golf club, conditions of a golf course, a distance to a green or hole on a green, obstacles or hazards on a golf course (e.g., sand traps, water hazards, etc.), and/or to detect activities and/or movement in an imaged scene, such as, for example, a golf swing, an impact of a golf club with a golf ball (i.e., a golf shot), a velocity of the golf ball, a velocity of a golf club, a trajectory of a golf ball, an angle of impact between a golf club and a golf ball, a face angle of a golf club at impact with a golf ball, a club path during golf swing, a location at which a golf ball makes impact with a golf club (heel, sweet spot, toe, etc.) for a golf shot or a practice shot, a spin rate of golf ball after it is hit by a golf club, a spin axis and/or direction of rotation of a golf ball after it is hit by a golf club, and the like. Using the image processing and/or machine vision techniques, the environment 100 can extract data from images that were previously captured and stored and/or can extract data contemporaneously from an imaged scene in real-time. By using both historic images and contemporaneously captured images, the environment 10 can provide recommendations and analyses that account for changes in conditions or characteristics and reflect the current conditions or characteristics of a golf course and/or a user's golf game.
According to one or more embodiments described herein, the golf shot prediction engine 14 can execute multiple of the trained machine learning models using the machine learning engine 50 in sequence and/or concurrently with one another to predict whether a golf shot was executed. As one example, the machine learning engine 50 can execute two or more trained machine learning models concurrently with one another such that each of the trained machine learning models being executed generates an output based on the data 16, 18, 20, and/22 and received user input. The golf shot prediction engine 14 can weight the outputs of the trained machine learning models such that the outputs of the trained machine learning models can be prioritized or ranked and may be combined to predict whether a golf shot was executed. According to one or more embodiments described herein, the golf shot prediction engine 14 can use a voting model in which the outputs of each of the trained machine learning models can count as a vote and the prediction with the most votes is chosen by the golf shot prediction engine 14. According to one or more embodiments described herein, the prediction of whether a golf shot was executed can be output to the user via the user interface 12. In some embodiments, the weighting employed by the golf shot prediction engine 14 can give different numbers of votes to different trained machine learning models. In some embodiments, the golf shot prediction engine 14 can dynamically adjust the weighting and/or voting model based on whether the prediction of whether a golf shot was executed was accurate, for example measured against a ground truth value received from the user via the user interface 12 indicating whether the predicted golf shot was an actual golf shot.
FIG. 2 depicts a flow chart of a method 200 for predicting whether a user executed a golf shot according to one or more embodiments described herein. The method 200 can be implemented using any suitable device, such as one or more of the computing device 502 of FIG. 5 , the server 700 of FIG. 7 , and/or the like including combinations and/or multiples thereof.
At decision block 202, the method 200 determines whether a user (e.g., golfer) was stationary long enough to potentially hit a golf shot. The method 200 can receive the course data 16 (as described herein) and golfer positional data 201. The golfer positional data 201 indicates a location of the user. For example, the golfer positional data 201 can indicate an approximate or exact location of the user relative to a golf course as determined from the course data 16. The golfer positional data 201 can be determined using GPS, for example, or another locating technique. Determining whether the user was stationary long enough to potentially hit a golf shot can be based on the location of the user (e.g., where on the golf course the user is located), information about the location (e.g., a type of the location, such as a tee box, fairway, hazard, rough, green, etc.), and an amount of time the user was at the location. For example, if the user is on an area of the golf course known to be a tee box, and the user is at the tee box for an amount of time greater than a first threshold (e.g., greater than one minute), it is determined that the user was stationary long enough to potentially hit a golf shot. As another example, if the user is on an area of the golf course known to be a green, and the user is on the green for an amount of time greater than a second threshold (e.g., greater than two minutes), it is determined that the user was stationary long enough to potentially hit a golf shot. It should be appreciated that various different temporal thresholds can be used and can vary depending on a user's location relative to a golf hole. According to one or more embodiments described herein, one or more threshold used at decision block 202 can be learned over time based on habits and tendencies of a particular user and/or group of users. If the user is not in a location for a threshold amount of time (e.g., the first threshold or the second threshold), or if the location is not indicative of a location in which a golf shot is typically executed (e.g., a refreshment station, a locker room facility, a pro shop, and/or the like including combinations and/or multiples thereof), it is determined at decision block 202 that no golf shot was executed, and the method 200 proceeds to block 214 (“no golf shot”).
If it is determined at decision block 202 that the user was stationary long enough to potentially hit a golf shot, the method 200 proceeds to block 204 where a trained machine learning model is selected from a plurality of trained machine learning models. Each of the plurality of trained machine learning models can be trained to predict whether a golf shot was executed but can vary depending, for example, on the user's location on a hole as determined by the course data 16 and/or the golfer positional data 201. For example, if the user is on a tee box, a trained machine learning model used to predict whether a golf shot was executed on a tee box can be selected at block 204. As another example, if the user is on the fairway and is far from the green (e.g., about 200 yards), a trained machine learning model used to predict whether a golf shot was executed using a fairway wood or long iron can be selected at block 204. As yet another example, if the user is on the fairway but is close to the green (e.g., within about 50 yards), a trained machine learning model used to predict whether a golf shot was executed using a short iron or wedge can be selected at block 204. As yet another example, if the user is on the green, a trained machine learning model used to predict whether a golf shot was executed using a putter can be selected at block 204. The various machine learning models can be trained in accordance with one or more of the techniques described herein, such as the training 402 shown in FIG. 4 , which is described in more detail herein.
According to one or more embodiments described herein, a machine learning model can perform the selection at block 204. That is, a higher-level machine learning model can select from the plurality of machine learning models, which are considered sub-models to the higher-level machine learning model. For example, the higher-level machine learning model can be trained to select which of the plurality of trained machine learning models (e.g., the sub-models) is selected at block 204, where each of the plurality of trained machine learning models (e.g., the sub-models) can be trained to predict whether a golf shot was executed as described herein.
The various trained machine learning models may differ in the types of input(s) used and/or in the way the prediction is made. For example, a user's swing characteristics are different for a full golf shot (e.g., a drive), a short golf shot (e.g., a chip or pitch), a putt, and/or the like including combinations and/or multiples thereof. As such, the various trained machine learning models may receive different inputs and/or analyze the inputs differently depending on the type of shot the user may make, which is often indicated by the user's position on a hole as determined by the course data 16 and/or the golfer positional data 201. Accordingly, at block 206, the method 200 receives inputs 207 depending on which trained machine learning model was selected. The inputs 207 can include, for example, one or more of the data 16, 18, 20, and/or 22 of FIG. 1 . Additional and/or alternative inputs 207 are now described with reference to FIGS. 3A-3G. In particular, FIGS. 3A-3G depict various information, data, models, and the like that can be used as inputs 207 to a machine learning model for predicting whether a user executed a golf shot according to one or more embodiments described herein.
One example of one or more of the inputs 207 includes user profile information for the user. In this example, the machine learning model that was selected was trained to take as input the user profile information for the user and to predict whether the user executed the golf shot. FIG. 3A depicts an example of user profile information 310. In this example, the user profile information 310 includes a handicap (HPC) 311 for the user, an age 312 of the user, and a gender 313 of the user. Other information can be included in the user profile information 310 according to one or more embodiments described herein.
Another example of one or more of the inputs 207 includes positional and hole history information. In this example, the machine learning model that was selected was trained to take as input the positional and hole history information and to predict whether the user executed the golf shot. FIG. 3B depicts an example of positional and hole history information 320. In this example, the positional and hole history information 320 includes a distance from a previous shot 321, a previous shot terrain 322, a number of strokes to get down 323, a strokes to get down (STGD) differential 324 (e.g., the expected number of shots to hole out from a given location), current location terrain 325, distance from previous shot 326, time stationary at hotspot 327, average stationary at hotspot 328, walking versus riding 329. Other information can be included in the positional on hole history information 320 according to one or more embodiments described herein.
Another example of one or more of the inputs 207 includes a putt swing probability 335 and/or a swing probability 336 generated by a hip movement swing detection 330. Hip movement of the user can be indicative of whether the user executed a golf shot. For example, a golf shot may have distinctive movement patterns in terms of hip movement that can be detected by a sensor associated with the user's hips, thus indicating a golf shot may have occurred. In this example, the machine learning model that was selected was trained to take as input the swing detection from a hip movement swing model and to predict whether the user executed the golf shot. The swing detection can be one or more of the putt swing probability 335 and/or the swing probability 336. A swing detection model 331 is used to generate the putt swing probability 335 (e.g., the probability that a putting golf swing occurred) and/or the swing probability 336 (e.g., the probability that a non-putting golf swing occurred). The swing detection model 331 uses accelerometer data 332, gyroscopic data 333, and/or magnetometer data 334 to determine the put swing probability and/or the swing probability 336. According to one or more embodiments described herein, a sample of the user's accelerometer data 332, gyroscopic data 333, and/or magnetometer data 334 is collected periodically and a window of those samples is created, which is fed into the swing detection model 331 to predict the swing probability (e.g., the putt swing probability 335 and/or the swing probability 336). For example, the user's accelerometer data 332, gyroscopic data 333, and/or magnetometer data 334 are sampled every 200 milliseconds, and a six second window of 30 samples is created using the sampled values, resulting in a tuple (accelerometer value, gyroscopic value, magnetometer value). The tuple is fed into the swing detection module 331 to predict the swing probability (e.g., the putt swing probability 335 and/or the swing probability 336). It should be appreciated that the frequency of collecting the data and/or the size (e.g., duration) of the window can vary in other embodiments. According to one or more embodiments described herein, the swing detection module 331 can predict the swing probability (e.g., the putt swing probability 335 and/or the swing probability 336) using less than all of the user's accelerometer data 332, gyroscopic data 333, and/or magnetometer data 334. For example, the swing detection module 331 can predict whether a swing occurred using the accelerometer data 332 and the gyroscopic data 333 without considering the magnetometer data 334. According to one or more embodiments described herein, one or more of the accelerometer data 332, the gyroscopic data 333, and/or the magnetometer data 334 are collected by respective accelerometer, gyroscope, and magnetometer sensors in a wearable computing device disposed on or near a waist of the user in order to detect movement of the user's hips. The putt swing probability 335 and/or the swing probability 336 is a value (e.g., a number between 0 and 1) indicative of the probability that a golf shot (e.g., a hit) was executed by a user. The swing detection model 331 may utilize other information as inputs according to one or more embodiments described herein. For example, the swing detection module 331 may consider the user's heading both relative to true north and relative to a center of a target (e.g., a green of a golf hole) captured periodically (e.g., every second). As another example, the swing detection module 331 may consider the user's speed captured periodically (e.g., every second).
Yet another example of one or more of the inputs includes a previous shot outcome model output 340. In this example, the machine learning model that was selected was trained to take as input the previous shot outcome model output and to predict whether the user executed the golf shot. The previous shot outcome model outputs information about a previous shot, such as a location where the previous golf shot occurred, a location (e.g., an approximate location, a predicted location, an exact location) where the golf ball ended, a type of previous golf shot (e.g., a drive, a chip, a putt), and/or the like including combinations and/or multiples thereof.
Another example of one or more of the inputs 207 is audio hit (e.g., golf shot) information, such as an audio hit probability 354. Audio can be used as an indication of whether a golf shot occurred. For example, a golf shot may have a distinctive audio profile that can be detected and used to determine that a golf shot occurred when the distinctive audio profile is detected. In this example, the machine learning model that was selected was trained to take as input the audio hit probability information and to predict whether the user executed the golf shot. Audio data 351 can be detected, such as by a microphone (e.g., the microphone 528 of FIG. 5 ). Audio processing 352 is then performed on the audio data 351, such as to reduce noise, isolate sounds typical of golf shots, and/or the like including combinations and/or multiples thereof. An audio machine learning (ML) hit detection model 353 can be trained to predict whether a golf shot occurred using the audio data 351 and/or an output of the audio preprocessing 352. The prediction is generated as audio hit probability 354. The audio hit probability 354 is a value (e.g., a number between 0 and 1) indicative of the probability that a golf shot (e.g., a hit) was executed by a user.
According to another example, one or more of the outputs 207 is head movement data 360. Head movement data of the user can be indicative of whether the user executed a golf shot. For example, a golf shot may have distinctive movement patterns in terms of head movement that can be detected by a camera or sensor associated with the user's head, thus indicating a golf shot may have occurred. In this example, the machine learning model that was selected was trained to take as input the head movement data and predict whether the user executed the golf shot.
Yet another example of one or more of the outputs 207 is a putt swing probability 375 and/or a swing probability 376 generated by a watch swing model 370. Wrist movement of the user can be indicative of whether the user executed a golf shot. For example, a golf shot may have distinctive movement patterns in terms of wrist movement that can be detected by a watch or other wearable computing device, thus indicating a golf shot may have occurred. In this example, the machine learning model that was selected was trained to take as input the swing detection from a watch swing model whether the user executed the golf shot. The putt swing probability 375 (e.g., the probability that a putting golf swing occurred) and/or the swing probability 376 (e.g., the probability that a non-putting golf swing occurred) can be used as one or more of the inputs 207. Similar to the swing detection module 331, a swing detection model 371 uses accelerometer data 372, gyroscopic data 373, and/or magnetometer data 374 to generate the putt swing probability 375 and/or the swing probability 376. According to one or more embodiments described herein, a sample of the user's accelerometer data 372, gyroscopic data 373, and/or magnetometer data 374 is collected periodically and a window of those samples is created, which is fed into the swing detection model 371 to predict the swing probability (e.g., the putt swing probability 375 and/or the swing probability 376). For example, the user's accelerometer data 372, gyroscopic data 373, and/or magnetometer data 374 are sampled every 200 milliseconds, and a six second window of 30 samples is created using the sampled values, resulting in a tuple (accelerometer value, gyroscopic value, magnetometer value). The tuple is fed into the swing detection module 371 to predict the swing probability (e.g., the putt swing probability 375 and/or the swing probability 376). It should be appreciated that the frequency of collecting the data and/or the size (e.g., duration) of the window can vary in other embodiments. According to one or more embodiments described herein, the swing detection module 371 can predict the swing probability (e.g., the putt swing probability 375 and/or the swing probability 376) using less than all of the user's accelerometer data 372, gyroscopic data 373, and/or magnetometer data 374. For example, the swing detection module 371 can predict whether a swing occurred using the accelerometer data 372 and the gyroscopic data 373 without considering the magnetometer data 374. The swing detection model 371 may utilize other information as inputs according to one or more embodiments described herein. For example, the swing detection module 371 may consider the user's heading both relative to true north and relative to a center of a target (e.g., a green of a golf hole) captured periodically (e.g., every second). As another example, the swing detection module 371 may consider the user's speed captured periodically (e.g., every second). According to one or more embodiments described herein, one or more of the accelerometer data 372, the gyroscopic data 373, and/or the magnetometer data 374 are collected by respective accelerometer, gyroscope, and magnetometer sensors in a smartwatch or similar wearable computing device disposed on or near a wrist of the user in order to detect movements of the user's wrists.
The inputs shown in FIGS. 3A-3G can be used individually, collectively, and/or in various suitable combinations as the inputs 207 to block 206 of the method 200 of FIG. 2 .
With continued reference to FIG. 2 , once the inputs 207 are received at block 206, the method 200 proceeds at block 208. At block 208, the method 200 predicts, using the selected trained machine learning model, whether a golf shot occurred (that is, whether a user executed a golf shot). The selected trained machine learning model uses the inputs 207 that are relevant to that model to make the prediction. FIG. 4 , which is described in more detail herein, shows how the prediction is made in the process inference 404. According to one or more embodiments described herein, the prediction (e.g., the prediction 424 of FIG. 4 ) generated by the selected machine learning model (block 204) using the inputs 207 is a numeric value (e.g., from 0 to 1) that indicates a likelihood of a golf shot having occurred. For example, a prediction of 0.9 may be a relatively high likelihood that a user executed a golf shot, while a prediction of 0.3 may be a relatively low likelihood that the user executed the golf shot. Other ranges and values can be used in other examples.
At decision block 210, the prediction from block 208 generated by the selected trained machine learning model is compared to a threshold. If the prediction satisfies the threshold (e.g., is greater than the threshold, is greater than or equal to the threshold), the method 200 proceeds to block 212 (“golf shot”) and indicates that a golf shot occurred. If, however, the prediction fails to satisfy the threshold (e.g., is less than the threshold, is less than or equal to the threshold), the method 200 proceeds to block 214 (“no golf shot”) and indicates that no golf shot occurred. The threshold can be any suitable value and can be set and/or adjusted by a user and/or automatically. For example, the threshold can be based on data (e.g., training data 412 of FIG. 4 ) used to train the machine learning models. The threshold can be adjusted over time in some embodiments based on false detections. For example, if the selected trained machine learning model generates a prediction that satisfies the threshold (thus, registering that a golf shot occurred), but no shot actually occurred, the user can flag the prediction as being a false detection, and the threshold can be adjusted to improve future predictions.
Additional processes also may be included, and it should be understood that the processes depicted in FIG. 2 represent illustrations, and that other processes may be added or existing processes may be removed, modified, or rearranged without departing from the scope of the present disclosure. It should also be understood that the processes depicted in FIG. 2 may be implemented as programmatic instructions stored on a non-transitory computer-readable storage medium that, when executed by a processor (e.g., the processor 502, the processor 702) of a computing system (e.g., the computing device 501, the server 700), cause the processor to perform the processes described herein
One or more embodiments described herein can utilize machine learning techniques to perform tasks, such as to predict whether a user executed a golf shot. More specifically, one or more embodiments described herein can incorporate and utilize rule-based decision making and artificial intelligence (AI) reasoning to accomplish the various operations described herein, namely to predict whether a user executed a golf shot. The phrase “machine learning” broadly describes a function of electronic systems that learn from data. A machine learning system, engine, or module can include a trainable machine learning algorithm that can be trained, such as in an external cloud environment, to learn functional relationships between inputs and outputs, and the resulting model (sometimes referred to as a “trained neural network,” “trained model,” and/or “trained machine learning model”) can be used to predict whether a user executed a golf shot, for example. In one or more embodiments, machine learning functionality can be implemented using an artificial neural network (ANN) having the capability to be trained to perform a function. In machine learning and cognitive science, ANNs are a family of statistical learning models inspired by the biological neural networks of animals, and in particular the brain. ANNs can be used to estimate or approximate systems and functions that depend on a large number of inputs. Convolutional neural networks (CNN) are a class of deep, feed-forward ANNs that are particularly useful at tasks such as, but not limited to analyzing visual imagery and natural language processing (NLP). Recurrent neural networks (RNN) are another class of deep, feed-forward ANNs and are particularly useful at tasks such as, but not limited to, unsegmented connected handwriting recognition and speech recognition. Other types of neural networks are also known and can be used in accordance with one or more embodiments described herein.
ANNs can be embodied as so-called “neuromorphic” systems of interconnected processor elements that act as simulated “neurons” and exchange “messages” between each other in the form of electronic signals. Similar to the so-called “plasticity” of synaptic neurotransmitter connections that carry messages between biological neurons, the connections in ANNs that carry electronic messages between simulated neurons are provided with numeric weights that correspond to the strength or weakness of a given connection. The weights can be adjusted and tuned based on experience, making ANNs adaptive to inputs and capable of learning. For example, an ANN for handwriting recognition is defined by a set of input neurons that can be activated by the pixels of an input image. After being weighted and transformed by a function determined by the network's designer, the activation of these input neurons are then passed to other downstream neurons, which are often referred to as “hidden” neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was input. It should be appreciated that these same techniques can be applied in the case of predicting whether a user executed a golf shot as described herein.
Systems for training and using a machine learning model are now described in more detail with reference to FIG. 4 . Particularly, FIG. 4 depicts a block diagram of components of a machine learning training and inference system 400 according to one or more embodiments described herein. The system 400 performs training 402 and inference 404. During training 402, a training engine 416 trains a model (e.g., the trained model 418) to perform a task, such as to to predict whether a user executed a golf shot. Inference 404 is the process of implementing the trained model 418 to perform the task, such as to predict whether a user executed a golf shot, in the context of a larger system (e.g., a system 426). All or a portion of the system 400 shown in FIG. 4 can be implemented, for example by all or a subset of the computing device 501 of FIG. 5 and/or the server 700 of FIG. 7 .
The training 402 begins with training data 412, which may be structured or unstructured data. According to one or more embodiments described herein, the training data 412 can include one or more of the data 16, 18, 20, 22, and/or the inputs 207 as shown in FIGS. 3A-3G. The training engine 416 receives the training data 412 and a model form 414. The model form 414 represents a base model that is untrained. The model form 414 can have preset weights and biases, which can be adjusted during training. It should be appreciated that the model form 414 can be selected from many different model forms depending on the task to be performed. For example, where the training 402 is to predict whether a user executed a golf shot, the model form 414 may be a model form of a neural network for making predictions. The training 402 can be supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, and/or the like, including combinations and/or multiples thereof. For example, supervised learning can be used to train a machine learning model to predict whether a user executed a golf shot. To do this, the training data 412 can include the data 16, 18, 20, 22, and/or the inputs 207 as shown in FIGS. 3A-3G and a ground truth value (e.g., “golf shot” or “no golf shot”). In this example, the training engine 416 takes as input the training data 412, makes a prediction as to whether a user executed a golf shot, and compares the prediction to the known label (e.g., “golf shot” or “no golf shot”). The training engine 416 then adjusts weights and/or biases of the model based on results of the comparison, such as by using backpropagation. The training 402 may be performed multiple times (referred to as “epochs”) until a suitable model is trained (e.g., the trained model 418).
Once trained, the trained model 418 can be used to perform inference 404 to perform a task, such as to predict whether a user executed a golf shot. The inference engine 420 applies the trained model 418 to new data 422 (e.g., real-world, non-training data). For example, if the trained model 418 is trained to predict whether a user executed a golf shot, the new data 422 can be one or more of the data 16, 18, 20, 22, and/or the inputs 207 as shown in FIGS. 3A-3G that was not part of the training data 412. In this way, the new data 422 (e.g., the data 16, 18, 20, 22, and/or the inputs 207 as shown in FIGS. 3A-3G) represents data to which the model 418 has not been exposed. The inference engine 420 makes a prediction 424 (e.g., whether a user executed a golf shot based on the new data 422) and passes the prediction 424 to the system 426 (e.g., the computing device 501 of FIG. 5 , the server 700 of FIG. 7 , and/or the like including combinations and/or multiples thereof). The system 426 can, based on the prediction 424, taken an action, perform an operation, perform an analysis, and/or the like, including combinations and/or multiples thereof. In some embodiments, the system 426 can add to and/or modify the new data 422 based on the prediction 424.
In accordance with one or more embodiments, the predictions 424 generated by the inference engine 420 are periodically monitored and verified to ensure that the inference engine 420 is operating as expected. Based on the verification, additional training 402 may occur using the trained model 418 as the starting point. The additional training 402 may include all or a subset of the original training data 412 and/or new training data 412. In accordance with one or more embodiments, the training 402 includes updating the trained model 418 to account for changes in expected input data.
FIG. 5 is a block diagram of an exemplary electronic device 500 that may be used to automatically track a performance of a user during a round of golf and/or to provide personalized recommendations or analysis to the user during the round of golf, which can include predicting whether a user executed a golf shot. The electronic device 500 can include a computing device 501 that includes one or more non-transitory computer-readable media for storing computer-executable instructions, code, or software for implementing the golf shot prediction engine 14, which can be included in a performance tracking and/or monitoring environment 555. An example of the performance tracking and/or monitoring environment 555 is described in U.S. patent application Ser. No. 17/572,015, filed Jan. 10, 2022, and entitled “Autonomous Tracking and Personalized Golf Recommendation and Analysis Environment,” the contents of which are incorporated by reference herein in their entirety.
The non-transitory computer-readable media may include, but are not limited to, one or more types of hardware memory, non-transitory tangible media (for example, one or more magnetic storage disks, one or more optical disks, one or more flash drives), and the like. For example, memory 506 included in the electronic device 500 may store computer-readable and computer-executable instructions or software for implementing exemplary embodiments of the environment 555, including the golf shot prediction engine 14. The computing device 500 also includes configurable and/or programmable processor, e.g., a processor 502 and associated core(s) 504, and optionally, one or more additional configurable and/or programmable processor(s) 502′ and associated core(s) 504′ (for example, in the case of computer systems having multiple processors/cores), for executing computer-readable and computer-executable instructions or software stored in the memory 506 and other programs for controlling system hardware. Processor 502 and processor(s) 502′ may each be a single core processor or multiple core (504 and 504′) processor.
Virtualization may be employed in the electronic device 500 so that infrastructure and resources in the electronic device 500 may be shared dynamically. A virtual machine 514 may be provided to handle a process running on multiple processors so that the process appears to be using only one computing resource rather than multiple computing resources. Multiple virtual machines may also be used with one processor.
Memory 506 may include a computer system memory or random access memory, such as DRAM, SRAM, EDO RAM, and the like. Memory 506 may include other types of memory as well, or combinations thereof.
A user may interact with the electronic device 500 through a visual display device 518, such as a touch screen, which may display one or more graphical user interfaces 519 render upon execution of the computer readable instructions, code, or software corresponding to the environment 555. The electronic device 500 may include other I/O devices for receiving input from a user, for example, a keyboard (virtual or physical) or any suitable multi-point touch interface 508, a pointing device 510 (e.g., a mouse or stylus), a microphone 528, and/or an image capturing device 529 (e.g., a camera or scanner). The computing device 500 may include other suitable conventional I/O peripherals. The image capturing device 529 can include an imaging sensor and can be used to capture images during a round of golf. As an example, the image capturing device 529 can capture images that can be processed by the processor(s) 502, 502′ and/or the environment 10 to recognize and measure objects in a scene imaged by the image capturing device 529, such as, for example, a type of golf club being used, conditions of a golf course, a distance to a green or hole on a green, obstacles or hazards on a golf course (e.g., sand traps, water hazards, etc.), whether a golf shot was executed, and/or to detect activities and/or movement in an imaged scene, such as, for example, a golf swing, different phases of a golf swing (e.g., back swing, forward swing, follow through), a tempo of a golf swing, an orientation of golf club during a golf swing, an impact of a golf club with a golf ball (i.e., a golf shot), a velocity of the golf ball, a velocity of a golf club, a trajectory of a golf ball, an angle of impact between a head or face of a golf club and a golf ball, a face angle of a golf club at impact with a golf ball, a club path during golf swing, a location at which a golf ball makes impact with a golf club (heel, sweet spot, toe, etc.) for a golf shot or a practice shot, a spin rate of golf ball after it is hit by a golf club, a spin axis and/or direction of rotation of a golf ball after it is hit by a golf club, and the like. The images captured by the imaging sensor can be used to track head movement and body movement of the user during a golf swing based on how the imaging sensor(s) move (e.g., how the field-of-view changes) during a golf swing. As an example, for embodiments in which the imagining sensor(s) are embedded, secured, or affixed to smart glasses, a (smart) hat, or belt worn by the user, the images captured by the imaging sensor(s) can be used to capture the user's movement during a golf swing. In addition, or in the alternative, an accelerometer having a fixed position and orientation relative to the imaging sensor(s) can detect movement of the imaging sensor, which can be used to determine how a user moves during a golf swing. Using the image processing and/or machine vision techniques, the electronic device 500 and/or the environment 10 can extract data contemporaneously from an imaged scene to automatically track a performance of a user during a round of golf and/or to provide personalized recommendations or analysis to the user during the round of golf. The machine vision algorithms and techniques can include, for example, Stitching/Registration, Filtering, Thresholding, Pixel counting, Segmentation, Inpainting, Edge detection, Color Analysis, Blob discovery and manipulation, Neural net processing, Pattern recognition, Optical character recognition, blurring, normalized lighting, greyscaling, OTSU, thresholding, erosion/dilation, convert correct hull, contour detection, blob/mass calculation normalization, and/or Gauging/Metrology.
The electronic device 500 may also include one or more storage devices 524, such as a hard-drive, CD-ROM, or other computer readable media, for storing data and computer-readable instructions and/or software that implement exemplary embodiments of the environment 555, including the golf shot prediction engine 14 described herein. Exemplary storage device 524 may also store one or more databases for storing any suitable information required to implement exemplary embodiments. For example, exemplary storage device 524 can store one or more databases 526 for storing information, such as user performance information, golf course information, performance statistics, user profiles, performance analysis, image data, and/or any other information to be used by embodiments of the environment 555. For example, the database 526 can store the data 15, 18, 20, 22, and/or the inputs shown in FIGS. 3A-3G. The databases may be updated manually or automatically at any suitable time to add, delete, and/or update one or more items in the databases.
The electronic device 500 can include a network interface 512 configured to interface via one or more network devices 520 with one or more networks, for example, Local Area Network (LAN), Wide Area Network (WAN) or the Internet through a variety of connections including, but not limited to, standard telephone lines, LAN or WAN links (for example, 802.11,T1, T3, 56kb, X.25), broadband connections (for example, ISDN, Frame Relay, ATM), wireless connections, controller area network (CAN), or some combination of any or all of the above. The network interface 512 may include a built-in network adapter, network interface card, PCMCIA network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the computing device 500 to any type of network capable of communication and performing the operations described herein.
In exemplary embodiments, the electronic device 500 can include a RF transceiver 530. The RF transceiver 530 can be configured to transmit and/or receive wireless transmissions via an antenna 532. For example, the RF transceiver can be configured to transmit one or more messages, directly or indirectly, to one or more sensor modules and/or to a remote system (e.g., environment 10 shown in FIG. 1 ) and/or can be configured to receive one or more messages, directly or indirectly, from one or more sensor modules and/or the remote system. The RF transceiver 530 can be configured to transmit and/or receive messages having a specified frequency and/or according to a specified sequence and/or packet arrangement. As one example, the RF transceiver 530 can be a BlueTooth® transceiver configured to conform to a BlueTooth® wireless standard for transmitting and/or receiving short-wavelength radio transmissions typically in the frequency range of approximately 2.4 gigahertz (GHz) to approximately 2.48 GHz. As another example, the RF transceiver 530 can be a Wi-Fi transceiver (e.g., as defined IEEE 802.11 standards), which may operate in an identical or similar frequency range as BlueTooth®, but with higher power transmissions. Some other types of RF transceivers that can be implemented by the sensor module circuitry includes RF transceivers configured to transmit and/or receive transmissions according to the Zigbee® communication protocol, and/or any other suitable communication protocol.
The electronic device can include a GPS receiver 534. The GPS receiver 534 can be configured to receive GPS satellite transmissions including GPS data, which can be used by the environment 555 being executed by the processor 502 of the electronic device 500 to monitor and/or track a geographic location of the electronic device 500 (e.g., a longitude and latitude of the electronic device). For example, for embodiments implemented in a golfing environment, the electronic device 500 can receive a broadcast signal from a GPS satellite and can process the GPS data included in broadcast signal to determine a geographic location of the electronic device 500, which can be utilized by the environment 555 to determine a geographic location of the electronic device 500 on a golf course, relative to a hole on the golf course, a distance the electronic device 500 travelled between consecutive golf shots, and/or any other location based information. According to one or more embodiments described herein, the GPS data can be used to determine and/or approximate a location of a user (e.g., a golfer) to determine the golfer positional data 201.
The electronic device 500 may be any computer system, such as a laptop, handheld computer, tablet computer (e.g., the iPad™ tablet computer), mobile computing or communication device (e.g., the iPhone™ communication device or an Android™ communication device), or other form of computing or telecommunications device that is capable of communication and that has sufficient processor power and memory capacity to perform the operations described herein. The electronic device 500 may run any operating system 516, such as any of the versions of the Microsoft® Windows® operating systems, the different releases of the Unix and Linux operating systems, any version of the MacOS® for Macintosh computers, any version of the Android operating system, any version of the iOS operating system for the Apple iPhone and/or iPad, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, or any other operating system capable of running on the computing device and performing the operations described herein. In exemplary embodiments, the operating system 516 may be run in native mode or emulated mode. In an exemplary embodiment, the operating system 516 may be run on one or more cloud machine instances.
In some embodiments, the electronic device 500 can receive RF transmissions from a sensor module and/or can generate data including acceleration information, image data, other swing information, an indication that the sensor module detected an impact between the golf club and an object, and/or an indication of a golf shot. In response to the receipt or generation of the information/data, the processor 502 of the electronic device 500 can execute the environment 555 to determine whether the impact associated with the received or generated information/data is a false positive golf shot or whether a golf shot was executed. If the electronic device 500 determines that the impact is a false positive golf shot, the environment 555 can be executed by the processor 502 to suppress or ignore the data/information included in the transmission or generated by the electronic device 500.
In exemplary embodiments, the processor 502 can “listen” for detection of pressure waves from sensor modules based on an output of the microphone 528. For example, when the environment 555 is being executed in the foreground or background, the processor 502 can monitor an input corresponding to an output of the microphone 528 to determine whether the microphone 528 detected a pressure wave propagating from one or more of the sensor modules (e.g., based on movement or vibrations of the transducer in response to the pressure wave). In response to receipt of electrical signals corresponding to the pressure wave at the input of the processor 502 from the transducer 528, the processor 502 can process the electrical signals to determine information/data based on the receipt of the pressure wave itself and/or based on information/data encode in and extracted from the pressure wave. For example, in some embodiments, the electronic device 500 can detect pressure waves propagating from a sensor module that include acceleration information, other swing information, sensor module identification information, an indication that the sensor module detected an impact between the instrument and an object, and/or an indication of a golf shot.
FIG. 6 is a block diagram of an exemplary embodiment of the performance monitoring and/or tracking environment 555 that can be implemented by embodiments of the electronic device 500 to monitor and/or track a user's golfing performance. The environment 555 can include a user interface 610, a profile management engine 620, and a performance tracking engine 630, and the golf shot prediction engine 14.
In exemplary embodiments, the user interface 610 can be programmed and/or include executable code to provide one or more graphical user interfaces (GUIs) 612 through which a user can interact with the environment 555. The GUIs 612 displayed to users can include data entry areas to receive information from the user and/or can include data outputs to display information to the user. Some examples of data entry fields include, but are not limited to text boxes, check boxes, buttons, dropdown menus, and/or any other suitable data entry fields.
The profile management engine 620 can be programmed and/or configured to receive, maintain, modify, and/or update a user profile. In exemplary embodiments, the user profile can be created by the user upon an initial execution of the environment 555. As one example, the processor can execute the engine 620 to request user information including, for example, a user name, gender, weight, height, golf handicap, stance (e.g., right or left), an experience level (e.g., number of years playing, a number of rounds played in the previous year), and/or any other suitable user information. As another example, the processor can execute the engine 620 to collect and/or setup golf club information including, for example, an identity of the golf clubs to which the sensor modules are or will be affixed, an association between the sensor modules and their corresponding golf clubs, an estimated distance an object (e.g., a golf ball) will likely travel when the user strikes it with each golf club, and/or any other suitable golf club information that can be utilized by the environment 555 to facilitate tracking and/or monitoring a user's performance during an activity (e.g., a round of golf). In exemplary embodiments, the user profile can be maintained, modified, and/or updated to include statistic information related to the user's past performance. In exemplary embodiments, the statistic information can include an average score, a handicap, an average distance an object travels for each of the golf clubs, a user performance on specific golf courses, and/or any other statistic information that can be utilized, maintained, and/or created based on the tracking and/or monitoring of a user's performance during an activity (e.g., a round of golf).
In exemplary embodiments, the performance tracking engine 630 can be programmed and/or configured to receive and/or maintain information corresponding to specific golf courses and/or holes at a specific golf course. For example, the engine 630 can receive and/or maintain a geographic map of the golf course including information related to the terrain of the golf course, a location of the holes on the golf course, a par for the holes on the golf course, and/or any other suitable information related to golf courses. In some embodiments, the golf course information can be maintained in a database of the remote system and the electronic device can request the golf course information from the database in response to an input from the user. In some embodiments, the golf course information can be stored on the electronic device executing the environment 555.
The performance tracking engine 630 can be executed by the processor to monitor transmissions from various sensor modules as described herein and to process the transmissions. For example, in exemplary embodiments, transmissions from the sensor modules can include information corresponding to accelerometer information, an indication of an impact between a golf club and an object (e.g., a golf ball or the earth), an indication of a golf shot, swing analysis information (e.g., a swing speed, a swing tempo, swing force, club face angle, swing plane, etc., represented via accelerometer output information), and/or any other suitable information related to an operation of the sensor module and/or a utilization of the golf club. The information received by the electronic device can be utilized upon execution of the engine 630 to identify a location at which a golf shot occurred, identify a number of golf shots that occurred for a particular hole, identify a golf score for a particular hole or course, provide a swing analysis, identify false positive impacts/golf shots (e.g., using criteria described herein), and the like. The information received from the transmissions can also be provided to the engine 630 to create, update, and/or modify statistic information in the user profile.
In exemplary embodiments, embodiments of the sensor modules (e.g., sensor module 110) can be integrated with an embodiment of the electronic device 500, where the components of the electronic device 500 can include the components of the sensor module 110 and the electronic device 500 can execute the performance monitoring and/or tracking environment 555, including the golf shot prediction engine 14.
FIG. 7 is a block diagram of one of many exemplary servers 700 for implementing embodiments of the environment 10, including implementing the golf shot prediction engine 14, in accordance with embodiments of the present disclosure. The server 700 includes one or more non-transitory computer-readable media for storing one or more computer-executable instructions or software for implementing exemplary embodiments. The non-transitory computer-readable media may include, but are not limited to, one or more types of hardware memory, non-transitory tangible media (for example, one or more magnetic storage disks, one or more optical disks, one or more flash drives), and the like. For example, memory 706 included in the server 700 may store computer-readable and computer-executable instructions or software for implementing exemplary embodiments of the environment 10 or portions thereof.
The server 700 also includes configurable and/or programmable processor 702 and associated core(s) 704, and optionally, one or more additional configurable and/or programmable processor(s) 702′ and associated core(s) 704′ (for example, in the case of computer systems having multiple processors/cores), for executing computer-readable and computer-executable instructions or software stored in the memory 706 and other programs for controlling system hardware. Processor 702 and processor(s) 702′ may each be a single core processor or multiple core (704 and 704′) processor.
Virtualization may be employed in the server 700 so that infrastructure and resources in the computing device may be shared dynamically. One or more virtual machines 714 may be provided to handle a process running on multiple processors so that the process appears to be using only one computing resource rather than multiple computing resources, and/or to allocate computing resources to perform functions and operations associated with the environment. Multiple virtual machines may also be used with one processor or can be distributed across several processors.
Memory 706 may include a computer system memory or random access memory, such as DRAM, SRAM, EDO RAM, and the like. Memory 706 may include other types of memory as well, or combinations thereof.
The server 700 may also include one or more storage devices 724, such as a hard-drive, CD-ROM, mass storage flash drive, or other computer readable media, for storing data and computer-readable instructions and/or software that can be executed by the processor 702 to implement exemplary embodiments of the environment 10 described herein.
The server 700 can include a network interface 712 configured to interface via one or more network devices 722 with one or more networks, for example, Local Area Network (LAN), Wide Area Network (WAN) or the Internet through a variety of connections including, but not limited to, standard telephone lines, LAN or WAN links (for example, 802.11, T1, T3, 56kb, X.25), broadband connections (for example, ISDN, Frame Relay, ATM), wireless connections (including via cellular base stations), controller area network (CAN), or some combination of any or all of the above. The network interface 712 may include a built-in network adapter, network interface card, PCMCIA network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the server 700 to any type of network capable of communication and performing the operations described herein. While the server 700 depicted in FIG. 7 is implemented as a server, exemplary embodiments of the server 700 can be any computer system, such as a workstation, desktop computer or other form of computing or telecommunications device that is capable of communication with other devices either by wireless communication or wired communication and that has sufficient processor power and memory capacity to perform the operations described herein.
The server 700 may run any server application 716, such as any of the versions of server applications including any Unix-based server applications, Linux-based server application, any proprietary server applications, or any other server applications capable of running on the server 700 and performing the operations described herein. An example of a server application that can run on the computing device includes the Apache server application.
In describing exemplary embodiments, specific terminology is used for the sake of clarity. For purposes of description, each specific term is intended to at least include all technical and functional equivalents that operate in a similar manner to accomplish a similar purpose. Additionally, in some instances where a particular exemplary embodiment includes a plurality of system elements, device components or method steps, those elements, components or steps may be replaced with a single element, component or step. Likewise, a single element, component or step may be replaced with a plurality of elements, components or steps that serve the same purpose. Moreover, while exemplary embodiments have been shown and described with references to particular embodiments thereof, those of ordinary skill in the art will understand that various substitutions and alterations in form and detail may be made therein without departing from the scope of the invention. Further still, other embodiments, functions and advantages are also within the scope of the invention.
Exemplary flowcharts are provided herein for illustrative purposes and are non-limiting examples of methods. One of ordinary skill in the art will recognize that exemplary methods may include more or fewer steps than those illustrated in the exemplary flowcharts, and that the steps in the exemplary flowcharts may be performed in a different order than the order shown in the illustrative flowcharts.

Claims

1. A method for predicting whether a user executed a golf shot, the method comprising:

receiving a selection of a machine learning model from a plurality of machine learning models, each of the plurality of machine learning models being trained to predict whether the golf shot occurred;

receiving inputs relevant to the machine learning model that was selected; and

predicting, using the machine learning model that was selected, whether the user executed the golf shot using the inputs relevant to the machine learning model that was selected.

2. The method of claim 1, further comprising, prior to receiving the selection of the machine learning model, determining whether the user was stationary long enough to potentially hit the golf shot.

3. The method of claim 2, wherein the determining is performed by comparing a time the user was stationary to a threshold, the threshold being based on information about the user and information about a golf course.

4. The method of claim 2, further comprising, responsive to determining that the user was stationary long enough to potentially hit the golf shot, receiving the selection of the machine learning model.

5. The method of claim 2, further comprising, responsive to determining that the user was stationary long enough to potentially hit the golf shot, determining that no golf shot occurred.

6. The method of claim 2, further comprising, prior to determining whether the user was stationary long enough to potentially hit the golf shot, receiving at least one of course information and user positional data.

7. The method of claim 1, wherein the selection of the machine learning model is based at least in part on a position of the user relative to a hole on a golf course.

8. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least user profile information for the user, the machine learning model that was selected being trained to take as input the user profile information for the user and to predict whether the user executed the golf shot.

9. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least positional and hole history information, the machine learning model that was selected being trained to take as input the positional and hole history information and to predict whether the user executed the golf shot.

10. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least a swing detection from a hip movement swing model, the machine learning model that was selected being trained to take as input the swing detection from the hip movement swing model and to predict whether the user executed the golf shot.

11. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least a previous shot outcome model output, the machine learning model that was selected being trained to take as input the previous shot outcome model output and to predict whether the user executed the golf shot.

12. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least audio hit probability information, the machine learning model that was selected being trained to take as input the audio hit probability information and to predict whether the user executed the golf shot.

13. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least head movement data, the machine learning model that was selected being trained to take as input the head movement data and predict whether the user executed the golf shot.

14. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least a swing detection from a watch swing model, the machine learning model that was selected being trained to take as input the swing detection from the watch swing model whether the user executed the golf shot.

15. The method of claim 1, wherein the inputs relevant to the machine learning model that was selected comprise at least: user profile information for the user, positional and hole history information, a swing detection from a hip movement swing model, a previous shot outcome model output, audio hit probability information, head movement data, and a swing detection from a watch swing model, the machine learning model that was selected being trained to take as input the inputs relevant to the machine learning model.

16. The method of claim 1, wherein the predicting comprises generating a prediction value and comparing the prediction value to a threshold.

17. The method of claim 16, wherein it is determined that the user executed the golf shot responsive to the prediction value satisfying the threshold.

18. The method of claim 16, wherein it is determined that the user did not execute the golf shot responsive to the prediction value failing to satisfy the threshold.

19. A system comprising:

a memory comprising computer readable instructions; and

at least one processor for executing the computer readable instructions, the computer readable instructions controlling the at least one processor to perform operations for predicting whether a user executed a golf shot, the operations comprising:

receiving inputs relevant to the machine learning model that was selected; and

20. A non-transitory computer-readable medium comprising instructions, wherein execution of the instructions by at least one processor causes the at least one processor to perform operations for predicting whether a user executed a golf shot, the operations comprising:

receiving inputs relevant to the machine learning model that was selected; and