In the relative Stone Age of 2022, USGA rules expert Craig Winter was hanging out in the living room of his Washington State home with a friend of his wife’s. The acquaintance was playing around on his computer with a whiz-bang new app called ChatGPT that allowed users with no coding chops to interact with artificial intelligence through a simple conversational text box.
When Winter finally convinced his wife’s friend to let Winter ask the super-bot a question, the subject matter surely came as no surprise: the Rules of Golf, more specifically a query about cart-path relief. In went the question and out came an answer, with no less than the USGA’s senior director of rules of golf and amateur status judging its merit.
“I was candidly really surprised with how good the answer was,” Winter recalled in a call with reporters Tuesday.
After more experimenting with Chat, Winter’s wheels began turning. So did his colleagues’. Could the USGA, on the back of its expertise in the rules space, team with a tech-solutions company to create an AI-powered rules engine that was better than anything any other provider could build?
The organization weighed proposals from several potential collaborators before settling on a pitch from one of its own corporate partners — professional services firm Deloitte — that, Winter said, “showed us that there was potential that was quite a bit more than what we had on our own plate.”
This was in April 2023.
On Wednesday, the USGA announced the fruits of that partnership: the launch of the pilot phase for Rules AI, a tool that will reside in the GHIN app and, the USGA says, give golfers “an accurate, streamlined way” to find answers to their rules questions.
Simply punch in your prompt — how do yellow stakes differ from red; what’s the penalty for carrying 15 clubs; what to do if a squirrel swipes your ball, etc. — and the app will deliver how-to-proceed guidance in authoritative language culled from the Rules of Golf, the Decisions on the Rules of Golf and the trove of rules questions that the USGA answers daily by phone, email and via its rules app (more than 20,000 such queries pour in every year). In total, it’s a vast resource of rules knowledge on which Rules AI has trained itself.
For the moment, the app is available only in beta mode to a cluster of handpicked USGA member clubs, but the association has plans to expand that test group to about 80,000 or 90,000 members of New York metropolitan area clubs later this month. The app will be available to all GHIN members by the spring of next year.
If you know anything about the USGA, you probably know it treats rules governance with the diligence of a detonation physicist handling Plutonium-239. Every word is pored over, every adjustment and amendment agonized over. So, it may come as some surprise that the USGA has entrusted the game’s commandments to a technology that while stunningly powerful also is prone to dispensing flawed information, if not outright hallucinating.
That’s where the last two years come in.
“Our approach from the very start is we need to provide accurate answers,” said Anthony Santora, the USGA’s managing director, IT, who joined the Tuesday call with Winter. “There’s a lot of misinformation out there on the internet.” Santora added that any prospect of Rules AI providing bad intel was a “non-negotiable. So, for us, we’ve been really taking this guardrail phased approach to everything.”
At first, that meant training the engine on the 25 primary rules in the rules book, along with those rules’ many subsections. Then came a more nuanced education by way of 35,000 questions and answers from the USGA’s rules-query database. That was followed by feeding the bot real-time rules questions that golfers emailed to the USGA — with the USGA’s rules department reviewing how effectively the bot answered them. This has been an essential step in grooming the app’s intelligence. As the USGA’s rules whizzes have flagged Rules AI providing inaccurate or incomplete answers, they have been able to proactively train it to not make the same mistake again. This legwork also would make it challenging for a competitor to swoop in and create an equally reliable rules tool.
“We’re able to continue to make this better and to be able to teach it because we’re hands on, looking at it, looking at questions coming in, and seeing answers that are coming out,” Santora said.
Week by week and month by month, the app got smarter and more sophisticated.
“I’ve spent a good amount of time dreaming about the possibility of this actually working,” Winter said, “but not really getting to that point until we were almost 16, 18 months in.”
Every potential question ladders up into one of 500 different rules topics — i.e., “out of bounds,” “cart-path relief,” etc. “When a question comes in, it’s going to ultimately land in the topic that is most similar to what this person is asking,” Winter said. “So all of that nuance is available for the system to be able to use this closed set of information that’s accurate to find the answer.”
That’s not to say Rules AI is perfect. No machine, no matter how thoroughly it has been trained, can be omniscient, especially when it comes to the Rules of Golf, where scenarios without precedent can seemingly appear out of thin air. When the app receives a question it feels unqualified to answer (and no doubt, as the app becomes more widely available, an army of fast-typing rules sharpies will revel in trying to stump the bot), it is wired to refer users to the USGA for further counsel.
Complex questions can take Rules AI 10 seconds or more to answer but more commonly asked questions will have what the USGA calls “golden answers” that the app will deliver almost instantaneously.
Winter stressed that Rules AI should not be used to supersede tournament or club committees’ expertise or rulings; it’s more so a reference that allows players easier and faster access to rules guidance, which likely also will lower the volume of rules queries the USGA fields the old-fashioned way.
“We’re not trying to put a rules official in golfers’ pockets,” Winter said. “This is about scaling the expertise of our rules staff, our rules experts, to golf around the globe.”
You must be logged in to post a comment Login