Last Monday morning a curious post hit Twitter, something baseball’s statistical community hadn’t really seen before. An outfit called Quality of Pitch had received a patent on their metric, as they presented it. The response was fairly immediate and unwelcoming, as you can see if you follow the link.
BREAKING: We have now received our patent from the USPTO for the method of combining measurements of movement, location and velocity to quantify pitches in baseball. This now certifies QOP as THE metric for calculating pitch quality. Look for QOP during future MLB broadcasts. pic.twitter.com/DeHOuTZRcp
— MLB Quality of Pitch (@qopbaseball) September 14, 2020
The tenor of the response probably had something to do with the fact that baseball’s sabermetric community has traditionally been a collaborative space, at least for those who come with good research and modest ambitions. The back and forth on message boards, comments sections, and Twitter can resemble academia: In their free time, researchers will look into something, try to explain it, publish their findings, and then wait to see what others think of their effort. If their work is replicated enough and inspires further advancement, it can find its way onto the pages of FanGraphs or Baseball Prospectus or Baseball-Reference and become an accepted part of the statistical canon.
Advertisement
Although there are patents in baseball, they’re often for more mechanical or physical things like Hawk-Eye’s camera tracking system. MLBAM’s Cory Schwartz has a patent on real-time pitch classification that has helped Statcast produce the pitch type displays you see at a ballgame and on broadcasts. What makes this patent a little different is the focus on a metric, as well as the efforts on the part of QOP to enforce the patents on their pitch quality metric.
This writer, as well as a freelance analyst for The Athletic, Ethan Moore, received a vaguely threatening direct message last Monday, about the same time the tweet went up. Moore has written about “stuff” metrics, and done research with test results and open-source coding, in partnership with this writer. We’ve published those results here all season in The Command & Stuff Report.
Once news of these direct messages spread, the reaction was even more heated than those to the original tweet.
“It’s moronic,” said Voros McCracken, a veteran sabermetrician.
“How is sending cease and desists to people who have already shown they will do unreasonable amounts of independent investigatory work for little to no money the right strategic move?” said Mike Rathwell, the CEO of Driveline Baseball. As an independent pitching development lab, Driveline has undergone multiple attempts to calculate pitch quality; you have to know what makes a pitch good in order to try and design pitches to do well, after all. “Why did they choose to do this? It’s not helpful to anyone.”
When speaking with multiple patent lawyers, it’s not clear the patent supports the language in the tweet and the DM.
“Lots of times I get people calling me, freaking out, saying ‘Hey I sell a machine for making widgets and this other company got a patent for a machine making widgets and they sent me a cease and desist letter and I’m screwed,’” said Hunter Adams, registered patent lawyer in Mobile, Ala., and proprietor of the AdamsIP firm. “The patent owner can say lots of things, patentable or not, in the written description of the patent, but to say you are infringing they have to prove that you are infringing on every single element in one claim.”
Advertisement
“If you’re not infringing on claim one, you’re not infringing,” he said more succinctly. “If four legs and a seat and a backrest is required in that claim, and your chair only has three legs, then you’re not infringing.”
The first claim on the QOP patent is a long one, but you can read the whole thing if you like. You might notice that the claim begins with the mention of a ton of hardware, hardware that would be familiar to anyone who has followed along as Major League Baseball’s Advanced Media (MLBAM) arm has installed PITCHf/x, TrackMan, and then Hawk-Eye systems in order to improve the quality of their data. SportVision’s PITCHf/x is a video-based system, TrackMan is a doppler radar system, and Hawk-Eye is an “imaging system.”
The QOP patent, issued to Greiner’s son Jarvis Greiner and Jason Wilson, describes a process that goes all the way from hardware to a display system, much like baseball’s statistics on BaseballSavant come all the way from the Hawk-Eye sensors to a “real-time” display system. Does it matter that MLBAM’s partners already have patents on their hardware, and that MLBAM has been able to produce real-time pitch ratings from those sensors for years? In 2010, they made available a statistic called “Nasty Factor” that combines velocity, sequence, location and movement.
MLB declined to comment on whether they had a relationship with QOP or were aware that their logo was being used on letters to analysts regarding this statistic. Other than MLBAM, it isn’t likely that there is one place that combines the hardware, the approach and the resources to violate every single part of the claim. And maybe that’s not the central concern for QOP’s Wayne Greiner and Jason Wilson, who were more worried about the name Moore gave his statistic.
“I maybe owe an apology,” said Greiner when contacted. “I definitely don’t want to give you the wrong idea. When we saw that he was referring to his metric as ‘Quality of Pitch’ that set off alarm bells. We wanted to protect the integrity of our work.”
Advertisement
Greiner and Wilson’s Quality of Pitch is a statistic that combines speed, location and movement into a single numeric value, which is an approach that makes sense. Though the pair of researchers have presented and written on their statistic in a few places, their data and code isn’t necessarily available for scrutiny, and that might be because of the peculiar way the statistic was born.
Greiner and Wilson began by filming a local pitcher throwing 30 pitches in front of a local coach. After that coach rated each pitch, the duo thought they understood which aspects of a pitch they should test against the millions of pitches tracked by baseball. Using a regression model, they found weights to relate these aspects of movement, velocity and location to outcomes. Now that rating is patented, and QOP is attempting to make it the standard by affirming that any other statistic is an infringement.
Most pitch quality metrics out there focus on the relationship between a pitcher’s pitches, because of seminal work on the subject that found that, for example, a changeup’s quality is best determined if you define that changeup off of the fastball, as per Harry Pavlidis. Even determining one pitch’s quality depends on aspects of the other pitches in the pitcher’s arsenal, in other words.
Pavlidis’ work fits into a long line of research on the subject that includes, but is not limited to, the following valuable pieces:
- Don Malcolm and Ken Adams on the Quality Matrix (QMAX) in 1997
- John Walsh on “Searching for the game’s best pitch” in 2008
- Jeremy Greenhouse’s Stuff metric in 2009
- Mike Fast on the value of velocity in 2010
- Nasty Factor from MLBAM in 2010
- Jon Roegele on Pitch Quality with location and velocity in 2013
- Harry Pavlidis on “What Makes a Changeup Good?” in 2013
- Eno Sarris on the effects of ride on the fastball (on defunct Fox site)
- Eno Sarris on “What Makes a Curve Good?” (on defunct Fox site) in 2015
- Alex Chamberlain on modeling whiffs and grounders with velo and movement in 2018
- Eno Sarris and Andrew Perpetua on “Using Movement, Velocity & Location to Predict a Swinging Strike” in 2018
- Eno Sarris on “What Makes a Slider Good?” in 2019
- Connor Kurcon on the location aspect of command in 2019
- Tom Tango on the effect of velocity on outcomes in 2019
- Tom Tango on the effect of movement on outcomes in 2019
- Aaron Sauceda’s ACES pitch quality metric in 2019
- Ethan Moore on his pitch quality metric in 2020
Generally, each of these authors submitted their work with links to past work as an effort to further the conversation, to contribute to the larger body of work that is trying to judge the quality of a pitch based on movement and velocity. More often these days, location has been removed or is treated as a separate part of the work because location varies more pitch to pitch — Ben Lindbergh found that pitchers miss their target by 13 inches on average — while velocity and movement stay more steady. In fact, when QOP was posted to Tom Tango’s blog, which often serves as a community flashpoint for this sort of conversation, the fact that location was included was part of the critique.
The normal flow within the community would be to post this sort of research, see the critiques, and work on the metric, hoping that a team would see the work and value it enough to give you a consulting gig or a job. Just ask McCracken, who made one of the bigger advances in sabermetrics when he discovered that pitchers had less control over the results on a ball in play than we’d thought. Should he have patented that stat, which he named DIPS back in 1999 when he posted it on the rec.sport.baseball forum?
“Patenting it would have done absolutely no good,” said McCracken. “The Catch-22 with DIPS is that it was extremely valuable, but the only way I could convince anyone of the value was by giving it away for free. If I didn’t give it away for free, nobody would know about it and the only way to monetize it would be betting on baseball. I was some nobody — if I said ‘I have this really valuable thing,’ nobody would know. Even if these guys have a valuable thing, how would any of us know?”
Advertisement
So researchers put their stuff out there to prove its usefulness, and eventually, it ends up on a site like FanGraphs or Baseball Prospectus. But it wouldn’t have gotten there without first being put out into the public sphere, as McCracken points out. Eventually, DIPS was turned into Fielding Independent Pitching by Tango and became the backbone of Wins Above Replacement on FanGraphs. Tango is now MLBAM’s senior data architect.
“If Tango didn’t make his stuff open-source, a lot of the stuff on FanGraphs wouldn’t be there,” said FanGraphs founder David Appelman. “WAR wouldn’t be there.”
But Greiner believed the way WAR ended up becoming mainstream was problematic.
“When we went to broadcasters they were interested in if it was patent-pending,” said Greiner of the QOP stat. “We don’t want to have a lot of confusion or loss of credibility. What’s diminished WAR is that there’s two or three versions and so people have thrown up their hands.”
“I think most fans don’t even know there are multiple WARs,” said broadcaster Jon “Boog” Sciambi. “If they knew, they’d find it confusing. For me, it’s more annoying that it’s not standard.” (But he also made clear that it doesn’t mean that he would support patenting a stat, since the world of broadcasting and sabermetric research aren’t necessarily the same thing.)
The reason there are multiple WARs is not only that nobody patented it, but that the community often functions as an academic one, where people publish results in an open-source manner so that the approach can be verified and furthered.
“I hope that here at Sports Reference we are seen as people that cite their sources and give credit to people that come before us and provided those insights that help us,” said Sports Reference president Sean Forman. “I’m an academic myself, I generally come from that background — sharing the basis of things is important.”
Advertisement
When Gregory J. Matthews created his version of WAR, called openWAR, with Ben Baumer and Shane Jensen in 2015, the entire focus was on creating something that would feel at home in academia, something that everyone could calculate and use.
“It was pretty hard to exactly reproduce other versions of WAR,” said Matthews, the director of data science at the Loyola University of Chicago (and senior statistical advisor to Baseball Prospectus). “So we wanted to take the statistical and scientific principles and apply them to WAR. We had rules going in: We had to use stats that were available, all of our code had to be public and available, and we also wanted to add a plus or minus factor to represent the uncertainty.”
“I don’t know much about patent law to really know whether they have any justifiable claims,” said one analyst about patenting QOP. “I just feel bad for independent 21-year-old analysts that will get scared out of doing any good research/publishing more on the subject because of this. This kind of thing has such a chilling effect on the game and research.”
“I know on social media some people are maybe offended we got this patent, or disappointed we got this patent,” said Greiner. “What they are interpreting is that we are looking at this from a business perspective, which we are not. We are fans of baseball like everybody else. We want to see baseball grow. We’re not looking at hindering future work of the sabermetrics community at all. We want to discourage blatant replication of the work. We could work with people to be a gatekeeper with this and take this research further.”
Moore’s work is posted with his code on his website, but as even Greiner admitted, Moore did not undergo the same process or use the same statistics as Greiner and Wilson.
“We know it’s not the same method,” Greiner agreed. “That message we sent you was more of a nudge. You just can’t blatantly go out with an entirely different method and call it the exact same thing as something else.”
Moore said he was open to changing the name of his statistic, and that any similarity between the names was unintentional. That’s more of a trademark question, however, as that process protects things like product names.
Advertisement
Patenting a stat may be at odds with the community’s values of collaboration and openness — but it’s also worth wondering if this sort of thing is a sign of the times, if the field is changing. Can sabermetrics continue to act like academia even as the industry it has attached itself to, baseball, grows into a multi-billion-dollar behemoth?
“It was more academia than academia itself, because back in the day we didn’t have that structure that academia has,” said McCracken. “We could claim that we were all open-source for some noble reason, but the reason we were open-source is because, back in the day, before Moneyball, it wasn’t worth anything. From that background, we got used to coming up with an idea, being completely open with everything, that’s what we’re used to. But that was just a historical artifact of what was going on back in the day; there was no way to sell any of it.”
“I grew up on rec.sport.baseball, where everyone was arguing, we were all presenting information,” Forman said of the old messaging boards that begat so much of modern analytical thought in baseball. “I would tie this change into the teams. Teams have started hiring the best analysts and then the information becomes proprietary. The other thing now is that the data is becoming proprietary, and it’s harder to recreate those things in the public space.”
Forman went on to say these trends in public analysis are wrong-headed — he’s written about this in the past — because open data can create an explosion of thought in the public space and create a sort of farm system of willing analysts. It has in the past. But teams now have more than 20 analysts looking into the data, looking for a way to get a leg up on the competition. Those are positions that rec.sport.baseball posters only dreamed about back in the day, but they are real jobs now, and coveted ones.
“Teams aren’t always the best actors,” said Forman, and so maybe the QOP analysts were just trying to protect themselves from that kind of cribbing off public work that can take place. Those analysts could theoretically take the idea and recreate it internally without Wilson and Greiner, if they didn’t have a stat like that already — and sources from at least five teams confirmed they have a quality of pitch statistic like this for internal use. And then there’s also the matter of the threatening letters to public analysts.
“I have sympathy for everyone involved,” McCracken said. “I don’t like being broke any more than the next guy. When you get a shot doing baseball instead of a normal job, it’s a very attractive thing and you want to give your best effort because it’s better than working for the insurance company. I understand trying to explore every avenue for potential monetization.”
“I don’t want the message going that we are maximizing profit, that’s not the case,” Greiner said.
Advertisement
Still: Researchers have been trying to define the quality of a pitch for over 20 years. It’s doubtful a dubious intellectual hold on a single statistic will halt this train of thought.
“It’s just an idea,” McCracken said. “Once you teach someone something, they know it, and forbidding them from using their knowledge doesn’t work. You can’t own an idea.”
(Top photo: Steve Mitchell / USA Today)