Surveillance is often described as an “art.” But what if we viewed it more as a science, and evaluated our success or failure by the numbers?
[quote align=”center” color=”#999999″]Baseball is an island of activity amidst a sea of statistics. ~Author Unknown[/quote]
How do you measure the success of your surveillance ops, of your operators, and of your investigations company? I’d argue that many of the ways we define success are pretty unscientific, based on the untested “conventional wisdom” of our field.
But what if we adopted the “Moneyball” approach—a more objective and quantifiable strategy for evaluating surveillance operators—much like the way the Oakland A’s re-thought their approach to scouting major leaguers in the early 2000s? By comparing investigators and surveillance vendors through statistical analysis, I think we can improve our game, by identifying specific areas we can improve to get better, more consistent results.
[quote align=”center” color=”#999999″]During my 18 years, I came to bat almost 10,000 times. I struck out about 1,700 times and walked maybe 1,800 times. You figure a ballplayer will average about 500 at bats a season. That means I played seven years without ever hitting the ball. ~Mickey Mantle, 1970[/quote]
Sabermetrics (Wikipedia): The empirical analysis of baseball, especially baseball statistics that measure in-game activity. (The term is derived from the acronym SABR—the Society for American Baseball Research.)
Surveilmetrics – Working Definition: The empirical analysis of surveillance statistics, especially statistics that measure on-surveillance case activity.
Lies, Damned Lies, and Statistics
Michael Lewis’s book, Moneyball: The Art of Winning an Unfair Game (and the movie starring Brad Pitt), poses a fascinating question: How did the Oakland A’s, one of the poorest teams in major league baseball, win so many games?
It’s a story of numbers and misplaced assumptions, and of a big, bold idea: What if everything people believed about building winning teams was wrong? And if so, is there a better way to win baseball games—say, by recruiting reliable workhorses who get on base a lot instead of flashy superstars who hit home runs?
In Moneyball, Michael Lewis profiles Bill James, the father of sabermetrics, and Billy Beane, a former high school baseball phenom who never lived up to his potential in the majors and went on to manage the Oakland Athletics. Lewis tells the story of how James and Beane convinced the A’s organization to try a new, analytical approach to determining a ballplayer’s potential as a big leaguer. The A’s went on to play competitively against teams with much larger payrolls, and made it to the playoffs in 2002 and 2003.
In the end, the A’s rejected the “normal” way things were done in major league baseball and dared to be different. Baseball doesn’t much care for different, but Oakland’s success proved that different is sometimes better.
“Conventional wisdom,” too often, is a belief system unsupported by evidence. And too often, it’s just plain wrong.
The Stats on Surveillance
It has been said that analysis can lead to paralysis. You never saw Hank Aaron or Barry Bonds standing in the batter’s box with a slide rule and a calculator.
Nor should the major league surveillance operative be plugging case variables into his laptop to decide if they should tail their subject down some dark dirt road or up a blind alley.
Many larger surveillance firms do track an investigator’s individual statistics—secretly. And they should. They want to know how each surveillance operative is doing in the field. And when the numbers are good, those numbers may be used for in-house competitive reasons: averages to be highlighted, to identify who the “good” investigators are so others may be motivated to follow. And when their numbers are bad, they can confidently inform their surveillance operative, based on your averages, we are sending you back down to the Toledo Mud Hens.
[quote align=”center” color=”#999999″]Baseball fans love numbers. They love to swirl them around their mouths like Bordeaux wine. ~Pat Conroy[/quote]
For Billy Beane and Paul DePodesta, the Harvard-educated head of R&D for the A’s, the most important player statistic was not how fast a guy could run or how well he could catch and throw a ball. Most of all, they cared about one thing: the player’s ability to GET ON BASE. They didn’t care whether he walked or hit ugly, blooping singles; they just wanted guys who, everyone say it with me, GOT ON BASE.
The same can be said for surveillance operatives. Successful surveillance operatives do one thing better than any of their counterparts: THEY SHOOT MORE VIDEO. Or as we said in the olden days, THEY SHOOT MORE TAPE. They flat-out just find ways to get more video, day in and day out.
Does that mean every case is a home run with five hours of video of a claimant shingling his house? Not at all. But an investigator who finds a way to SHOOT MORE VIDEO then everyone else will, statistically speaking, be more likely to get something that helps our clients.
DVP Daily Video Percentage: 16.00 Minutes of Video DIVIDED BY 8.00 hours of surveillance = 200 DVP (Yes, if you shoot 80 minutes of video – you will be batting a DVP of 1000 and if you shoot 120 minutes you will have a DVP of 1500 and so on. And yes – you should drop the decimal point.)
AVP Average Video Percentage: Average of DVP’s calculated over several cases = AVP. Example: DVP’s added and totaled for each case = 4440. That total divided by 20 cases = 222 AVP OR you could break it down further; 750 minutes of video shot over a course of twenty cases (40 days surveillance / 320 hours of surveillance) = 750 minutes of video divided by 320 hours of surveillance = 234 AVP
I shot my first work comp video in 1988 and base my thesis on my history in this business (mostly work comp surveillance), which includes working for a regional firm and a national firm (who did track statistics to a degree). I’ve found that operatives who shoot the most amount of video tend to be the same operatives who shoot high percentages of solid video evidence.
Of course, there’s also the question of what is solid evidence and what is subjective evidence. But that’s not up to the investigator to determine. Even when a PI gathers video he believes may refute a claim, the medical reviewer or plaintiff’s attorney might review the video and contend that the claimant is not exceeding restrictions or is not exceeding what they say they can’t do.
Many times on a work comp surveillance job, we never know how the case ends up. So I believe that success on a work comp case comes down to the operative who consistently shoots many minutes of video of the claimant’s activities. These are the operatives who are going to consistently get on base—and ultimately, show the client and the court a true and unbiased picture of the claim. That, in the end, should be everyone’s goal.
Putting the Numbers into Context
Using surveillance statistics can illuminate success and failure. It identifies ratios that can be broken out by investigator, case manager, company, client, region, etc. A vendor manager for B.I.G. Insurance Company might then track their surveillance vendors’ (AVP) Average Video Percentage, which would allow them to better compare one national firm to a regional firm or to that local vendor. It might highlight how one national surveillance vendor is hitting home runs in Michigan but is striking out in Florida. It will show how a regional vendor has had a slip in their AVP from one year to another and a local vendor has made great strides to up their game. It might even identify if cases assigned by a certain claims adjuster are not very successful even though the vendor they are using is doing generally good work elsewhere.
But you need to put these statistics into further context. It wouldn’t be fair to compare a set of cases and their AVPs from a surveillance vendor in Manhattan, Kansas to a surveillance vendor’s average in Manhattan, New York. These are two different ballparks, and some ballparks are more surveillance friendly than others. I would suggest comparing statistics by region to better clarify the numbers.
Do surveillance clients track these statistics? Do they even care? In the end, I do think surveillance statistics will speak to an investigator’s success ratios over time. And without these numbers, no one will really know how successful a surveillance operative or company actually is.
[quote align=”center” color=”#999999″]No matter how good you are, you’re going to lose one-third of your games. No matter how bad you are, you’re going to win one-third of your games. It’s the other third that makes the difference. ~Tommy Lasorda[/quote]
Clients tend to remember your home runs, like the time you got video of the claimant roofing houses for cash under the table. They’ll also remember the strikeouts, like the time you got burned bad on a case, and fecal gravity was set in motion, and the subject of the surveillance called the cops and his attorney, and that attorney called the insurance companies attorney, who yelled at the claims manger, who yelled at the adjuster, who yelled at the surveillance company’s case manger, who screamed at the surveillance operative, who muttered something about unhelpful traffic light patterns and the vagaries of rush hour traffic in Omaha, Nebraska when the College World Series crowds are in town. (Sorry, a little hyperbole in an attempt to make a point.)
To be clear, clients don’t usually yell. They usually say nothing and just don’t use you for awhile, or ever again. But without the statistics, they don’t have a true overall understanding of what a particular surveillance company or individual investigator’s averages are, the degree of success they are having with their cases, and how those averages might compare to others. The surveillance client makes hiring decisions based on a vague “gut” feeling they have for the vendor’s work, and by a few anecdotal wins or —which like in baseball, may be misleading without sound statistics to steer by.
Fielding Errors: On Losing Your Subject
In Moneyball, Michael Lewis recounts how Bill James thought the recording of fielding errors in baseball was nonsensical in the modern age. James believed that “errors” were identified based on one person’s moralistic judgement. And he pointed out that the fielder had to do something right (be in the correct position in the first place) for the error to occur, and that what he did right went unacknowledged.
Losing your subject on surveillance is judged just as harshly. Many clients and case managers view losing a subject as evidence of some deep moral or character flaw. This would never happen to a real surveillance operative, they declare, with sanctimonious disbelief, from their desks.
Let me explain something to all the surveillance case managers and clients out there: Surveillance operatives do NOT enjoy losing their subject or getting burned on a case. It is the most damnable, frustrating feeling in the world.
[quote align=”center” color=”#999999″]If a woman has to choose between catching a fly ball and saving an infant’s life, she will choose to save the infant’s life without even considering if there are men on base. ~Dave Barry [/quote]
I once lost a group of elk hunters as we drove though a Wyoming blizzard. It was 3:00 in the morning, and I’d been awake for more than 24 hours. Fierce gusts were sweeping semi-trucks off the highway, and I’d been tailing the subject for most of the night.
None of those details mattered. I was considered lower than dirt for allowing such a thing to happen.
I know many surveillance case managers and clients are saying, “But some surveillance operatives lose their subjects all the damn time. It is just as frustrating on our end.” I get it. If I had a staff of investigators, I would track the number of times each investigator lost their subject during the course of a surveillance. But I would only view it as a passing number to put their work into context—not as moral compass, but as a guidepost that maybe further training is required. And I’d want to know more than just the number. After all, it could be that I’m sending my best operatives on the toughest missions, which essentially sets them up to “fail” more often.
Certainly, there are investigators who just don’t get the concept of surveillance. In fact, many don’t. Surveillance requires grit, guile, and common sense. All three are required. If one is missing, they probably are not going to succeed.
But it should be the numbers that illuminate that fact, not subjective emotion.
What the Numbers Reveal
When it comes to surveillance, I believe that statistics are important. Bill James spoke eloquently of the role of statistics—how, through language, the numbers describe the hidden meaning of baseball: “And it is not just baseball that these numbers, through a fractured mirror, describe,” he says in Moneyball.
“It is character. It is psychology, it is history, it is power, it is grace, glory, consistency, sacrifice, courage, it is success and failure, it is frustration and bad luck, it is ambition, it is overreaching, it is discipline. And it is victory and defeat, which is all that the idiot sub-conscious really understands.”
He could just as easily been talking about the inner world of surveillance operations—its overreaching and frustration, its wins and losses. We’ve all been too bold and gotten burned, or played it cautious and lost our subject. But the very best of us have learned from those experiences and gone on to become solid, consistent players for our company team.
Will those in the surveillance industry give a flying screwball about any of this? I do not know. Will a vendor manager for B.I.G. Insurance Company seek out me and other “surveilmetricians” to bring deeper meaning and understanding to their surveillance statistics? Will they take my ideas to heart and learn to better vet and evaluate their surveillance vendors? I do not know.
Is there a chance I will be drafted by the Kansas City Royals or the Boston Red Sox? Statistically speaking, probably not.
End of Report
About the Author:
Steve Koenig has more than twenty-five years of experience investigating cases for insurance companies, attorneys, corporations, and private parties. He owns an investigations firm, Koenig Investigative Agency, in Nebraska, and is founder and past president of the Nebraska Association of Licensed Private Investigators. You can check Steve out on LinkedIn.
Steve Koenig and Ken Mitchell co-authored the quirky crime novel, They Call Her Ed, which can be found via paperback or on Kindle at Amazon.com. Any questions for Steve, contact him at Steve.PrivateEye1@gmail.com.