Beware the Inferential Leap From a Single Data Point

If you are trying to understand the complex underlying personal characteristics of your client, a potential employee, or even yourself, it goes without saying that you need a good tool (i.e., a good assessment) to measure these not-so-obvious personal features. This is the heart of psychometrics: the science of measuring individual differences (that is, our unique personal characteristics). Typically psychometrics involves gathering multiple responses related to a characteristic to ensure you’re really measuring what you say you’re measuring, and that the measurement is the same time after time.

Making assumptions or judgments about someone from a single question is tricky business (perhaps unless we’re talking about pain or satisfaction, where the smiley face scale is actually appropriate and does a good job of assessing those characteristics reliably). But let’s deal with things that are a little more complex.

What if you’re interested in understanding how you or your prospective client views status, that is, the way he views his success and how that success should be shared with others? Status is one of the attitudinal factors in Financial Perspectives, an assessment of financial attitudes and outlook. From a financial-planning perspective (and perhaps even investment-management perspective), understanding our attitudes towards displaying status to others can help prepare us for financial plans: status-seeking or status-displaying behaviors may have to take a backseat to other, more economically focused goals, depending on your net worth and retirement plans. As documented in the book Stop Acting Rich, often these displays of status are merely emulating what we believe is consistent with “rich,” but in reality all the while the car payment and the travel that requires a passport is eating away at our ability to be financially independent. Those who view status as something that should be displayed through what we drive, buy, and wear, and/or as a competition with the Joneses, may find it challenging to begin a path towards sustained economic success as the price of displaying that (perceived) status rises with every new model of gadget.

But back to the topic at hand. If you were interested in learning about your prospective client’s view of the importance of displaying status to others, you could ask him what kind of car he drives, and if that answer is something of the luxury persuasion, you might conclude that he wishes to transmit or broadcast his success through his vehicle purchases. In fact, you don’t even have to ask the question: you could just see what pulls into the parking lot when you’re meeting or check out his Instagram account. There’s plenty of information there from which to make an inferential leap. From this one piece of information, you might attribute all sorts of other characteristics to this individual.

But there is a problem with inferring personal characteristics from one piece of information. One car or one decision does not a millionaire next door make, nor does one decision necessarily indicate an under-accumulator of wealth. We’ve discussed this point in the past: judging a book by its proverbial cover works both ways. Just like cutting coupons doesn’t make us frugal if we go on $10,000 vacations that we have to carry on a credit card, our car is only one piece of our money-attitudes picture.

Instead, in order to get a deep and accurate picture of personal characteristics, we need information about a wide range of patterns of behaviors. This can be obtained through answers to multiple questions about status attitudes. We need to know why he drives what he drives, and what it means to him to demonstrate his net worth (or more likely his income level) to others through a car. We need to know whether status is something he’s willing to sacrifice in order to meet long-term financial goals, or if demonstrating success to others now is a predominant objective. This involves reliable and valid measurement, which is at the heart of good assessment design.

One piece of information just isn’t enough here, and advisors and clients would do well to consider a pattern of behaviors and/or a reliable measure of attitudes to better understand the complexities of characteristics. Achieving this deeper and more reliable understanding of the pattern of behaviors requires harvesting more data points–i.e., a longer test. The reality is: a more reliable measurement requires a little more time up front.

Speaking of time, maybe you’re thinking that one day soon self-report assessments can be replaced by AI, Big Data, social media, or some combination thereof. Indeed many financial services organizations are already working on these types of innovations. Some of the big companies with names you know are providing ways to not-quite-scrape-but-sort-of-scrape social media profiles so that their advisor/salespeople can contact you when they “learn” that you’re having a baby, or your kids are going to college, or you’re changing jobs. This area was hailed as the biggest “innovation” at one of the fintech conferences I attended last year, which seemed to me to be innovative only in being less than transparent. I’m thinking that Mark Zuckerberg’s testimony before Congress last week on issues of data privacy may have changed how “innovative” that feels now.

We’re continuing to monitor and think critically about this type of technology here at DataPoints. But sooner or later advisors will have to wrestle with the difficult question of how their clients may feel about this approach to learning about patterns of behavior.

Leave a Reply

Your email address will not be published. Required fields are marked *