Texas Valuation Professionals

Information ➡️ Knowledge ➡️ Wisdom

Our access to information has increased exponentially, but our access to wisdom seems to be more and more diminished.

Knowledge requires information, but is not the same thing as information. Similarly, wisdom requires knowledge, but is also distinct from that knowledge.

What are the links between them, and why is 'information' exploding while 'knowledge' and 'wisdom' seem to be in short supply?

I suspect the answers to that question are many, and I certainly don't have them all. I, however, will submit one set of answers: thought and contemplation.

For information to be transformed into knowledge, we must think about the information with which we have been presented. When we think about it we are structuring it in our minds: considering it, organizing it, and forming a story about it. Humans are natural story-tellers, after all. Gradually, that story becomes knowledge. The raw materials of information have been thus shaped into a tool to help us make sense of the world.

For that knowledge to become wisdom, we must move beyond thought to contemplation. We need to sit down and ponder that knowledge for a while, along with the raw materials from whence it came. It is slow, non-linear work, and we can't truly contemplate with an agenda. Why? Because, the transformation from knowledge to wisdom requires considering not just if we told the right story, but if it is a story worth telling. Is the tool we built going to help us build something good...something of real value?

It's unlikely that you've read this far and haven't had AI cross your mind. AI is amazing, but it can't actually create knowledge, and certainly not wisdom. It can present us with a shocking amount of information, seemingly well-refined raw materials. It can even potentially help guide us toward knowledge; but ultimately, we are the ones who have to do the thinking. We are the ones who have to slow down enough so that contemplation becomes possible.

Thinking and contemplation can't be replaced by greater and greater quantities of information. If we want to be knowledgeable and wise, at some point we must stop generating information so that we can take the time to think about it.

The implications for the valuation profession are many and varied, as they are in many other industries and professions. When was the last time we paused long enough to see how far we've come, and then contemplate where it is that we are headed (and if it is even a worthy destination)?

This actually isn't an anti-AI post, it is a pro-human post. A quick rejection of AI lacks just as much wisdom as a quick embrace of it. There is simply nothing quick about wisdom, and the appraisal industry (no less than the rest of the world) is in dire need of people who will make the effort to gain it.

This is the 4th post in a series where I'm exploring the field of causal inference as it relates to valuation. In the last post, I compared the common appraisal method of grouped pair analysis with the causal inference technique of Coarsened Exact Matching (CEM).

In this post, I'll explain how one of my recent techniques fits in. That technique is called the Layered Node Contrast Analysis (LNCA).

CEM vs. LNCA

Coarsened Exact Matching and Layered Node Contrast Analysis are both trying to create more credible comparisons from observational data; but, they are designed for somewhat different purposes.

Coarsened Exact Matching (CEM) places observations into defined ranges or categories for important characteristics. It then matches observations that share the same combination of categories and removes groups that do not contain observations on both sides of the feature being studied. The result is a matched sample intended to reduce imbalance before estimating the effect.

Layered Node Contrast Analysis (LNCA) is an appraisal-specific method for developing adjustment evidence. LNCA builds a separate analysis for each target characteristic (i.e., adjustment factor). It orders the other variables according to their relationship with that target and adds them as controls one layer at a time. Within each layer, LNCA compares matched groups, but it does not assume that matching on categories makes the groups fully equivalent. It also:
-measures the remaining imbalance between the contrasted groups;
-gives less weight to less-clean comparisons;
-corrects the price difference for measured imbalances; and
-reconciles the evidence across layers

So, both methods use coarsened matching and require overlap between the two sides of the comparison.
The primary difference is what happens after that:
CEM generally creates a matched sample under one selected matching design and then estimates the effect from that sample.
LNCA creates a sequence of increasingly controlled comparisons, corrects for remaining imbalance within those comparisons, and uses the combined evidence to develop an appraisal adjustment indication.

Neither method eliminates omitted-variable bias, market noise, or uncertainty. Those will continue to plague any and all methods!

CEM has the advantage of being an established causal-inference method with a formal matching framework.
LNCA is designed around a different practical problem: extracting and reconciling adjustment evidence from the type of limited, highly interrelated market data appraisers typically work with.

They share much of the same logic, but they organize and use the evidence differently.

This is post 3 exploring what the valuation profession can learn from the field of causal inference.

Grouped Pairs vs. Coarsened Exact Matching

Coarsened Exact Matching (CEM) is a matching method that places observations into defined ranges or categories for important characteristics, then compares only those groups that contain observations on both sides of the feature being studied.

In appraisal terms, properties might be grouped by defined ranges for GLA, age, lot size, quality, and location. Homes with and without the target feature are then compared only when they share the same combination of those categories.

Grouped paired-sales analysis and CEM are therefore trying to solve essentially the same problem:
-How can we compare properties with and without a feature while limiting the influence of their other differences?

Grouped pairs typically begin with two broader groups (such as homes with pools and homes without pools) and use filtering or judgment to improve comparability.

CEM begins by defining comparability more explicitly and applying those controls jointly.

Both methods:
-use multiple observations rather than relying on one pair;
-attempt to control confounding property differences;
-trade sample size for improved comparability;
-remain vulnerable to unobserved differences.

The primary distinction is one of structure:
Grouped pairs compare groups judged to be reasonably similar. CEM constructs matched groups under explicit and reproducible rules.

That does not automatically make CEM superior.

Broad categories preserve more observations but permit more residual differences. Narrow categories improve comparability but may leave too little support.

The value of CEM to appraisal may therefore be less about replacing grouped pairs and more about making several parts of the analysis explicit:
-What characteristics are being controlled?
-How similar is similar enough?
-Where does the market actually provide overlap?
-Which observations do not have a credible counterpart?

A larger group comparison is not automatically stronger evidence (relative to traditional paired sales). Its credibility depends on how convincingly the comparison approximates the missing counterfactual (see the prior posts in this series for a better understanding of 'counterfactual').

Next, I'll share a little more about a technique I'm working on which extends Coarsened Exact Matching concepts into the appraisal environment.

I'm continuing to explore the field of causal inference as it relates to appraisal. This time I'm focused on paired sales.

The ideal paired sale in theory is two properties that differ in only one meaningful characteristic.

That actually only solves one mathematical problem: isolation of the variable.

It would not solve the second problem: meaningful inference.

Suppose two otherwise identical homes sold, one with a pool and one without, and the price difference was $50,000. That would be a clean observation of the difference between those two transactions, but would it establish that the market typically pays $50,000 for a pool?

Not really.

It would still be only one observation, affected by the motivations, negotiations, exposure, information, and random pricing variation present in those two transactions: a unique human interaction which will never be repeated.

And there is the difficulty...without additional observations, we would not know whether $50,000 was near the center of the market reaction or an unusual result.

A paired sale is attempting to estimate an outcome that cannot be observed directly: What would the same property have sold for, at the same time, if the feature had been different?

In causal inference, that missing alternative outcome is called the counterfactual.

Paired-sales analysis therefore faces two separate questions:
1. Can we construct a credible substitute for the missing counterfactual?
2. Do we have enough evidence to infer a broader market reaction from the observed comparison?

A strong pair may answer the first question reasonably well, but it does not, by itself, answer the second.

Grouped pair analysis attempts to add context by introducing more observations. It might solve the context problem, but at what cost?

I've been thinking lately about the field of Causal Inference.

Causal inference is the field concerned with determining whether (and to what extent) an observed outcome was caused by a particular treatment, condition, or event.

Appraisers observe the outcome of price, and much of the analytical techniques we employ are focused on understanding what caused that particular price. When we understand causation, we can better predict potential pricing outcomes for a subject property.

The language differs somewhat, but the concepts overlap a great deal. For example, appraisers wouldn't often use the term 'counterfactual', but we use the concept all the time. A counterfactual is the outcome that would have occurred under an alternative condition that did not actually happen.

💡 That pretty much describes the whole concept of adjustments in a sales comparison approach.

I'm going to explore this a little more in my next posts...

I wanted to share an image which I'm going to use to begin the AVS Tech Talk on July 21st.

We are going to talk about models, specifically as they pertain to adjustment support. You may not think of 'model' as the term you'd most associate with sales comparison analysis techniques, but a model is just a representation of something else, often something which can't be directly observed.

Let's consider Leonardo da Vinci's 'The Vitruvian Man':
In the illustration, The Vitruvian Man represents the reality which can't be directly observed. Observation can only take place as mediated by data. Data is just the artifact that reality leaves behind.

We can't forget that we don't try to model the data, we try to model the REALITY.

Again, looking at the illustration if you didn't have the first frame to look at you might be able to overlay a model which looks like a bird or a turtle. It is possible that those model selections might even match a given set of data better than any other model, but they would still give an entirely wrong impression of reality.

Of course, we must also recognize that the best model will always reflect a somewhat distorted view of reality. I like The Vitruvian Man as something to contemplate for this. The Vitruvian Man given an impression of a man who can and does MOVE. The reality that appraisers deal with, like da Vinci's artwork, is constantly in motion. The data too is always shifting...

I hope this teaser of the upcoming discussion has piqued your curiosity. If so, join us!

The American Valuation Society has been putting on these Tech Talks for free, so follow the link if you'd like to register.

https://us06web.zoom.us/webinar/register/2917824897191/WN_a3IEpgjWT2ukVEk5MDZTrA #/registration

With the addition of discrete weighting labels (most, less, or no weight) in UAD 3.6, I think we need to step back and look at what it means to reconcile ADJUSTED sales prices.

I think that the default setting for appraisers, users of appraisals, and the makers of appraisal policy has been to confuse the recognition of the UNADJUSTED similarity of a sale with how to treat the ADJUSTED price of that sale in the reconciliation.

The unadjusted and adjusted prices reflect two fundamentally different bundles of property attributes. The unadjusted bundle is that represented by the comparable sale, and the adjusted bundle is that represented by the subject. That is hugely important to recognize before moving into reconciliation.

In short, the adjusted sales prices are no longer supposed to encode dissimilarity. The whole point of quantitative analysis is to remove the price impact of those dissimilarities before reconciliation.

So, the question becomes: How should we reconcile? That depends largely upon the definition of market value.

If the definition of market value includes the word "probable", then you must explicitly deal with probability within the appraisal. There is no better place to do that than the reconciliation.

Reconciliation in Quantitative Analysis This is a brief discussion of the role of sales comparison reconciliation. This video challenges the conventional narrative around reconciliation.

Epistemic Closure and the Dunning-Kruger Effect

I've written about the Dunning-Kruger effect before, but I've noticed something which I'm actually hesitant to point out...epistemic closure.

Dunning-Kruger called the first peak of overconfidence "Mount Stupid". Ouch...I've been there and have the T-shirt.

I'm calling the second peak "Epistemic Closure". This phenomenon describes the blindness to knowledge which you don't already possess. What I'm referring to is not knowledge in general, but rather to a depth of knowledge in a particular area. This is similar to the concept of the Overton Window, where one is blind to knowledge outside of ones window of perception regarding what is possible.

The person at the peak of Mount Stupid confidently declares "I'm GOOD at this!"
..and later hopes that nobody was around to hear it.

The person at the peak of Epistemic Closure confidently declares "I'm an EXPERT at this!"
..and then posts it on social media.

Of course I'm hesitant to point this out, because I'm still trying to get this log out of my own eye. The stumble off of the mountain of Epistemic Closure can be a painful one. I've been there more times than I care to admit and have the bruises to prove it.

I suspect that the peak of Epistemic Closure is a recurring problem for all of us. There will always be the temptation to believe that I have arrived, only to find myself at another false summit.

Of course, this concept has application well beyond valuation, but for those in an industry where you are paid for your opinion, the temptation is real.

May we all learn to treat the path toward toward expertise as a perpetual journey and not a peak upon which to plant our flag. To do that we must only allow confidence to increase in direct proportion to our humility.

Since I wrote this article I heard a quote which I've been thinking about a lot. I think it speaks to much of what I was trying to explain in the article.

“The poet's eye, in a fine frenzy rolling,
Doth glance from heaven to earth, from earth to heaven;
And as imagination bodies forth
The forms of things unknown, the poet's pen
Turns them to shapes and gives to airy nothing
A local habitation and a name.”

This line from Shakespeare's "A Midsummer Night's Dream" speaks to something which I've been trying to articulate for some time. According to Shakespeare, the poetic eye is what can bring shape to that which can be apprehended, but never fully comprehended.

Valuation analysts likewise must build models based on what has been apprehended (perceived), but can never build a model which will contain comprehension (understanding). In that way appraisers are like poets. Real estate data is messy and complex, and the human behavior encoded within that data is also messy and complex, but often in different ways. The appraiser must apprehend markets which can never be fully comprehended. The appraiser must then turn them "to shapes" and give to the "airy nothing" of human behavior "a local habitation and a name". That local habitation and name is market value.

So, our models (whether an entire approach to value or a sub-problem within an approach such as adjustment support) must be built as 'perception-containers' which yield insights, not answer generators which yield conclusions.

We cannot ask our models to build themselves from messy and complex data. Mathematical models behave like gases; they expand to the shape of the container in which they are placed. The valuation analyst must first perceive and only then build a container for the math to fill.

What does that mean in practice?
➡️ You must choose the method which fits your perception of the market, not just develop your perception of the market from the results of a method.
➡️ If you don't know how to guide it, AI will select a method for you, and it won't be based on your perception of the market either.

We can't outsource our "poet's eye" or our models will be meaningless.
We can't outsource our "poet's pen" or our conclusions will be misleading.

Curiosity in the Age of AI Artificial Intelligence (AI) will never be your most powerful tool for real estate appraisal.   With all of the rapid development of AI, you may think that assertion is bold. Maybe you have just begun experimenting with it and are amazed. Maybe you are well beyond experimentation and have