"Lies Damn Lies and Statistics"

Written by David Braun, MAI, SRA

You're using statistics whether you know it or not.

This article considers the advantages and disadvantages of some of the analysis methods available to trend values over time. It focuses on the method in which the unadjusted sale prices are trended by use of a scatter plot and linear trend-line. After reading this article, click on (or copy to browser) the following link to see the associated video and data set: http://bit.ly/q98OVE.

The volume of data, rather than the technique used, establishes a statistical analysis. Three sales in an adjustment grid is not considered a statistical analysis, but put 30 or more sales on that same grid and the analysis would be considered statistical in nature. Statistical analysis has been used in the mass appraisal of real property for more than a quarter of a century (typically assessor offices). By 2005 most appraisers could search their MLS, tax records, or private database for comparables and then export them directly to a spreadsheet. This opened the door to the possibility of appraisers using statistics in their everyday appraisal work. Prior to this, many commercial appraisers had been delving into statistical analysis when they performed a market condition analysis; and when they would trend various data by plotting it on a scatter chart and inserting a trend-line. While appraisers continue to perform these analyses, the Fannie Mae requirement to fill out the 1004MC form pushed thousands of residential appraisers to perform statistical analysis on a daily basis. Many appraisers have been meeting this challenge by improving their ability to use spreadsheet applications.

Now that they have the ability to transfer and manipulate large amounts of sales data, appraisers are searching for new analysis methods. Many tried and true statistical techniques are applicable to the analysis of real property. This overlap is depicted in the illustration below:

Consider that an appraisal consists of a series of opinions and conclusions formed by the appraiser. These are generated to help the intended user make a decision. An examination of the Uniform Residential Appraisal Report (URAR) including the 1004MC form revealed around 30 opinions or conclusions that could be made with statistical analysis techniques. We will examine one of these conclusions: the trend in property values over time. This conclusion is specifically requested on most residential forms, and is a necessary part of a thorough market analysis for residential or commercial markets.

Sally, a local residential appraiser, is considering the best technique to determine the trend in value for the subjects market area. In this area, properties are very similar in terms of location, style, design, etc. but have some variance in the gross living area (1,600 to 2,300 square feet); have one to four bedrooms, one to three baths, and are 10 to 20 years old. Sally found 50 sales (to view this data: http://bit.ly/q98OVE) that have occurred in the past 12 months. Four of these represent the sale and resale of the same two properties. Here are a few recognized techniques to form an opinion or conclusion of the trend in values over time. 1. Examine the sale and resale of properties in the market area (paired sales

analysis). 2. Trend the adjusted sale prices of properties in the market over time (where everything is adjusted for except market conditions) by using a linear trend-line on a scatter plot. 3. Trend the unadjusted sale prices of properties in the market over time by using a linear trend-line on a scatter plot. Questions: Which of these three methods should Sally choose? What is the reasoning for that choice? What is the main drawback to the best method? What statistical technique could Sally employ when performing the best method? First lets consider what we mean by best. A method would be considered the best when it is both effective and efficient. Effective means that it leads the appraiser to the correct conclusion. Efficient considers the time it takes to perform the analysis. A method must be effective or its use may result in a misleading conclusion. This would deem the analysis method as not being credible. Sale/Resale Method The problem with method one is that it is typically ineffective. This is partially because there are typically not enough properties that have sold and resold in a specific market. A statistician might refer to this as having an insufficient number of samples or observations. In addition, there is something called random variance, meaning properties do not typically sell at their intrinsic values. They typically sell in some range of plus or minus percent of their intrinsic values based the subjective motivations of human beings. If the random variance is 5 percent then a property worth $200,000 is expected to sell in a range of $190,000 to $210,000. In a stable market the property might happen to sell for $195,000 and resell a year later for $205,000 based on the expected random variance. These sales prices indicate an appreciation rate of 5.13 percent (($205,000 - $195,000) / $195,000). However, the

indicated value change represents random variance, not changes in real property values. Unfortunately, many appraisers start with a preconceived idea of what the value trend is and then only report the sale and resale of a property when it confirms their preconception. This is similar to how a drunk uses a lamppost: more for support than illumination. Trending the Adjusted Sale Prices Method The 50 sales could all be presented on a sales grid and be adjusted for everything (GLA, bedrooms, baths, and age) except the market conditions (time adjustment). This would typically take some extra time, but some appraisers are able to auto-load sales directly from their MLS onto the sales grid. Modern appraisal form software packages will also apply an adjustment to all 50 sales at one time. This method could be possible with this type of automation. The partially adjusted sales prices and their associated dates would then be plotted in a scatter chart and a linear trend-line would be inserted. Lets consider the effectiveness of this analysis method. The adjustment amounts are critical. If the line-item adjustment rates are applied at the wrong rate, then the analysis will be amiss. Even if the proper adjustments are made, each adjusted sale will have some amount of random variance. This can sometimes distort the actual value trend. Trending the Unadjusted Sale Prices This method is becoming popular among appraisers when there are many sales to work with (typically more than 30 sales). This method is very efficient as it does not take long to search for sales in a market, download the sale prices and dates of sale, scrub the data and then plot them on a scatter chart. The resulting chart appears to be very defendable. To get a feel of its effectiveness, lets test the procedure by applying it to the data set (Sallys list of sales).

The Test While many spreadsheets are available and effective for appraisers to use, I will be referring to Microsoft Excel. The following specific Excel features will be utilized in testing this method of analysis: the Forecast function the Correlation function the Randbetween function the scatter chart a linear trend-line macros a Monte Carlo simulation

The test is devised as follows: 1. Develop a hypothetical residential market where the value trend is stable. 2. Set thresholds for the related correlation, range of the property components and acceptable error in the percentage change in value. 3. Set the ranges of values that the random numbers can change (see Table below). 4. Run a Monte Carlo simulation which repeatedly and randomly changes the numbers for the applicable property components (variables) and their values (coefficients) many times. 5. Record the percentage change in value during the period in question for each simulation. 6. Determine the validity of the technique based on the number of times the analysis produced an answer greater than the acceptable error in the percentage change in value.

It is important to understand that the sales used are from a virtual market. This market was constructed as a stable market. A stable market means that there is no relationship between the sale dates and the values. See the associated video for more information on this. Analysis of Trending Values over Time In the chart below sale prices over time are trended for the 50 sales. It clearly indicates a market where values are increasing.

This result is disturbing as the model was designed so that the market is stable. How can trending the sale prices over time indicate values are appreciating when the market is really stable? "There are three kinds of lies: lies, damn lies, and statistics." -Mark Twain

In this case the answer is correlation. The following correlation matrix shows that there is a 0.8 correlation coefficient between the sale date and the gross living area. The largest degree of correlation is 1.0 (or -1.0).

A review of the market data clearly shows that the more recent sales just happened to be larger homes. In this case the chart: Analysis of Trending Values Over Time is actually reflecting that the houses sold tended to be larger over time and therefore higher-priced. This fusing of the influence of gross living area onto the sales date is an example of multicollinearity. Multicollinearity results when the correlation between two or more property characteristics distorts the related individual variable coefficients.

Results of the Monte Carlo Simulation In each simulation the variables (property components) and the coefficient (value of each component) were randomly changed within the range limits presented in the One-Unit Housing Trends chart. Based on these random numbers, the sale prices recalculated with each simulation. The forecasted change in values for the beginning and end of the time period were monitored. The trending method was considered to give a false indication when the trend in value exceeded the testing thresholds of +/3 percent.

The test consists of 10,000 individual simulations run by a macro. The results of these simulations demonstrate that when the market is stable, trending sale prices over time will indicate an increasing or decreasing trend in values 72.5 percent of the time. In addition, it will provide an incorrect conclusion 30.6 percent of the time even when none of the property characteristics has a correlation coefficient to the sale dates greater than 0.2. These results indicate that trending the unadjusted sale prices over time for this data set is not an effective method of forming an opinion of the trend in value over time. Answers to the questions: Which of these three methods should Sally choose? Answer: Method 2: Trending the adjusted sale prices over time. What is the reasoning for that choice? Answer: Choices 1 and 3 analysis of the sale and resale and trending the unadjusted sales prices over time are not reliable, therefore not effective. What are the main drawbacks to the best method (trending the adjusted sale prices over time)? Answer: The line-item adjustments must be correct and random variance can distort the results. What statistical technique could Sally employ when performing the best method? The answer to this question was not presented in the material. The answer is multiple linear regression analysis. Regression analysis identifies the correct line-item adjustments and does a good job of dealing with the random variance that is inherent in real property markets. It uses a statistical process to calculate the P-factor, which identifies the likelihood that there is no linear relationship between the sale dates and the sale prices. Some regression programs can also indicate if there is a curvilinear relationship between the sale dates and sale prices. If there is no linear and no curvilinear relationship, then by definition the market is stable. If there is a

linear relationship then the related coefficient will indicate the amount and direction of the trend in values. If a curvilinear relationship exists then dummy fields can be created to identify and measure the nonlinear curve. See the associated video (http://bit.ly/q98OVE ) for more information on regression analysis.

Conclusion This article identified some of the weaknesses associated with traditional and statistical analysis techniques used to trend values over time. It revealed that there are many conclusions and opinions in an appraisal that can be formed by applying statistical analysis. When should appraisers begin using statistical analysis? Well, many appraisers are unknowingly applying statistical techniques in their appraisals today. For example, the Excel spreadsheet places the linear trend-line on a scatter plot via the sum of least squares method. By definition, this is a regression analysis. Appraisers are employing multiple statistical analyses every time they fill out a 1004MC form. This article and video presents many statistical terms and concepts with which appraisers should become familiar. Appraisers are not expected to be able to perform Monte Carlo simulations or the macros that run it. Click on (or copy to browser) the following link in order to view the associated video and data sample: http://bit.ly/q98OVE.