Even though picking and choosing which data points to consider and which to ignore may be considered borderline to manufacturing data, it is often common practice to assume an equal percentile of data points at the top and bottom of a sample to be outliers, i.e. anomalies, not representative of the true distribution of the variable. Consistent with this assumption, such data points are thrown out of the sample. In Figure 3 (shown below) we experiment with the 5th and 95th percentiles of the sample. This definition of outliers causes the range to be bounded between 9.57 (5th percentile) and 30.23 (95th percentile). Consequently, the whole 2000 tech bubble, as well as some of the lowest values recorded during the depressions of the 1920s and 1930s, are ignored.
This approach implicitly assumes that all the relevant information that can be extrapolated from the P/E 10 has to do with its relative position within the 5th-95th percentile range, and that any values outside of this range do not provide any additional information. For instance, if the P/E 10 decreases from 15 to 10, this may be considered a move from undervaluation to extreme undervaluation. However, if the P/E 10 decreases further from 10 to 5, the market will simply remain extremely undervalued.
Perhaps the most important effect of capping a distribution at given percentiles and eliminating outliers is the one of bringing the average, median and mid-range of the capped distribution closer to each other. In our example, the median of the capped P/E 10 distribution remained the same at 15.85, and so did the percentile of the most recent reading of 25.90. However, the average decreased from 18.06 to 17.69 and the mid-range decreased from 26.70 to 19.90. In this framework, the most recent P/E 10 of 25.90 is still 63.45% above its historical median of 15.85 and still ranks at about the 90th percentile of the historical sample. However, the same P/E 10 now appears slightly more overvalued when compared to the average (from 43.40% overvalued to 46.38% overvalued) and 30.17% overvalued when compared to its mid-range (from 2.99% undervalued).
We can therefore conclude that, when we contemplate the presence of a certain degree of outliers in the historical distribution, the current level of the P/E 10 indicates a consistent high level of overvaluation regardless of whether it is compared to its long-term average, median, ranking, or mid-range.