Chapter 4

PREDICTOR TESTING METHODS

The Predictors
Parallax has been producing neural-network based financial predictors since 1990
and so it is an integral part of our business to validate these predictors using
reliable statistical methods. With each predictor, we need to answer the following
set of questions:
1. What price behavior is being predicted?
2. What is the effective duration of the predictor?
3. How statistically significant is the predictor at each time step forward?
4. Is the predictor effective on all time scales?
5. Is the predictor effective on all financial series?
6. What combinations of predictors are the most effective?
The following section is an overview of a simple and reliable statistical method
and an 8-year analysis of our present stable of predictors. In order to carry out
this analysis, it is necessary to measure the price action during the time period
immediately following each prediction event. We call this the post-predictor or
outrun period.
74

Post-Predictor Z Scores
We are interested in characterizing the post-predictor time period using a scaleindependent method that allows comparisons between financial series. The Z
Score is the most appropriate measure for this job. A Z-Score is the measure of
how many standard deviations price has moved away from its price at the
prediction event, assuming that the probability of either an up or down move is
random at 50%.
By measuring local volatility at the prediction event, a normal probability
distribution can be drawn going forward in time that acts as a roadmap for
subsequent price moves.
prediction event.

The map is centered at the closing price of the

Each day the map widens according to normal diffusion

velocities, which are proportional to 1/Time, representing the region where the
future price is most likely to be found. An example of such a probability map is
shown below for the stock Home Depot on Feb 3, 2005:

75

Viewed from the side at two time steps, diffusion acts to spread out the region
where we might find the stock price as time elapses:

Diffusion causes
expected prices to
spread out over
time

At each time step, there is a larger standard deviation and the same mean. If we
represent the actual price achieved at each step in terms of that standard
deviation, we produce a series of Z-Scores. For example, if the price at a
prediction event is $5, and then moves to $7 on day ten with a standard deviation
of $1.60, then the Z-Score on day ten would be (7-5)/1.6 = 1.25. This means
that price moved 1.25 standard deviations above the price at the prediction event.
Since the standard deviation continues to increase, in order to maintain the same
Z, price would have to increase by the same relative amount.

76

If all post-predictor prices for all financial series are converted to Z-Scores, and if
the predictors do not work, and if markets are random, then at every time step, a
histogram of all the Z-Scores would be expected to result in a standardized
normal distribution, with mean of zero and a standard deviation of one N(0,1) as
shown below:

We of course hope that our predictors actually predict non-random price

behavior, so the degree of deviation from the normal curve is critical.

Are Financial Series Random?

We have assumed that price movement is best characterized by a random walk
model, but this might not be the case. There is gathering evidence for other
characteristic distributions such as a Biased Random Walk, Truncated Levy
Flights, or Cauchy distribution.

Our solution is to produce a quantitative

background distribution based on randomly selected dates across our entire 3000
day test period, and for all of the 2500 stocks being considered. The figure below
77

shows this background distribution of Z-scores corresponding to random

prediction dates, and a normal distribution based on a random walk assumption
plotted together. It is clear from this figure that a strict random walk assumption
is inappropriate during our 8 year test period.

Instead there appears to be a

positive return bias.

To illustrate this another way, we could ask what percentage of buy predictions
are winners (Z>0) if the timing is random, and plot this percentage each day
following the randomly selected purchase dates. Normally this would be 50%,
but since the background distributions is positively biased, the figure below
shows the percent winners for randomly selected buys climbs steadily over time.
The reverse is of course true for randomly selected sells (mirror image).

78

On a weekly scale the effect is even more pronounced, as shown in the next
figure. I guess this could be called the dartboard effect, in that even a random
selection of stocks during this period showed a 59% win rate at 30 weeks after
purchase, and shorting was decidedly unprofitable.

79

We will use these curves as benchmarks for our predictors over this test period
and stock set. In order to have a non-random buy predictor then, we need to see
a win rate in excess of 59% at 30 weeks for example, and sells would need a win
rate better than 41%.
So far we have examined the background distribution of Z-Scores and the
percentage of winners. What we still need to know is how much gain is possible
from randomly selected buys and sells, so that each predictors excess gain profile
can be evaluated. The pictures below show the background performance for
randomly selected buys and sells:

80

81

Parallax Predictors
Using the randomly selected background distribution we found in the last section,
we will now statistically evaluate our main stable of predictors using a chi-squared
test. The Chi-squared test is designed for comparing distributions and a p<0.01
constitutes a rejection of our predictor distributions as random. We will also look
at the excess %winners and excess %gain versus the random benchmarks shown
in the last figure. The following is a list and a qualitative description of each
predictor:
Predictor

Description

Duration

ExtremeHurst

Measures the degree of positive feedback present in price advances and

predicts when the critical point has been reached. Price behavior in the
post-predictor period is expected to be flat to down. This is an end-oftrend predictor.

ExtremeHurst

Measures the degree of positive feedback present in price declines and

predicts when the critical point has been reached. Price behavior in the
post-predictor period is expected to be flat to up. This is an end-oftrend predictor.

Estimated at 2030 bars bars can

be any scale
provided
sufficient liquidity
Estimated at 2030 bars bars can
be any scale
provided
sufficient liquidity
Estimated at 2030 bars bars can
be any scale
provided
sufficient liquidity
Estimated at 6
months to a year

Extension (high)

Extension (low)

ExtremeHurst

Compression

Price Wizard
(Undervalued)

Price Wizard

(Overvalued)

Precision Turn

(high)

Precision Turn

(low)

Measures the degree of negative feedback present in price and predicts

when the critical point has been reached. Price behavior in the postpredictor period is expected to be characterized by a significant
volatility increase and high trend persistence. This is a new trend
predictor.
Individual stock prices are found using neural networks by
simultaneously weighing all major stores of fundamental value in the
context of industry, sector, and economy. If the determined price is
less than the actual price, then the stock is undervalued and considered
a buy
Individual stock prices are found using neural networks by
simultaneously weighing all major stores of fundamental value in the
context of industry, sector, and economy. If the determined price is
more than the actual price, then the stock is overvalued and considered
a sell
Individual trend change cycle dates are found using a method which is
blind to high/low, but sensitive to the degree of trend change. Sells are
determined using a local Hurst measure to confirm an uptrend into the
turn date
Individual trend change cycle dates are found using a method which is
blind to high/low, but sensitive to the degree of trend change. Buys
are determined using a local Hurst measure to confirm an downtrend
into the turn date

82

Estimated at 6
months to a year

Estimated at 6
days
Estimated at 6
days

ExtremeHurst Extension Buy & Sell Predictors

Our ExtremeHurst extension predictor is Parallaxs most predictive tool. It
measures the degree of positive feedback present in price and predicts when the
end-of-trend critical point has been reached using the principle of discrete scale
invariance. Price behavior in the post-predictor period is expected to be flat or
opposite of the preceding trend. On a daily scale, we have accumulated Z-Scores
for all 30 days following the signal. The distribution of these Z-Scores on day
eleven, shown in the figure below, shows the strong positive bias, with 60% of
the returns being positive by day eleven, with a p-value = 0.00000005. This
means that these are not random signals, and that market behavior leading up to
the prediction event conditioned what happened in the post-prediction period.
ExtremeHurst is scale invariant by definition, so this same effect should be
present on weekly, monthly, or intra-day periods as well.

83

The ExtremeHurst sell predictor is extremely fast, showing 56% winners at only
4 days from the signal, with a Chi squared test p-value = 6.7x10-124. This means
that these are not random signals, and that market behavior leading up to the
prediction event significantly conditioned what happened in the postprediction period.

ExtremeHurst is scale invariant by definition, so this same

effect should be present on weekly, monthly, or intra-day periods as

well.

84

Looking at the percentage of winning predictions for both ExtremeHurst

predictors compared to the background rate yields the following picture:

The way to interpret this picture is that we gain a significant advantage over
random for both buy and sell predictors for at least 30 days. Extension buy
signals yield the best percentage of winning predictions at 8% more than the
random background after 22 days. This tells us that if we had a large sample of
trades that were triggered by this predictor, we could expect that at 22 days, a
total of 55%+8% = 63% of the trades would be ahead. Remember that our
random trade sample had 55% winners at 22 days, so we would beat it by 8%.
This chart also tells us that we can hold trades for a relatively long time. Despite
the win rate staying constant, average portfolio returns would climb steadily.
85

The sell signal is different and faster. We immediately achieve a 7% edge above
random at 3 days and that edge diminishes steadily over 30 days. This might
make us design a faster sell trading strategy than what we would use for the long
side.
So far we have examined the overall distribution of Z-Scores for ExtremeHurst,
and the percentage of times that a buy signal goes up or a sell goes down. The
win rate is only part of the story however. What we still need to know is how
much gain is possible from these buys and sells in excess of the background
market performance. The pictures below show a gain analysis for ExtremeHurst
buys and sells. Each plot shows the median percentage gain with error bars at the
40th and 60th percentiles and the average of all returns. Note that the average is
not as good a measure of performance because of the overemphasis on outlier
signals. The count of signals is shown in parenthesis.

86

87

On a weekly scale, the power of our ExtremeHurst extensions becomes even

more evident. Remember that 0% on the graph is equivalent to a random win
rate, so any excursion above zero stacks the deck in our favor. Weekly scale
extension buy predictions reach their maximum effect after 8 weeks, with an
overall win rate that is 15% over the random background. Again we see that the
extension sell signals are faster and give only a 2.5% edge over random.

88

89

Price Wizard Fundamental Value Predictor

Price is often out of sync with value, being bid up or down based on future
expectations and irrational trend persistence. Our model incorporates all major
stores of corporate value, normalized simultaneously for sector, industry, and
economic factors. The figure below shows the Z-Scores 30 weeks following a
stock moving to undervalued from overvalued.
positive bias.

..

90

Note the significant (p=0)

91

92

93

Precision Turn Cycle Predictor

..

94

95

96