Step by Step Testing Pattern : Analyzing the Results

TESTING THE THIRD DAY

If you remember, the first step we took to gain a better understanding of the morning star pattern was to look at the results of testing the pattern with several variations of the definition in the requirements for the third day. Using that as our starting point, I reviewed a number of the initial testing charts and noticed many of the evening star patterns had third days much lower than the first day of the pattern. I call these “low3” and an example is shown in Figure 7.4.

FIGURE 7.4: LOW3 EVENING STAR IN TNH ON 04/10/ 07

LOW3 TEST

Figure 7.4 shows that TNH was in an uptrend and formed a large white body on 04/05, followed by a gap up with a small body, which then was followed by a black body that closed below the midpoint of the first day of the pattern. It is a low3 because the third day of this pattern closed well below the first day of the pattern, which is within the definition of most evening stars but not the type of example that is shown in most texts.

Since the low3 style of evening star looks different than what is usually shown for a evening star pattern, I tested to see what the effect would be of requiring that day three have a close greater than the low of the first day of the pattern. This requirement resulted in patterns that looked more like what you often see in books and magazines, but it actually hurt the annualized ROI numbers. It appears better to allow the bar of the third day of the pattern to extend below the bar of the first day.

Results can be improved by requiring that the third day of the pattern have a body length that is the largest body length in the last five days. An example of this type of pattern is shown in Figure 7.5, where TIE showed an evening star with a long body on the third day of the pattern. During the next few days after the evening star formed, TIE dropped about 17%, leading to a nice profit for short positions.

FIGURE 7.5: LONG DAY 3 BODY EVENING STAR IN TIE ON 05/12/06

Taking only long-bodied evening stars during the 01/03/06 to 05/01/07 test period resulted in improved annualized ROI and the percentage of winning trades. This variation in the requirements for the third day of the evening star pattern did not improve results enough to beat buy and hold, but perhaps when combined with other parameter changes, this could result in a more interesting trading pattern.

GAPS

A number of traders feel the evening star pattern is stronger if the third day of the three day pattern opens with a gap down. Adding a filter requiring the open on the third to be less than the close of the second day of the pattern resulted in a small increase in ROI from negative 18% to negative 15.7%, and left the winning trade percentage about the same. Neither of these results is significant enough to warrant changing the basic pattern parameters of the evening star trading pattern.

The gap tested is a basic gap in which the open is lower than the previous day’s close. This type of gap is relatively common, and, as we have seen, is not helpful in terms of improving the trading results of the evening star pattern. When the gap is larger and the high falls below the low of the previous day’s trading, it forms a “white space” on the chart and is much more visible than the typical gap. An example of a “white space gap” is shown in Figure 7.6.

FIGURE 7.6: WHITE SPACE GAP ON DAY 3 OF EVENING STAR IN BTU ON 02/07/06

In Figure 7.6, BTU has been in an uptrend and forms an evening star pattern, which completes on 02/07/06. The third day of the evening star pattern starts with a strong gap down, and BTU opens below the close of the previous day. When a stock opens below the previous day’s low, it results in a “white space gap.”

Only taking evening star patterns with white space gaps on the third day of the pattern during the 01/03/06 to 05/01/07 test period, results in 312 trades showing an annualized ROI of slightly less than 8%; and more than 51% winning trades as shown in Figure 7.7. These results are not strong enough to use the pattern for trading, but do indicate the power of white space gaps of the third day since this is the first parameter change that has resulted in a positive annualized ROI.

FIGURE7.7: TSET RESULTS FOR WHITE SPACE GAP ON DAY 3 EVENING STARS

TESTING THE SECOND DAY

WHITE SPACE GAPS

One might think that if a white space gap on day three of the pattern helped the trading results more than anything else so far, then perhaps using a white space gap on day two instead of just a simple gap would also improve results. Figure 7.8 shows the test results for trading only evening star patterns with white space gaps on day two of the pattern. This parameter change resulted in significantly fewer trades and a larger annualized ROI loss than the basic definition with which we started. Our testing showed that white space gaps improve results when they occur on day three of the pattern and reduce results when they occur on day two of the pattern.

FIGURE 7.8: TEST RESULTS FOR WHITE SPACE GAP ON DAY 2 EVENING STARS

If the requirement that day two of the pattern show a gap up is eliminated, the test results were similar to the initial evening star test results shown in Figure 7.3, with the exception that, as expected, a lot more trades occurred. In this particular test period, it appears a gap on the third day of the pattern is more important than a gap on the first day, and a white space gap on the third day is more powerful that just a simple gap.

In addition to the requirement for a gap up, the basic definition of the evening star pattern requires a small body on day two, and allows it to be either black or white. Testing the performance of the evening star pattern with variations in these parameters will help us to understand how important they are.

BODY SIZE

Most of the literature on evening star patterns does not attach a specific definition to “small” body size on day two of the pattern. The basic definition for a small body on day two used for the results in Figure 7.3 was the body size must be less than 60% of the day one body size. Table 7.1 indicated the test results were similar for body sizes 40% or more of the first day’s body size. The results made an obvious improvement when the requirement was changed to 30% or less.

TABLE 7.1 EFFECT OF DAY 2 BODY SIZE ON EVENING STAR TEST RESULTS 01/03/06 TO 05/01/07

Remember, there are a number of different issues that can cause effects in backtesting results. This is why I am not interested in changing a parameter unless there is a clear difference in using it. In the first five lines of Table 7.1, the annualized ROI results only vary by about 4%. The effects of time frames, database quality, slippage, etc., can also account for a few percentage points difference. When the body size for day two drops to 30% or less of the day one body size, the annualized ROI loss is halved; that kind of result is something worth listening to.

Since changing the definition of “small body” on day two of the pattern to less than 30% of the day one body size cuts the annualized ROI loss in half and still results in over 1,700 trades during the 01/03/06 to 05/01/07 test period, I will be incorporating this into the standard definition and using it in subsequent tests. I have not yet incorporated the requirement for a day three white space gap down into the definition because it significantly reduced the number of trades.

BODY COLOR

The other parameter of the second day is that the body color does not matter; it can be either black or white. This idea was tested by adding a filter to the evening star definition that resulted in only taking trades when the body on the second day of the pattern was white. The results were interesting because the annualized ROI loss dropped from minus 18% to slightly under minus 6%. This effect also held up when testing in the longer time frame of 01/02/04 to 05/01/07.

I noticed an interesting effect when testing white body candlesticks on day two of the pattern that may affect some traders. When charting programs draw candlesticks, the horizontal lines used to draw the top and bottom of the candlestick body have a width. Because of the width of these lines, some very small body candlestick patterns appear to be black body candlesticks due to the very small amount of white space in the body of the candlestick. This is something worth keeping in mind.

As an example, Figure 7.9 shows an evening star pattern that occurred in ESL on 05/08/06, where the third day of the pattern is marked by an up arrow. The second day appears to be a black body in the chart. Closer inspection reveals that the open was 45.88 and the close was 46.28, indicating the stock had an up day and a white candlestick. The candlestick just appears to be black because the white space between the open and close is small compared to the line width used to draw the top and bottom parts of the candlestick body. This should not be an issue when using a computer scan to find the pattern, but it can lead traders to skip over some white-bodied candlesticks when visually scanning charts.

FIGURE 7.9: EVENING STAR IN ESL ON 05/08/06

TESTING THE FIRST DAY - WHAT IS BIG

The first day of the evening star pattern has the simplest definition; it just has to be a “big” black candlestick. The definition of a black candlestick is unambiguous. The definition of “big” leaves room for traders to interpret the word differently and thus trade different patterns. In order to get a better idea of what definition to use, the test was run several times using various definitions for “big,” with the results listed in Table 7.2.

TABLE 7.2 EFFECT OF DAY 1 BODY SIZE ON EVENING STAR TEST RESULTS 01/03/06 TO 05/01/07

To get the results for Table 7.2, the test was run six times during 01/03/06 to 05/01/07. The first line shows the results for taking all evening star trades during the period when the first day had a larger body than the previous day. The second line shows the results for evening star patterns whose first day body was larger than the bodies on the two previous days and so on to the last line, where the first day of the evening star pattern was required to have a body larger than any of the previous six days.

The third line in Table 7.2 has the best results, and in fact turns a losing system into a winning percentage. The 3% gain is still below the 14% annualized ROI for buy and hold, so this is not yet a pattern I would focus on trading, but we are definitely beginning to understand how to define the different parameters in the pattern. 

Based on the results of Table 7.2, I altered the parameters of the first day of the evening star pattern to require that the black body of the first day be larger than any body of the three previous candlesticks. An example of what this looks like is shown in Figure 7.10, in which an evening star pattern appeared in RGLD on 12/29/06. Note that the body on the first day of the three day pattern was the largest body in the last three days.

FIGURE 7.10: THREE DAY WIDE RANGE EVENING STAR IN RGLD ON 12/29/06

ANALYZING THE RESULTS

The testing of the evening star pattern to this point has indicated it performs better when two modifications are made to the basic definition. These are 1) the body of the first day is the largest body in the last three days, and 2) the second day’s body is less than 30% of the first day’s body. Running the evening star test during the 01/03/06 to 05/01/07 period using this modified definition yielded the results shown in Figure 7.11.

FIGURE 7.11: RESULTS OF MODIFIED EVENING STAR DURING 01/03/06 TO 05/01/07

We also determined that white space gaps on the third day of the pattern were one of the stronger parameter changes, but significantly reduced the number of trades. Adding that requirement to the modified definition results in the interesting results of Figure 7.12. This is the first set of parameters that shows an annualized ROI above buy and hold with a percentage of winning trades above break even.

FIGURE 7.12: TEST RESULTS OF MODIFIED EVENING STAR WITH DAY 3 WHITE SPACE GAP

The results of Figure 7.12 look interesting. The issue is that only 106 trades were produced during the test period, so the results may not be specific to the test period. The results of testing the modified definition with white space gaps on day three in other test periods is shown in Table 7.3.

TABLE 7.3 MODIFIED PATTERN WITH DAY 3 WHITE SPACE TEST RESULTS IN VARIOUS TIME PERIODS

Table 7.3 indicates that the modified pattern with day three white space gaps consistently generates 40 to 50 trades a year. It is good that the pattern generates about the same number of trades every year. Systems that generate a bunch of trades in one time period and none in another may be suspect. The problem with this modification to the evening star pattern is since it only generates a few trades every year, it is difficult to add additional filters to investigate further the reasons for these results.

The results of using the pattern beat buy and hold annualized ROI in recent times, and start to fall behind in longer time periods. It may be that the market conditions themselves have a stronger effect on evening star pattern results than variations in the pattern definition. Since strong bull and bear periods typically last for months not years, it is problematic to test this pattern in a number of bull and bear cycles and still obtain enough trades in each cycle to be significant.

Backtesting has provided us with insights into the pattern, but there are reasonable limits to what can be done. It is similar to a microscope with a given resolution. Looking for things smaller than the resolution of the microscope is problematic. Looking for meaningful results based on fewer and fewer trades can lead to unreliable results. It is not unusual in the process of testing trading patterns to find something in which the results are inconclusive in longer time frames or do not yield enough trades to be statistically significant. When this happens, you simply investigate another path.