This tutorial will walk you step-by-step through troubleshooting a scan that is running very slowly. In our previous tutorials, we looked at a scan that was returning no results, and a scan that was returning too many results. In this tutorial, we'll look at a scan that may be returning the expected results, but takes too long to run.
Our scanning tutorials assume just a basic knowledge of common technical indicators and chart patterns. This tutorial may build on skills learned in prior scanning tutorials. Links to these prior tutorials are provided where appropriate.
By the end of this tutorial, you will be able to troubleshoot a scan that runs very slowly, determining which clauses are causing problems and how to rearrange them for better scan speed. Once you've mastered these skills, be sure to check out our other scanning tutorials to learn about and practice other aspects of scanning.
Tip: We recommend printing out this page so you can refer to it as you follow along on your computer.
Open the Advanced Scan Workbench and enter the following scan clauses in the Scan Criteria Box:
[Special K > 0] and [group = SP500]
This scan searches for S&P 500 stocks that have a Special K indicator value greater than zero.
Click the Check Syntax button to make sure everything was typed in correctly, then click Run Scan.
If you are not familiar with the process for running a scan in the Advanced Scan Workbench, we recommend accessing our tutorial on this topic, which will help you build and run scans like the one listed above.
Check out the scan results in the new browser window (or new tab in your current browser window) that opens up. Notice how long it takes for the results to be displayed. It will likely take several seconds; it may even take so long that you get a message saying that the scan took too long to run.
The Scan Engine always processes the clauses from left to right. It filters out stocks that don't meet the first clause, and then checks the remaining stocks against the second clause, and so on. So, let's take a look at what's happening when the scan engine processes this particular scan.
First, it is checking all the symbols in our database to see if any of them have a Special K value greater than zero. The Special K calculation is very resource-intensive, involving several exponential moving averages (and each EMA calculation requires all the historical data points for that symbol). Imagine calculating this value for the tens of thousands of symbols in our database.
Once it has finished these calculations, it ends up with a much smaller list of the symbols that have a Special K value greater than zero. Then it moves on to the second clause, and checks those stocks to see if they're on the list of 500 stocks in the S&P 500. This is a much quicker process, requiring no number crunching at all.
Let's try reversing the order of those two clauses in our scan.
Update the clauses in the Scan Criteria Box so they look like this:
[group = SP500] and [Special K > 0]
Now the scan engine will first quickly check each symbol in our database to see if it is in the S&P 500. Then it will move on to the second clause and do the Special K calculations for just those 500 stocks that are in the S&P 500.
Click the Check Syntax button, then click Run Scan and notice how long it takes for the scan results to be displayed.
The scan results are the same as before (assuming your previous scan didn't time out), but the results are displayed far more quickly. Moving clauses for any technical indicators with complicated calculations to the end of your scan can drastically improve the scan's processing speed.
Once you are happy with the speed of your scan, be sure to save it, so you can easily run it again at a later date.
If you are not familiar with the process for saving your scan in the Advanced Scan Workbench, we recommend accessing our first Writing Scans Tutorial, which will walk you step by step through saving your scan.