Recently, I had lunch with an ex-coworker who made the jump out of high-tech and into financial services as an investment manager. During the course of conversation, he showed me the trading analysis system that he had written in Java to allow him to try various investment strategies and perform different sorts of technical analysis on stocks and indexes. I got interested in doing some analysis myself and this seemed like a fun project to try in Lisp and see how it worked.

When you look at various functions that are common in technical analysis, you'll find that they are very similar at the macro level, but differ slightly at a micro level. For instance, you often want to operate on a sequence of closing prices and compute a function, a moving average for example, over those prices. This is ready-made for MAPCAR:

CL-USER> (mapcar #'a-function *prices*)

Given that a function like a moving average is parameterized with the number of samples in the average, I wrote a function that returns a function that computes a moving average with a specific period. Thus, it's easy to say:

CL-USER> (mapcar (moving-average 3) '(1 2 3 4 5 6 7 8 9 10))
(NIL NIL 2 3 4 5 6 7 8 9)

The function returned by MOVING-AVERAGE returns NIL for those data points that occur before it has seen the number of data points required for the average. In this case, since we need 3 data points for the average, the first two results returned are NIL.

But it's even better than that. When you use a function like a moving average with MAPCAR, you have to save some old data values that are passed to the function by MAPCAR such that they can be used later on in the calculation. I started to write MOVING-AVERAGE in the straightforward way that you'd expect when I realized that I was churning out a bunch of code to save the "historical prices." That code was common to any function, like a moving average, that need to save some data points for later. So I factored that out into a separate function called MAKE-HISTORY-FUNCTION that returns a function that saves some state and calls another function on that state each time it is updated. The code is probably easier to read than my explanation of it:

(defun make-history-function (periods evaluator)
 "Returns a function that takes a series of data points, one at a
time, and returns the results of calling the evaluator with a sequence
storing PERIODS data points. The function returns NIL before PERIODS
data points have been evaluated. If the data point is NIL, the
function returns NIL and does not store the data point in the
history."
 (let ((history '())
	(count 0))
   (lambda (element)
     (if element
	  (progn
	    (push element history)
	    (incf count)
	    (if (>= count periods)
		(progn
		  (setf history (subseq history 0 periods))
		  (funcall evaluator history))
		nil))
	  nil))))

Notice how the closure that is returned keeps hold of the PERIODS and EVALUATOR variables. Closure make it so easy to parameterize the returned function without having to worry about creating extra variables in which to explicitly store that parameterization information.

So, MOVING-AVERAGE then simply becomes:

(defun moving-average (periods &key (key #'identity))
"Creates a function that computes a moving average over PERIODS data points."
 (make-history-function periods (lambda (history)
				   (average history :key key))))

where AVERAGE is a function that simply averages all the data values in a sequence:

(defun average (sequence &key (key #'identity))
 (let ((len 1))
   (/ (reduce #'(lambda (x y) (incf len) (+ x y)) sequence :key key)
      len)))

Now, the neat thing here is that it's easy to create other functions that operate similarly to MOVING-AVERAGE. For instance, we can compute Bollinger bands like so:

(defun bollinger-bands (periods &key (key #'identity))
 "Creates a function that computes Bollinger bands over PERIODS data
points. The function returns a list with the average, upper, and lower
Bollinger band points."
 (make-history-function periods
			 (lambda (history)
			   (let* ((avg (average history :key key))
				  (std (std-dev history :key key))
				  (2std (* 2 std))
				  (upper (+ avg 2std))
				  (lower (- avg 2std)))
			     (list avg upper lower)))))

In this case, STD-DEV computes the standard deviation of a sequence (I won't show the definition here). Notice that BOLLINGER-BANDS just concentrates on computing Bollinger bands. All the mechanics of iterating through a sequence and saving data points are wrapped up in MAPCAR and MAKE-HISTORY-FUNCTION.

We can compute Bollinger bands for a really small data set as follows. Note that for real Bollinger bands you would typically use a 20-sample period.

CL-USER> (mapcar (bollinger-bands 3) '(1 2 3 4 5 6 7 8 9 10))
(NIL NIL
    (2 3.6329932 0.36700678)
    (3 4.632993 1.3670068)
    (4 5.632993 2.3670068)
    (5 6.632993 3.3670068)
    (6 7.632993 4.367007)
    (7 8.632994 5.367007)
    (8 9.632994 6.367007)
    (9 10.632994 7.367007))

We can now easily develop other functions like the stochastic indicator:

(defun stochastic (key-high key-low key-close
		   &key (%k-periods 5) (%d-periods 5))
 "Creates a function that computes the stochastic indicator over the
specified number of periods for %K and $D. This function requires
access to high, low, and closing prices for all data points and so
accessor functions must be passed to the function to retrieve the data
from each sample."
 (let ((%d-history (moving-average %d-periods)))
   (make-history-function %k-periods
			   (lambda (history)
			     (let* ((high (maximum history :key key-high))
				    (low (minimum history :key key-low))
				    (close (funcall key-close (elt history 0)))
				    (%k (* 100 (/ (- close low)
						  (- high low))))
				    (%d (funcall %d-history %k)))
			       (list %k %d))))))

Note how the MAKE-STOCHASTIC function uses a moving average under the hood to do some of its work.

The things I really noticed about doing this exercise in Lisp versus some other programming languages I have used in the past:

• Lisp makes it very easy to refactor and extract common functionality into separate functions. Because function composition is so simple in Lisp, this can be done at a much finer granularity than with other programming languages. In this case, the functionality of list traversal, storing historical data points, and actually computing the underlying technical functions are all in separate functions that come together to calculate what we want.
• Lexical scoping and closures simplify things greatly. Being able to just reference lexical free variables and have the right behavior occur under the hood is a huge time saver.
• Having the list data type built-in with a standard printed representation was perfect for interactive testing. Just hop to the REPL and input a test case and check the output. Very easy.
• This would have been a lot more code in Java. Part of that would be caused by the intrinsic verbosity of Java. The rest of it would be caused by the inevitable object decomposition that would have been done. Rather than just using lists of numbers to represent a stream of price quotes, I'm sure I would have created a Price class and others that would have ballooned the code substantially.

Next on tap is to take all this textual information and start drawing some graphs with it.