Books of Note

Practical Common
LispThe best intro to start your journey. Excellent coverage of CLOS.

ANSI Common
LispAnother great starting point with a different focus.

Paradigms of Artificial Intelligence
ProgrammingA superb set of Lisp examples. Not just for the AI crowd.

Wednesday, November 10, 2004

Investing with Lisp 

Recently, I had lunch with an ex-coworker who made the jump out of high-tech and into financial services as an investment manager. During the course of conversation, he showed me the trading analysis system that he had written in Java to allow him to try various investment strategies and perform different sorts of technical analysis on stocks and indexes. I got interested in doing some analysis myself and this seemed like a fun project to try in Lisp and see how it worked.

When you look at various functions that are common in technical analysis, you'll find that they are very similar at the macro level, but differ slightly at a micro level. For instance, you often want to operate on a sequence of closing prices and compute a function, a moving average for example, over those prices. This is ready-made for MAPCAR:

CL-USER> (mapcar #'a-function *prices*)

Given that a function like a moving average is parameterized with the number of samples in the average, I wrote a function that returns a function that computes a moving average with a specific period. Thus, it's easy to say:

CL-USER> (mapcar (moving-average 3) '(1 2 3 4 5 6 7 8 9 10))
(NIL NIL 2 3 4 5 6 7 8 9)

The function returned by MOVING-AVERAGE returns NIL for those data points that occur before it has seen the number of data points required for the average. In this case, since we need 3 data points for the average, the first two results returned are NIL.

But it's even better than that. When you use a function like a moving average with MAPCAR, you have to save some old data values that are passed to the function by MAPCAR such that they can be used later on in the calculation. I started to write MOVING-AVERAGE in the straightforward way that you'd expect when I realized that I was churning out a bunch of code to save the "historical prices." That code was common to any function, like a moving average, that need to save some data points for later. So I factored that out into a separate function called MAKE-HISTORY-FUNCTION that returns a function that saves some state and calls another function on that state each time it is updated. The code is probably easier to read than my explanation of it:

(defun make-history-function (periods evaluator)
 "Returns a function that takes a series of data points, one at a
time, and returns the results of calling the evaluator with a sequence
storing PERIODS data points. The function returns NIL before PERIODS
data points have been evaluated. If the data point is NIL, the
function returns NIL and does not store the data point in the
 (let ((history '())
	(count 0))
   (lambda (element)
     (if element
	    (push element history)
	    (incf count)
	    (if (>= count periods)
		  (setf history (subseq history 0 periods))
		  (funcall evaluator history))

Notice how the closure that is returned keeps hold of the PERIODS and EVALUATOR variables. Closure make it so easy to parameterize the returned function without having to worry about creating extra variables in which to explicitly store that parameterization information.

So, MOVING-AVERAGE then simply becomes:

(defun moving-average (periods &key (key #'identity))
"Creates a function that computes a moving average over PERIODS data points."
 (make-history-function periods (lambda (history)
				   (average history :key key))))

where AVERAGE is a function that simply averages all the data values in a sequence:

(defun average (sequence &key (key #'identity))
 (let ((len 1))
   (/ (reduce #'(lambda (x y) (incf len) (+ x y)) sequence :key key)

Now, the neat thing here is that it's easy to create other functions that operate similarly to MOVING-AVERAGE. For instance, we can compute Bollinger bands like so:

(defun bollinger-bands (periods &key (key #'identity))
 "Creates a function that computes Bollinger bands over PERIODS data
points. The function returns a list with the average, upper, and lower
Bollinger band points."
 (make-history-function periods
			 (lambda (history)
			   (let* ((avg (average history :key key))
				  (std (std-dev history :key key))
				  (2std (* 2 std))
				  (upper (+ avg 2std))
				  (lower (- avg 2std)))
			     (list avg upper lower)))))

In this case, STD-DEV computes the standard deviation of a sequence (I won't show the definition here). Notice that BOLLINGER-BANDS just concentrates on computing Bollinger bands. All the mechanics of iterating through a sequence and saving data points are wrapped up in MAPCAR and MAKE-HISTORY-FUNCTION.

We can compute Bollinger bands for a really small data set as follows. Note that for real Bollinger bands you would typically use a 20-sample period.

CL-USER> (mapcar (bollinger-bands 3) '(1 2 3 4 5 6 7 8 9 10))
    (2 3.6329932 0.36700678)
    (3 4.632993 1.3670068)
    (4 5.632993 2.3670068)
    (5 6.632993 3.3670068)
    (6 7.632993 4.367007)
    (7 8.632994 5.367007)
    (8 9.632994 6.367007)
    (9 10.632994 7.367007))

We can now easily develop other functions like the stochastic indicator:

(defun stochastic (key-high key-low key-close
		   &key (%k-periods 5) (%d-periods 5))
 "Creates a function that computes the stochastic indicator over the
specified number of periods for %K and $D. This function requires
access to high, low, and closing prices for all data points and so
accessor functions must be passed to the function to retrieve the data
from each sample."
 (let ((%d-history (moving-average %d-periods)))
   (make-history-function %k-periods
			   (lambda (history)
			     (let* ((high (maximum history :key key-high))
				    (low (minimum history :key key-low))
				    (close (funcall key-close (elt history 0)))
				    (%k (* 100 (/ (- close low)
						  (- high low))))
				    (%d (funcall %d-history %k)))
			       (list %k %d))))))

Note how the MAKE-STOCHASTIC function uses a moving average under the hood to do some of its work.

The things I really noticed about doing this exercise in Lisp versus some other programming languages I have used in the past:

Next on tap is to take all this textual information and start drawing some graphs with it.


I'm wondering about your result on bollinger bands, shouldn't you have
((2 4.0 0.0) (3 5.0 1.0) (4 6.0 2.0) (5 7.0 3.0) (6 8.0 4.0) (7 9.0 5.0) (8 10.0 6.0) (9 11.0 7.0))
as the result over '(1 2 3 4 5 6 7 8 9 10)?

I've been coming to the same conclusion--lisp is the language that technicians should use to test systems. I'm working through Practical Common Lisp as I write this.

Just curious, have you come across LUSH? That looks like the perfect platform for trading system testing, with access to C libraries, and a lisp interface.

Rob, I have not looked at LUSH previously. Very interesting. I'll check it out in more detail moving forward. Looks like a great way to do some interactive work.

Post a Comment

Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?