Efficient C++ library for analog reservoir computing neural networks (Echo State Networks).
2007-2008, Georg Holzmann
Reservoir computing is a recent kind of recurrent neural network computation, where only the output weights are trained. This has the big advantage that training is a simple linear regression task and one cannot get into a local minimum. Such a network consists of a randomly created, fixed, sparse recurrent reservoir and a trainable output layer connected to this reservoir. Most known types are the "Echo State Network" and the "Liquid State Machine", which achieved very promising results on various machine learning benchmarks.
This library should be an open source (L-GPL) and very efficient implementation of Echo State Networks with bindings to scientific computation packages (so far to python/numpy, Pure Data and octave are in work, everyone is invited to make a Matlab binding) for offline and realtime simulations. It can be extended in an easy way with new simulation, training and adaptation algorithms, which are function objects and automatically used by the main classes.
For a theoretical overview and some papers about Echo State Networks see: Echo State Networks.
The library can be used with double or singe precision floating points and the interface of the main Echo State Network class is documented at aureservoir::ESN.
All different activation functions, simulation, training and adaptation algorithms, etc. can be changed at runtime.
Implemented simulation algorithms (see simulate.h):
Implemented training algorithms (see train.h):
Implemented reservoir adaptation algorithms:
Gaussian-IP reservoir adaptation method for tanh neurons by David Verstraeten, Benjamin Schrauwen and Dirk Stroobandt (see aureservoir::ESN::adapt)
New algorithms can be added by just deriving from the appropriate base class and overloading the relevant methods.
Aureservoir is free software. You can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation. Either version 2.1 of the License, or (at your option) any later version.
Releases of aureservoir can be found at http://sourceforge.net/projects/aureservoir/.
To get the latest source of aureservoir one can check out the SVN repository with the following instruction set:
If you don't know how to use SVN checkout the description at aureservoir subversion page .
svn co https://aureservoir.svn.sourceforge.net/svnroot/aureservoir aureservoir
The sourcefore project page of aureservoir is http://sourceforge.net/projects/aureservoir/, there you can also browse the code online at SVN browse.
To get aureservoir finally working one needs to manage two steps: first the installation of all dependencies, second the compilation of aureservoir itself.
Here I will describe only the procedure of compiling the aureservoir library and its python bindings.
Installation of all dependencies:
Aureservoir uses the C++ library FLENS, which is very fast and elegant to use (see my benchmarks at Sparse Matrix Library Benchmark).
Therefore you have to install the CVS version of FLENS first. Detailed instructions on how to do that can be found at Obtain and Install FLENS .
Python and Numpy must be installed.
Additionally FFTW3 is needed for FFT calculation.
Aureservoir uses SCons as build tool, you need that too.
Compilation of aureservoir:
The C++ aureservoir library consists of header files with template classes only, therefore you don't have to compile the library itself.
Small intro how to use SCons:
- In general SCons checks for all dependencies and the compilation can be started with the command scons.
- To install the compiled binary use scons install with root privileges.
- Additionally you can set an alternative path for installed libraries and some optimization flags, the available options can be seen with scons -h.
- Then you can set these option in example with scons arch=pentium4. The options will be stored so that you only have to set them once.
Important: You should first try to compile the C++ example in aureservoir/examples, because it gives more debug information in case something is going wrong !
To do so go into the directory aureservoir/examples and compile it with the command scons or use scons -h to see which additional options can be set.
Now change to aureservoir/python and try to compile the python bindings.
Again type scons or scons -h to see additional options and compile the library.
Afterward type scons install with root privileges if you want to install it system wide.
That's it, you should be able to use it now, so have a look at the Examples .
You can also try if all the included tests are successful, change to aureservoir/python/tests and type python run_all_tests.py.
Let me know if you have any problems, I am sure we will find a solution !
Most of the examples are in python and located in aureservoir/python/examples.
However, as the aureservoir python syntax is the same as in C++ one can use these examples analogous in C++.
Following python examples are included ATM:
Also one very basic C++ example is included:
See also examples.
The algorithms are extensively tested. All are recalculated in python and tested against the C++ implementation.
The python tests are in the directory aureservoir/python/tests. However, there are still some (old) C++ tests in aureservoir/tests using cppunit.
To run all the python unit tests change to aureservoir/python/tests and type python run_all_tests.py. This should give you an OK at the bottom of the output if everything works.
Please don't hesitate to report problems, requests or any other feedback to grh _at_ mur _dot_ at.
Feel free to also use the bug and feature request tracker at the sourceforge page aureservoir tracker.
Generated on Wed Mar 12 21:16:05 2008 for aureservoir by