1.1. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is work in progress, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author does not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

1.2. Formats

For the sake of completeness, and for the case that you are interested in some other formats of this summary document, here they are:

• HTML (HyperText Markup Language), many HTML files (one for every section), for viewing with any browser

• HTML (HyperText Markup Language), one big HTML file

• TXT (ASCII Text)

• RTF (Rich Text Format)

• PDF (Portable Document Format)

• PS.GZ (Compressed Postscript)

• SGML (Standard Generalized Markup Language)

• LYX (LaTeX frontend LyX)

	RTF: Page numbers
	In order to get correct page numbers in Microsoft Word, type the following after opening the document: CTRL-END CTRL-A F9 In Word Viewer 97, you must instead do: CTRL-END ALT V N ALT V P See The OpenJade RTF backend for more details.

	IMPORTANT: Downloads for offline reading!
	If you want to download the HTML or RTF formats for offline reading, you will need to download the images as well - PNG for HTML and BMP for RTF, including the callouts! To save you the hassle, I have compiled the following zipped tar archives for offline reading:

• TAR.GZ (Compressed TAR Archive), many HTML files with images

• TAR.GZ (Compressed TAR Archive), one big HTML file with images

• TAR.GZ (Compressed TAR Archive), SGML file with images

• TAR.GZ (Compressed TAR Archive), RTF file with images

A tarball containing all the above is also available:

• TAR.GZ (Compressed TAR Archive), All files

The newest versions of all the documents presented here contain some major improvements over older ones:

• New professional look through CSS.

• New footer icons. If you click on them, they will validate the current page for HTML and CSS respectively.

• HTML code that validates as HTML 4.01 transitional.

1.3. License

Copyright © 2002-2006 Chris Karakas. All rights reserved.

1.4. Conventions

To use the access keys, you have to simultaneously press a modifier key, which may vary from browser to browser. For example in NN6+/Mozilla, the modifier key is ALT, so you have to use ALT-N to go to the next page, and ALT-P to come back. In other browsers such as IE6, the access keys just give focus to the associated link, so the sequence becomes ALT-N Enter . Try it, you'll like it!

3.1. RPM packages

I compile binary and source RPM packages for the software I need and don't have on my SuSE Linux system - either because the available version from SuSE is too old, or because there is no package available at all. Of course one can use the usual 3-line mantra:

./configure
make
make install

to install a package from source, but having a RPM package helps keep track of the files installed on a system, as well as their dependencies.

I offer binary and source RPMs for the following software, as well as any dependent packages:

• LyX, a LaTeX and DocBook editor.

• GNU MP library (GMP), a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating point numbers.

• Octave, a high-level language, primarily intended for numerical computations.

• Yacas, (Yet Another Computer Algebra System), a small and highly flexible general-purpose computer algebra system and programming language.

• po4a, (po for anything) project goal is to ease translations (and more interestingly, the maintenance of translations) using gettext tools on areas where they were not expected like documentation.

• poedit, a cross-platform gettext catalogs (.po files) editor.

• Anjuta, a versatile Integrated Development Environment (IDE) for C++ and C++ on GNU/Linux.

• Graphviz, an open source graph visualization software. See How to use Graphviz to generate complex graphs for a tutorial.

• wxWidgets (wxGTK), a single, easy-to-use API for writing GUI applications on multiple platforms.

• Code::Blocks, a free C++ IDE, built specifically to meet the most demanding needs of its users.

• GTK+ 2 (package gtk2), a multi-platform toolkit for creating graphical user interfaces. Offering a complete set of widgets, GTK+ is suitable for projects ranging from small one-off projects to complete application suites.

• MySQL Administrator, a powerful visual administration console that enables you to easily administer your MySQL environment and gain significantly better visibility into how your databases are operating.

• MySQL Query Browser, a visual tool for creating, executing, and optimizing SQL queries for your MySQL Database Server. The MySQL Query Browser gives you a complete set of drag-and-drop tools to visually build, analyze and manage your queries.

• gtkhtml, a HTML rendering/editing library designed to be easily embedded into applications that require lightweight HTML functionality. Many programs that use GTK+, like the MySQL Query Browser, need the gtkhtml2 package.

• Dia, is inspired by the commercial Windows program 'Visio', though more geared towards informal diagrams for casual use. It can be used to draw many different kinds of diagrams. It currently has special objects to help draw entity relationship diagrams, UML diagrams, flowcharts, network diagrams, and many other diagrams.

• The GIMP, (GNU Image Manipulation Program) is a powerful image composition and editing program, which can be extremely useful for creating logos and other graphics for Web pages. The GIMP has many of the tools and filters you would expect to find in similar commercial offerings, and some interesting extras as well. The GIMP provides a large image manipulation toolbox, including channel operations and layers, effects, sub-pixel imaging and anti-aliasing, and conversions, all with multi-level undo.

Use the command

rpm -Uhv xxx.rpm

to install the binary RPMs. I used the build package of SuSE to compile my own source RPMs from source, as described for example in Compiling SuSE Source Packages. Note that up to and including version 9.0, SuSE uses the usedforbuild string on a comment line of the spec file to list all needed files for a successful compilation of the package.

The packages are build for SuSE 9.0. To rebuild a package for your own version of (SuSE) Linux, use the source RPMs and the build and/or rpmbuild commands, as described in How to compile an older version for a newer system in RPM.

5.1. Master's thesis

My Master's thesis (called “ diploma work” in german) at Kaiserslautern dealt with the mathematical aspects of some neural networks, the exact title being “ Neural learning rules for linear association and separation”. After giving a formal mathematical definition of neural networks, I consider the linear association problem and some so-called “ learning rules” for its solution:

• Hebb's rule

• Delta rule

• Generalized inverses and the Greville algorithm

I show that the Greville algorithm has the same general form as the Delta rule but does not have its limitations regarding orthogonality (Hebb) or linear independence (Delta) of the input vectors and infinite repetition of the learning process (in fact, it converges to a solution after a finite number of steps, so that the input vectors need only presented once to the network).

Continuing from linear association to linear separation, I consider

• the Perceptron rule

• the relaxation rule

• the Ho-Kashyap rule

I give a generalized convergence proof for the Perceptron learning rule. I present corrected proofs of the relaxation and Ho-Kashyap rules and touch on a solution by linear programming. As you can read in a 1997 paper by Prof. G. Labonté, the scientific community did not seem to realize that the relaxation method can be used in the context of neural networks. In my 1992 Master's thesis, I devote a whole chapter (5.2, pp.30-37) to the treatment of the relaxation method for the linear separation problem which is fundamental to neural networks (you can transform a non-linear separation problem to a linear one, using an aproppriate non-linear transformation of the input space).

Finally all methods are investigated theoretically as well as practically regarding their asymptotic time behaviour.

If you are interested in a printed copy of my Master's thesis (in german) and the source code (FORTRAN), you can buy it online: Master's thesis on Neural Networks. The following table contains the key data: