Posted: 03 February 2006 at 9:59pm  IP Logged



Assignment Description
The questions are based on those in the Tlearn Workbook, Chapter 5 the XOR problem
The purpose of the coursework is for the student to become more familiar with Tlearn and use it to produce some results investigating the effect of training rate and momentum and starting point using a simple LOGIC problem the XOR.
Question 1 concerns investigating the effects of
i) training rate
ii) momentum
iii) starting point i.e. random number generator value
on the training times of an XOR neural network .
1 a)
The objective is to show that the no. of epochs ( sweeps / size of training set ) is a function of
i) training rate ii) momentum ii) starting point.
Since we have three independent variables it is important to keep two of them constant as we vary the third.
Select training options from the network menu. Choose a small reasonable value of momentum between 0.0 and 0.9. Keep this constant. Train the network using a range of different values of the training rate between 0.05 and 0.95. Determine the number of passes taken by the simulator to solve the XOR problem to a given stopping criterion. I suggest a stopping criterion of 0.075 on the RMS error. Repeat with different starting points i.e. fixing the random seed value using random value box.
Complete a table of training passes to the criterion – see example below.
Remember choose a fixed value for the momentum e.g. Momentum 0.5
XOR Problem : Sweeps to 0.075 RMS criterion
Random No. Training Rate
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
1
2
3
5
7
11
13
17
19
23
Minimum sweeps
Maximum sweeps
Average Sweeps
Note you will have to set another criterion about the maximum sweeps you are going to allow. Also what you are going to do about a net that still hasn't learned after the maximum period. Has this network failed or should it be given the maximum number of epochs. Calling it a failure and not using it in the average is easier.
Find the optimum training rate. Discuss your results.
b) Choose the best training rate and repeat a) producing a table of sweeps for each value of momentum from 0.1 0.9.
Choose an optimum momentum for your optimum training rate.
Hand in your tables and graphs plus your conclusions.
2. Train the network. Determine its hidden layer representation. That is determine the outputs of the hidden layer units for each input. You can get the hidden layer representation by changing the configuration file xor.cfg . Under the SPECIAL: section change the selected nodes to include the hidden layer only or as well. After training when you Probe Selected Nodes you will get their outputs in the output window.
How does it relate to our LOGICAL discussion in the lecture of the way the neural network could solve the problem.?
Plot the hidden layer representation on the same graph as the input layer representation.
Hand in your graph and conclusions
3. The XOR problem is actually the PARITY problem. If the number of ones in the input string is ODD output a 1 otherwise EVEN so output a 0.
Set up a neural network for the 3 parity problem.
data teach
0 0 0 0
0 0 1 1
0 1 0 1
0 1 1 0
1 0 0 1
1 0 1 0
1 1 0 0
1 1 1 1
How many units in the hidden layer do we need to solve this problem? Consider it logically as we did for the XOR. Hint : can we make the hidden layer units sensitive to the number of 1's?
Use tlearn. Train a network to solve the parity 3 problem. Look at the hidden layer representation. What is it doing to solve the problem?
Can we use a smaller hidden layer. Try 2.
Hand in relevant documentation showing your results
4. Try creating an XOR solution using the architecture shown below. Think about it first as we did with the
221 architecture in the lecture. Can you create this architecture and train the network with Tlearn
Hand in relevant documentation showing your results
