SIGCSE 2001 DC Application – Lisa C. Kaczmarczyk
University
of Texas at Austin
Working
Title: Training a Neural Network to Classify a Subset of Calculus Problems
Introduction
This proposal discusses an interdisciplinary project currently underway with a collaborator in partial fulfillment of course requirements in a Cognitive Science class. It is intended as a feasibility study for my Dissertation in Computer Science Education. In order to properly acknowledge the invaluable and equal contributions of my project partner Siddharth Sriram, this proposal is written in the third person.
Our project utilizes a neural network to model a subset of classification learning in Calculus. According to Mathematics Instructors with whom we have spoken, beginning Calculus students have a great deal of trouble deciding which method of Integration to apply to seemingly simple problems. This problem occurs prior to and distinct from computation of the answer. Learning to select the optimal method of Integration is not easily taught or learned and occurs over a variable period of time. This problem is well suited for attempting to model with a Neural Network.
Neural networks are ideal for classification problems in which supervised learning is involved. The task of the network is to learn how to map an input pattern to a target output pattern that denotes the target class. The network learns how to accomplish this via “practicing” with some amount of training data. At some point the network is determined sufficiently “trained”; from this point forward, the input of new data should be properly classified as well. If in fact the network can produce the correct output for the majority of input patterns in a test data set, the network can be said to generalize well. Thus it has “learned” to correctly classify from a given domain of information.
Models of adult learning often involve complex cognitive processes that are neither linear nor algorithmic. Problems in this domain are ripe for modeling in a connectionist environment such as a neural network. The dominant theory of human learning today is based upon Constructivist Learning Theory, the vocabulary of which is found not only in Psychology but Education and Artificial Intelligence as well. The essence of the Constructivist model can be seen in the structure and behavior of a neural network when one studies the interaction between input, propagation of signals and adjusting of weights to reflect successful and non successful pathways and finally in selection of a desired output. So-called “hidden units” comprise a form of memory that enables the network to build up a collection of resolved sub problems that can be utilized in future unforeseen situations. Recurrent neural nets are a mechanism by which higher order incremental learning processes can be modeled.
When we first began to develop an experiment with neural networks and adult learning we focused on the idea of training a neural network to classify conceptual understanding by college students in a Computer Science class. Further investigation revealed that this was a much larger task than could be undertaken in one month by two researchers. After discussions with faculty members in several disciplines we selected the problem, previously described, from the domain of Calculus learning.
Current Status of the Study (as
of 11/00) and Relationship to Future Work:
Selecting the exact cognitive task to model.
Many “real” adult learning problems fall under the heading of non algorithmic incremental learning, yet selecting one that can be quantified without losing its qualitative essence has proven extremely challenging. Calculus provided a compromise selection balancing the strengths of each researcher. When the project is extended into a Dissertation project, the problem domain is expected to change.
Creating a training strategy.
This has perhaps been the most challenging aspect of the project. Experimenting with creative methods for quantification of data, as described above, has caused us to gain new appreciation of the challenges and potential inherent to cognitive modeling with neural networks.
Obtaining Information on Student Errors
We are conducting clinical interviews with faculty in the Mathematics department. This data is assisting in development of training data and will be elaborated upon in the summative analysis. When the project moves into a Dissertation stage, it is expected that data will be obtained directly from students as well as faculty in whatever problem domain is selected. The Dissertation is expected to contain a rich analysis and development of two-way contribution between pedagogical implication and neural network development.
Training the network; Testing the network on data not used in training. (Followed by Analysis of results.)
At the time of this writing, we are approaching this stage in the project.
I am completing my second year of doctoral studies. I am scheduled to take qualifying exams in January-February 2000. I intend to proceed with development of a formal Dissertation proposal during Spring 2000.
Practice public speaking about research; feedback from others in Computer Science on content, format, themes. Advice from anyone who has tackled similar problems. Sharing of ideas.
Desai, Nirav S. & Miikkulainen, Risto. “Neuro-Evolution and Natural Deduction”. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN-2000, San Antonio, TX), to appear in 2000. [This article tackles a problem with similarities to my project]
Elman, Jeffrey L. “Incremental learning, or The importance of starting small”. [I’m trying to find the full reference info. Very useful information on Incremental Learning]
Gomez, Faustino & Miikkulainen,
Risto. “Incremental Evolution of Complex General Behavior”. (1997)
Adaptive Behavior 5 pp. 317-342. [This article contains practical information
about Incremental Learning]
Roth, Wolff-Michael. “Artificial
Neural Networks for Modeling Knowing and Learning in Science”. (2000)
Journal of Research in Science Teaching. Vol. 37, No. 1. pp. 63-80 [This
article introduces Neural Networks & potential pedagogical issues nicely]