Neural networks with a self-refreshing memory: knowledge transfer in sequential learning tasks without catastrophic forgetting
Authors:
Bernard Ans; Stephane Rousset
DOI:
10.1080/095400900116177
Publication Frequency:
4 issues per year
Subjects:
Cognitive Artificial Intelligence.;
Cognitive Psychology;
Cognitive Science;
Computational Linguistic & Language Recognition;
Connectionism/Neural Nets;
Cybernetics;
Formats available:
PDF
(English)
View Article:
View Article (PDF)
Abstract
We explore a dual-network architecture with self-refreshing memory (Ans and Rousset 1997) which overcomes catastrophic forgetting in sequential learning tasks. Its principle is that new knowledge is learned along with an internally generated activity reflecting the network history. What mainly distinguishes this model from others using pseudorehearsal in feedforward multilayer networks is a reverberating process used for generating pseudoitems. This process, which tends to go up to network attractors from random activation, is more suitable for capturing optimally the deep structure of previously learned knowledge than a single feedforward pass of activity. The proposed mechanism for ?transporting memoryγ without loss of information between two different brain structures could be viewed as a neurobiologically plausible means for consolidation in long-term memory. Knowledge transfer is explored with regard to learning speed, ability to generalize and vulnerability to network damages. We show that transfer is more efficient when two related tasks are sequentially learned than when they are learned concurrently. With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.
|
| Keywords: Sequential; Learning; Catastrophic; Forgetting; Self-REFRESHING; Memory; Pseudorehearsal; Reverberating; Process; Memory; Transport; Long-TERM; Memory; Consolidation; Knowledge; Transfer |
| view citations (4) |

Download Citation

CiteULike
Del.icio.us
BibSonomy
Connotea