User:Veritas Aeterna/Work in Progress, Symbolic Machine Learning

Symbolic machine learning approaches were investigated to address the knowledge acquisition approach. One of the earliest is Meta-DENDRAL. It is described below, in a discussion of the DENDRAL expert system, by Ed Feigenbaum, from a Communications of the ACM interview, Interview with Ed Feigenbaum:

One of the people at Stanford interested in computer-based models of mind was Joshua Lederberg, the 1958 Nobel Prize winner in genetics. When I told him I wanted an induction "sandbox", he said, "I have just the one for you." His lab was doing mass spectrometry of amino acids. The question was: how do you go from looking at a spectrum of an amino acid to the chemical structure of the amino acid? That's how we started the DENDRAL Project

... Meta-DENDRAL was the culmination of my dream of the early to mid-1960s having to do with theory formation. The conception was that you had a problem solver like DENDRAL that took some inputs and produced an output. In doing so, it used layers of knowledge to steer and prune the search. That knowledge got in there because we interviewed people. But how did the people get the knowledge? By looking at thousands of spectra. So we wanted a program that would look at thousands of spectra and infer the knowledge of mass spectrometry that DENDRAL could use to solve individual hypothesis formation problems.

We did it. We were even able to publish new knowledge of mass spectrometry in the Journal of the American Chemical Society, giving credit only in a footnote that a program, Meta-DENDRAL, actually did it. We were able to do something that had been a dream: to have a computer program come up with a new and publishable piece of science.

Meta-DENDRAL used a generate-and-test technique to generate plausible rule hypotheses that were then tested against spectra. Successful rules explained mass spectrometry data.

More generally, Ross Quinlan invented decision tree learning approaches to statistical classification, starting with ID3 and then later extending it to C4.5. Decision trees create interpretable classifiers, with human-interpretable classification rules.

Tom Mitchell introduced version space learning which describes learning as search through a space of hypotheses, with upper and lower boundaries encompassing all viable hypotheses consistent with examples seen so far. In computational learning theory, Valiant introduced Probably Approximately Correct Learning (PAC Learning), a framework for mathematical analysis of machine learning. In addition to concept learning from examples, inductive logic programming allowed logic programs to be synthesized from traces.

Symbolic machine learning could also incorporate background knowledge. For example, analytic learning was learning concepts that satisfied requirements by proving the requirements were satisfied by the concept definition.

As an alternative to logic, Roger Schank introduced case-based reasoning (CBR), one that focuses on problem-solving cases, retrieving these, and then adapting them to solve similar problems. Another alternative, genetic algorithms and genetic programming are based on an evolutionary model of learning, where sets of rules were encoded into populations, the rules governed the behavior of individuals, and selection of the fittest pruned out sets of unsuitable rules over simulated generations.

Symbolic machine learning encompassed more than learning by example. Learning by analogy constructed problem solutions from similar problems seen in the past, and modifying their solutions to fit the current situation. Learning by explanation involved learning from examples coupled with explanations of how they satisfied concept definitions. Learning from experts similarly learned from explanations provided by experts.Learning from instruction involved taking human advice and determining how to operationalize it in specific problem situations and incorporating its guidance for the future.

Symbolic machine learning was applied not only to concept learning, but also learning rules, heuristics, and problem-solving.