Commun. KISS 18 (8): 2000, pp. 78-89
(Communications of the Korea Information Science Society)
Michael Conrad, Klaus-Peter Zauner
Department of Computer Science, Wayne State University
Detroit, MI 48202, USA
biocomputing@cs.wayne.edu.
Abstract---Today's computers are built up from a minimal set of standard pattern recognition operations. Logic gates, such as NAND, are common examples. Biomolecular materials offer an alternative approach, both in terms of variety and context sensitivity. Enzymes, the basic switching elements in biological cells, are notable for their ability to discriminate specific molecules in a complex background and to do so in a manner that is sensitive to particular milieu features and indifferent to others. The enzyme, in effect, is a powerful context sensitive pattern recognizer. We describe a tabletop pattern processor that in a rough way can be analogized to a neuron whose input-output behavior is controlled by enzymatic dynamics.
Keywords: artificial neural networks, pattern recognition, biological information processing, enzyme kinetics, signal processing
In 1904 a school teacher found that he was able to successfully teach his horse, Clever Hans, elementary mathematics. If he asked Hans for the sum of two digits, the horse would tap out the correct result. It turned out, however, that the horse was not doing arithmetic. It used minute involuntary reactions of its teacher to decide when to stop tapping (O'Grady, Dobrovolsky, and Aronoff 1997, p. 612). At the time, the summary judgment was that Hans was not at all clever. But just try to program a computer to perceive and learn how to utilize subtle changes in expression. From the viewpoint of today's computer technology adding the digits seems trivial compared to the pattern recognition problem the horse chose to solve.
The eloquence with which biological organisms handle pattern processing tasks can be traced to molecular pattern recognition capabilities of macromolecules (Conrad 1992). Proteins are particularly versatile in this respect. Protein catalysts, or enzymes, are capable of discriminating and acting on specific molecular shapes in a complex milieu and doing so in a manner that is selectively sensitive to the milieu context.
To what extent does this intrinsic capability of macromolecules percolate up into the perception-action capabilities of a Clever Hans? The question clearly has much to do with the capabilities of neurons. Is the neuron a mere summator of its inputs, reacting only to an average field, or is it itself a powerful molecular computer? One of the great pioneers of neural computing, Warren McCulloch, made a comment pertinent to this point:
``For our purpose of proving that a real nervous system could compute any number that a Turing machine could compute with a fixed length of tape, it was possible to treat the neuron as a simple threshold element. Unfortunately, this misled many into the trap of supposing that threshold logic was all one could obtain in hardware or software. This is false. A real neuron, or Crane's neuristor, can certainly compute any Boolian [sic] function of its inputs---to say the least!'' (McCulloch 1965, pp. 393, 394)
To this it is perhaps worth adding that in the cerebral cortex of the mouse, for example, the average number of synapses (inputs) per neuron is 8000 (Schüz 1995). It is awkward even to think of Boolean logic in the face of such high connectivity.
We can note another bit of history, connected with the perceptron concept of Rosenblatt (1962). This concept indeed used an essentially threshold neuron. Rosenblatt found a simple, effective learning algorithm for a single layer perceptron. The idea was severely criticized by Minsky and Papert (1969). The single layer perceptron could not be used to discriminate linearly inseparable patterns. The exclusive-or (XOR) operation is the simplest example. Multilayer perceptrons can of course do this job, as can laboratory rats (Griffith, Davis, and Kause 1968). But then the definite learning algorithm used by Rosenblatt no longer applied. Evolutionary methods of learning could be employed. However, for various reasons the computer science community at the time was not prepared to accept self-organizing systems and evolutionary methods. Handcrafted approaches were the order of the day. Neural nets and evolutionary approaches had to sit on the sidelines until relatively recently.
There is another way out of the XOR limitation. The neuron, as noted by McCulloch, need not be a simple threshold element. We will show here that individual enzymes can be used to perform the XOR operation. Neurons and other biological cells contain thousands of interlinked enzymes. Our assumption is that the opportunity seeking process of evolution uses these enzyme networks to implement complex information processing functions at the cellular level. We have developed a tabletop prototype, a crude artificial neuron of sorts, that makes it possible to investigate how such molecular pattern recognizers could be utilized in a device context. Conceivably the pattern recognition virtuosity of natural biological cells is based on similar operative principles.
Enzymes are proteins that act as highly specific catalysts capable of discriminating particular substrates in a complex chemical background. The catalytic function performed is controlled by the enzyme's shape---its 3-D spatial structure (Friedrich 1984). This structure is largely determined by amino acid sequence. The structure is not rigid; it undergoes considerable conformational motion, i.e., rotation around atomic bonds (Frauenfelder, Park, and Young 1988; Yon, Perahia, and Ghélis 1998). Which subset of conformational states is favored is a function of an enzyme's physiochemical environment. Catalytic activity is critically dependent on conformational state and therefore provides a sensitive and convenient probe for conformation change. The intricate conformational dynamics of the enzyme fuses signals from its physiochemical milieu and modulates catalytic activity. The enzyme is in effect an implementation of a function that maps numerous selected variables presented as physiochemical context into output communicated as catalytic activity.
The cell, as noted earlier, contains thousands of enzymes. In single celled organisms, such as amoeba or paramecium, all the information processing is mediated by the internal molecular network. Chemical signals from the environment are communicated through the cell membrane, either directly or by triggering the release of internal chemicals through interaction with membrane receptors (see Fig. 1). The signals influence the conformational dynamics of specific enzymes in these networks, leading to cascades of reactions that eventually culminate in the action of the cell. In neurons, for example, transmitters impinging on the external membrane may be transduced to cyclic nucleotide molecules within the neuron that serve as second messengers. These can affect target proteins on DNA, on the various internal fibrous structures of the cell (the cytoskeleton) or on the membrane. The target proteins then activate effector proteins; for example, membrane channel proteins that control the nerve impulse (Liberman, Minina, and Golubtsov 1975).
Fig. 1.
Conformational signal processing in biological cells.
Impinging chemical signals directly or indirectly affect the
internal milieu of the cell. The conformational dynamics of
proteins and other macromolecules is selectively sensitive to
these milieu features. These nonlinear dynamics in effect
process the milieu influences to yield internal or external
cellular actions.
The cell in Fig. 1 is schematically pictured as a kind of mixing chamber. In fact the interior of the cells of higher organisms is highly structured, with many fibers and membranous interfaces. The fibers, as noted above, are referred to as the cytoskeleton, since they are responsible for maintaining the structure of the cell. They are sometimes thought of as micro-muscle, since they contribute significantly to cellular motions and to the movement of materials within the cell. Some lines of evidence suggest that the cytoskeletal fibers also act as a kind of micronervous system within cells and neurons that serves to coordinate internal activities (like the highly choreographed process of cell division) and plausibly to mediate more subtle forms of signal processing pertinent to the cell's capability of integrating external signals in space and time (Matsumoto and Sakai 1979; Liberman et al. 1985; Hameroff 1987).
The tabletop device to be described is a highly abstracted version of this complex intracellular processing. Three basic elements enter into this abstraction. Macro signals must be transduced to a form that affects the activity of an enzyme or a collection of enzymes. The action of the enzyme must not be confined to an averaging process that could be performed by a transistor (e.g., OR, AND, NAND). If this were the case all of the powerful shape-based recognition activity of the enzyme would be lost. Programmability should not be imposed since this requires each component to have a context independent description that would vitiate the distinctive advantage of enzyme-driven computing, namely the possibility of utilizing the vast number of potential interactions in multiple ways for multiple purposes. The key to the power of the system is self-organizing dynamics at the level of the enzyme and at the level of macromolecular networks. If the system could be programmed prescriptively, like a digital machine, then all the power of this self-organization would have to be suppressed. Self-organizing systems after all have a mind of their own. Evolutionary adaptive approaches are called for (Conrad 1985; 1990).
We constructed a tabletop prototype that uses the enzyme malate dehydrogenase (MDH) to classify input signal patterns. MDH occurs in a wide variety of species including the microbial world and in plants. In our experiments we used mitochondrial MDH from pig heart, a homodimer with each monomer consisting of 314 amino acids (Gleason et al. 1994).
MDH catalyzes the oxidation of malate to oxalacetate by reducing the oxidized form (NAD+) of nicotinamide adenine dinucleotide (NAD) to the reduced form NADH.
For high pH the equilibrium of the reaction is on the right side. The time course of the reaction is affected by the sensitivity of the enzyme to chemical context, i.e., the milieu in which the reaction takes place. The milieu is composed from a number of fixed components, such as the substrates and pH buffer, plus the components representing the signals. In the experiments to be described here MgCl2 was used as the signaling substance. Similar effects can also be achieved with CaCl2, but the method is not restricted to ions or to any definite number of signal carriers.
The device itself is schematically illustrated in Fig. 2. Signals are injected into a mixing chamber in two 1 ml portions, taken in any of four possible combinations from the two signaling solutions. Solutions representing 0 and 1 contain substrate in the same quantity, but the 1-solution in addition contains signaling ions. The quantity of MgCl2 representing a 1-signal is chosen so that the absorbance resulting from a single dose of signaling substance is maximally separated from the absorbance produced either by a double dose or no dose (whichever is closer).
Fig. 2.
Flow system used in the XOR experiments. Signals are
injected from any two of four syringes (S1 to S4). The
1-syringes (S1 and S2) are filled from reservoir R1 that
contains signal substance (Mg2+) plus substrate. R2, the
reservoir for the 0-syringes (S3 and S4), contains
substrate in the same concentration as R1 but no signaling
substance. Signals are injected through valves (labeled
by V) into the mixing chamber. Enzyme solution (MDH and
NAD+ in buffer) is stored in the thermally isolated
reservoir R3. The reaction is initiated by injecting this
solution, using syringe S5, into the mixing chamber. The
reaction mixture is drawn by a peristaltic pump into a
flow cuvette installed in the spectrophotometer. The
absorbance of the product NADH at 339 nm serves as output
signal. Reservoir R4 contains distilled water which is
used to wash the system clean after the input pattern is
processed. T valves (T1 to T6) are used to switch from
processing to clearing. Tubes labeled atm (atmosphere)
are air in/outlets.
The reaction mixture (a fluid phase traveling in an air-filled tubing, Fig. 3) is pumped to the spectrophotometer. The accumulation of NADH serves as output signal. After processing is completed the system is cleared for the next input pattern by pumping distilled water through it.
Fig. 3.
Two phase transport utilized in the tabletop implementation. The
reaction medium (fluid phase) travels in air-filled (gas-phase)
tubing from the mixing chamber to the detector.
The output signal is sent to a computer. If the signal is above a single prescribed threshold level a 1 is displayed, otherwise a 0. Several trials are run to calibrate the threshold. When the reaction mixture reaches the cuvette the absorbance increases. This increase is used to trigger a countdown to a subsequent absorption measurement that decides the output. The actual implementation is pictured in Fig. 4.
The XOR operation requires a device to say yes (give a 1 output) in response to binary signal inputs 01 and 10 and to say no (give a 0 output) to 00 and 11 input patterns. The operation is linearly inseparable, corresponding to the fact that the patterns to be placed in the yes and no categories cannot be separated by a single threshold (in contrast to NAND, AND and OR operations). An element whose response is linear apart from a single threshold cannot perform the XOR, since it would be necessary for it to fire when the strength of the combined input exceeds a lower threshold and not fire when it exceeds a higher threshold. If the response of the element is nonlinear (strictly speaking nonmonotonic) then it is possible to eliminate the need for the higher threshold and therefore to convert the linearly inseparable pattern recognition problem to a linearly separable problem.
An enzyme, to satisfy this requirement, must increase its activity in response to one dose of the signaling substance but decrease it in response to two doses. We found that MDH satisfies the requirement with respect to MgCl2 used as a signaling substance. Thus when MDH is used as the enzyme in the mixing chamber the device yields a 0 output when the input is 00, a 1 output when the input is 10 or 01, and a 0 output when the input is 11.
Fig. 5 illustrates a series of 42 XOR tests, with the trigger set for an average 10 second response. The threshold was set based on the first measurement of the three distinguishable input patterns (since 01 is the same as 10). Only one input pattern was incorrectly classified. It is possible to choose a threshold that separates the cases more securely (but all settings would leave at least one failure in this series). Ten seconds is near the limit of the device in its present form, due in part to the type of pump and spectrophotometer employed. The reaction that drives the device is the only fundamental limiting factor. This can be sped up by increasing the concentration of enzyme or by warming the reaction fluid. The important point is that MDH serves as a transform that converts a linearly inseparable pattern recognition task into a linearly separable one.
Fig. 5.
Pattern grouping with MDH as illustrated by XOR task. The three
distinguishable input patterns are grouped into two output
categories depending on whether the amount of product formed is
above or below the threshold (T). The time when measurements used
for classification are taken is controlled by a trigger mechanism.
The times fall between 9 and 11 seconds after start of the
reaction, with two exceptions. The last measurement in the 00 and
also the 11 pattern sets were triggered early, leading to low
absorbance values (denoted by A). The 10 second absorbance values
recorded were below the threshold and hence would still have been
correctly classified. A 0-signal is represented by 1 ml of 7.1 mM
malate and 112 mM glycine. A 1-signal is represented by 1 ml of
190 mM MgCl2, 7.1 mM malate, and 112 mM glycine. Both signal
solutions are adjusted to pH 10.5 with NaOH. Two ml of input
signal solution constitute an input pattern. The reaction is
initiated when this combines with 1 ml of MDH-NAD+ solution
(5.3 mM NAD+ in MOPS buffer adjusted to pH 7.4 with NaOH).
The XOR operation was implemented by taking a snapshot of the response of our tabletop neuron at a particular point in time as output value. The chemically encoded signals were present at the start of the reaction. The course of the reaction was determined by the initial reaction conditions. Reducing the amount of enzyme slows the reaction down. Its progress can then conveniently be followed and is shown in Fig. 6 for various amounts of MgCl2.
Fig. 6.
Measured absorbance change over time for various Mg2+
concentrations. The reaction medium contained 4.8 mM L-malic
acid, 1.8 mM NAD+, and 13.2 mM MOPS
(3-[N-morpholino]propanesulfonic acid, used to buffer the enzyme
(porcine heart mitochondrial MDH) and NAD+ solutions) and was
buffered by 92 mM glycine adjusted to pH 10 with NaOH. The
protocol was derived from the assay described by
Englard and Siegel (1969).
[enlarge]
As can be seen from the crossing curves in Fig. 6, a change in MgCl2 concentration has a qualitative effect on the course of the reaction.
To utilize the phenomenological enzyme behavior we need to decide on an encoding for the signal patterns to be processed. A simple encoding scheme represents a 1-signal arriving on a signal line by a fixed amount of one substance and a 0-signal by the absence of this substance. Such an encoding allows for the implementation of commutative operations (x o y = y o x) only, since a change in the order of the operands will not lead to a change in the chemical milieu. Therefore 2-bit input patterns will give rise to 3 possible milieu states, here called a for a 00-input, b for either 01- or 10-input, and c for 11-input. If operations are to be implemented that are not commutative, a larger number of signaling substances could be used. It is then possible to encode the signal line together with the signal, i.e., the state of the line.
With the signal encoding decided upon, the possible milieu conditions at the start of the reaction are known. Differences in the starting milieu are mapped by the reaction into different absorbance values. From the absorbance measurements in Fig. 6 it is possible to calculate how the reaction groups the input signal patterns. The time development of the grouping is shown in Fig. 7 as distances among the responses to the 3 possible milieu states.
Fig. 7.
Absorbance difference (ΔA) in response (r) to the
three possible milieu states that can result from 2-bit input
patterns (a = 00, b = 01 or 10, c = 11).
The curves are computed from data in Fig. 6,
assuming a 1-signal is represented by 66.7 mM MgCl2.
For implementing a desired input-output map the minimum difference in the response to signal patterns that need to be differentiated is important, since this distance corresponds to the signal strength. Fig. 8 shows the signal strength for the XOR operation.
Fig. 8.
Time development of XOR signal strength. The signal strength
(Δs) is given by r(b) - Max(r(a), r(c));
cf. Fig. 7 for notation. The increase in noise
for longer reaction times is associated with the high absorbance
values at those times. The curve is calculated assuming the same
signal encoding as in Fig. 6.
The signal strengths for particular classifications change with the progress of the reaction. Signals arriving at different times will therefore meet different states of the reaction medium and accordingly affect the reaction differently. Enzyme controlled devices could be used in this way to integrate signals in space and time, in analogy to the spatiotemporal processing performed by natural biological cells.
Note that the time axes in the figures discussed here are relative to the reaction speed. To implement pattern classification it is possible to run the reactions much faster (as was done in the experiment discussed in section 4). However, a high reaction speed is not convenient for studying the time course of the reaction.
Our prototype could be migrated to a more practical device using microfluidics (Hadd et al. 1997; Unger et al. 2000). Such lab-on-a-chip modules could be adapted for desired function by varying the coding of the inputs and tuning the reaction milieu (as in the tabletop experiments above), choosing the reaction parameters used for readout (Kessler 1994), varying and combining enzymes, or modifying enzymes through directed evolution (Beaudry and Joyce 1992; Gao et al. 1997). The addition of more types of signaling substances and coupling to other enzymatic reactions should increase the complexity of the patterns that can be processed. MDH is frequently used as an indicator reaction in assays and therefore protocols for linking to other enzymes are available (cf. Williamson and Corkey 1969). Our expectation is that the computational capabilities of many enzymes could usefully be investigated using the methodology outlined here.
The evolutionary adaptation approach is called for due to the context sensitive dynamics of enzyme networks. This is incompatible with conventional programmability. It might be possible to eliminate the context sensitivity, but this would abrogate the unique advantage of enzyme-based computing (Conrad 1988).
We envision migrating elaborated signal processing modules to microchip devices that can be integrated with conventional electronic signal processing. The decrease in reaction volume would increase speed. Alternative designs using enzyme immobilization techniques (DuVal, Swaisgood, and Horton 1985) and coupling to dyes for optoelectronic readout could be utilized (Whitaker 1969; Michal, Möllering, and Siedel 1983).
The tabletop device can be used to experiment with such hybrid architectures. Servos intended for model airplanes can be employed as shown in Fig. 9 to operate the valves and syringes.
Fig. 9.
Interfacing the artificial enzymatic neuron with a conventional
architecture. Servos from radio controlled models can be operated
through pulse-width modulation by a digital computer. The servo
on the left side steers a T-valve and the servo on the right
positions a syringe pump.
Automation should speed the development of a useful repertoire of biochemical neurons, but this is still a big task. In the meantime it is possible to experiment with simulated neuromolecular architectures. The idea is to use neurons that draw on internal signal integration mechanisms to perform complicated input-output transforms and to combine these into structures that can perform coherent perception-action tasks. Our group has developed a series of such models (e.g., Kampfner and Conrad 1983; Kirby and Conrad 1986; Conrad et. al. 1989; Ugur and Conrad 1999). An architecture developed with J.-C. Chen is indicative. This consists of cellular automaton neurons that model cytoskeletal signal processing. The complex internal dynamics of the neurons allows for the evolution of a wide variety of input-output transforms. Redundant subnetworks can evolve independently and the resulting neurons are then harvested and combined in different ways by a higher level evolutionary algorithm. The system has been applied to a variety of problem domains, including maze navigation (Chen and Conrad 1994), Chinese character classification (Chen and Conrad 1997), and most recently to hepatitis diagnosis (Chen 2000).
Architectures such as the above provide important hints about the adaptive procedures that would be pertinent to systems of real neurons. The simulated neurons of course have much less power than would be possible with actual biochemical embodiments. That useful functionality is obtained with a simulated system suggests that replacing the simulated biochemical neurons with actual biochemical neurons would yield much more powerful computational capabilities.
The results reported here appear to have implications for the neuron doctrine. Most technical neural computing models still assume essentially rather simple neurons. Many brain models take their cue from these technological systems, despite the evident complexity of real neurons. The fact that a single enzyme type can transform a linearly inseparable problem to a linearly separable one strongly suggests that real biological neurons have capabilities that far exceed those typically represented in current neurocomputing models.
Our working hypothesis is that multi-enzyme extensions of the enzyme-driven system prototyped here could transform difficult pattern grouping problems into forms manageable by conventional techniques. Devices of this type could serve as molecular co-processors that provide novel computational synergies for digital machines. In time networks of artificial molecular neurons may be evolved for a wide variety of special purpose applications that are refractory to currently available technologies. Much remains to be done, but perhaps some day it will be possible to achieve an artificial Clever Hans.
Acknowledgments---This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. ECS-9704190, CCR-9610054 and EIA-9729818.