AUGMENTING HUMAN INTELLECT : A Conceptual Framework. By: Dr. Engelbart 1962
: Detailed Discussion of the H-LAM/T System Human using Lauguage, Artifacts, Methodology, in which he is Trained
SHOW CONTENTS MENU
The Source of Intelligence
When one looks at a computer system that is doing a very complex job, he sees on the surface a machine that can execute some extremely sophisticated processes. If he is a layman, his concept of what provides this sophisticated capability may endow the machine with a mysterious power to sweep information through perceptive and intelligent synthetic thinking devices. Actually, this sophisticated capability results from a very clever organizational hierarchy so that pursuit of the source of intelligence within this system would take one down through layers of functional and physical organization that become successively more primitive.
To be more specific, we can begin at the top and list the major levels down through which we would pass if we successively decomposed the functional elements of each level in search of the "source of intelligence." A programmer could take us down through perhaps three levels (depending upon the sophistication of the total process being executed by the computer) perhaps depicting the organization at each level with a flow chart. The first level down would organize functions corresponding to statements in a problem-oriented language (e.g., ALGOL or COBOL), to achieve the desired over-all process. The second level down would organize lesser functions into the processes represented by first-level statements. The third level would perhaps show how the basic machine commands (or rather the processes which they represent) were organized to achieve each of the functions of the second level.
Then a machine designer could take over, and with a block diagram of the computer's organization he could show us (Level 4) how the different hardware units (e.g., random-access storage, arithmetic registers, adder, arithmetic control) are organized to provide the capability of executing sequences of the commands used in Level 3. The logic designer could then give us a tour of Level 5, also using block diagrams, to show us how such hardware elements as pulse gates, flip-flops' and AND, OR, and NOT circuits can be organized into networks giving the functions utilized at Level 4. For Level 6 a circuit engineer could show us diagrams revealing how components such as transistors, resistors, capacitors, and diodes can be organized into modular networks that provide the functions needed for the elements of Level 5.
Device engineers and physicists of different kinds could take us down through more layers. But rather soon we have crossed the boundary between what is man-organized and what is nature-organized, and are ultimately discussing the way in which a given physical phenomenon is derived from the intrinsic organization of sub-atomic particles, with our ability to explain succeeding layers blocked by the exhaustion of our present human comprehension.
If we then ask ourselves where that intelligence is embodied, we are forced to concede that it is elusively distributed throughout a hierarchy of functional processes--a hierarchy whose foundation extends down into natural processes below the depth of our comprehension. If there is any one thing upon which this 'intelligence depends' it would seem to be organization. The biologists and physiologists use a term "synergism" to designate (from Webster's Unabridged Dictionary, Second Edition) the "...cooperative action of discrete agencies such that the total effect is greater than the sum of the two effects taken independently..." This term seems directly applicable here, where we could say that synergism is our most likely candidate for representing the actual source of intelligence
Actually, each of the social, life, or physical phenomena we observe about us would seem to derive from a supporting hierarchy of organized functions (or processes), in which the synergistic principle gives increased phenomenological sophistication to each succeedingly higher level of organization. In particular, the intelligence of a human being, derived ultimately from the characteristics of individual nerve cells, undoubtedly results from synergism.
It has been jokingly suggested several times during the course of this study that what we are seeking is an "intelligence amplifier." At first this term was rejected on the grounds that in our view one's only hope was to make a better match between existing human intelligence and the problems to be tackled, rather than in making man more intelligent. But deriving the concepts brought out in the preceding section has shown us that indeed this term does seem applicable to our objective.
Accepting the term "intelligence amplification" does not imply any attempt to increase native human intelligence. The term "intelligence amplification" seems applicable to our goal of augmenting the human intellect in that the entity to be produced will exhibit more of what can be called intelligence than an unaided human could; we will have amplified the intelligence of the human by organizing his intellectual capabilities into higher levels of synergistic structuring. What possesses the amplified intelligence is the resulting H-LAM/T system, in which the LAM/T augmentation means represent the amplifier of the human's intelligence.
In amplifying our intelligence, we are applying the principle of synergistic structuring that was followed by natural evolution in developing the basic human capabilities. What we have done in the development of our augmentation means is to construct a superstructure that is a synthetic extension of the natural structure upon which it is built. In a very real sense, as represented by the steady evolution of our augmentation means, the development of "artificial intelligence" has been going on for centuries.
The human and the artifacts are the only physical components in the H-LAM/T system. It is upon their capabilities that the ultimate capability of the system will depend. This was implied in the earlier statement that every composite process of the system decomposes ultimately into explicit-human and explicit-artifact processes. There are thus two separate domains of activity within the H-LAM/T system: that represented by the human, in which all explicit-human processes occur; and that represented by the artifacts, in which all explicit-artifact processes occur. In any composite process, there is cooperative interaction between the two domains, requiring interchange of energy (much of it for information exchange purposes only). Figure 1 depicts this two domain concept and embodies other concepts discussed below.
Fig.1: The Two Sides of the H-LAM/T System
Where a complex machine represents the principal artifact with which a human being cooperates, the term "man-machine interface" has been used for some years to represent the boundary across which energy is exchanged between the two domains. However, the "man-artifact interface" has existed for centuries, ever since humans began using artifacts and executing composite processes.
Exchange across this "interface" occurs when an explicit-human process is coupled to an explicit-artifact process. Quite often these coupled processes are designed for just this exchange purpose, to provide a functional match between other explicit-human and explicit-artifact processes buried within their respective domains that do the more significant things. For instance, the finger and hand motions (explicit human processes) activate key-linkage motions in the typewriter (couple to explicit-artifact processes). But these are only part of the matching processes between the deeper human processes that direct a given word to be typed and the deeper artifact processes that actually imprint the ink marks on the paper.
The outside world interacts with our H-LAM/T system by the exchange of energy with either the individual or his artifact. Again, special processes are often designed to accommodate this exchange. However, the direct concern of our present study lies within the system, with the internal processes that are and can be significantly involved in the effectiveness of the system in developing the human's comprehension and pursuing the human's goals.
Concepts, Symbols, and a Hypothesis
Before we pursue further direct discussion of the H-LAM/T system, let us examine some background material. Consider the following historical progression in the development of our intellectual capabilities:
- Concept Manipulation--Humans rose above the lower forms of life by evolving the biological capability for developing abstractions and concepts. They could manipulate these concepts within their minds to a certain extent, and think about situations in the abstract. Their mental capabilities allowed them to develop general concepts from specific instances, predict specific instances from general concepts, associate concepts, remember them, etc. We speak here of concepts in their raw, unverbalized form. For example, a person letting a door swing shut behind him suddenly visualizes the person who follows him carrying a cup of hot coffee and some sticky pastries. Of all the aspects of the pending event, the spilling of the coffee and the squashing of the pastry somehow are abstracted immediately, and associated with a concept of personal responsibility and a dislike for these consequences. But a solution comes to mind immediately as an image of a quick stop and an arm stab back toward the door, with motion and timing that could prevent the collision, and the solution is accepted and enacted. With only non-symbolic concept manipulation, we could probably build primitive shelter, evolve strategies of war and hunt, play games, and make practical jokes. But further powers of intellectual effectiveness are implicit in this stage of biological evolution (the same stage we are in today).
- Symbol Manipulation--Humans made another great step forward when they learned to represent particular concepts in their minds with specific symbols. Here we temporarily disregard communicative speech and writing, and consider only the direct value to the individual of being able to do his heavy thinking by mentally manipulating symbols instead of the more unwieldly concepts which they represent. Consider, for instance, the mental difficulty involved in herding twenty- seven sheep if, instead of remembering one cardinal number and occasionally counting, we had to remember what each sheep looked like, so that if the flock seemed too small we could visualize each one and check whether or not it was there.
- Manual, External, Symbol Manipulation--Another significant step toward harnessing the biologically evolved mental capabilities in pursuit of comprehension and problem solutions came with the development of the means for externalizing some of the symbol-manipulation activity, particularly in graphical representation. This supplemented the individual's memory and ability to visualize. (We are not concerned here with the value derived from human cooperation made possible by speech and writing, both forms of external symbol manipulation. We speak of the manual means of making graphical representations of symbols--a stick and sand, pencil and paper and eraser, straight edge or compass, and so on.) It is principally this kind of means for external symbol manipulation that has been associated with the evolution of the individual's present way of doing his concept manipulation (thinking).
It is undoubtedly true that concepts which people found useful ended up being symbolized in their language, and hence that the evolution of language was affected by the concepts the people developed and used. However, Korzybski (4) and Whorf (5) (among others) have argued that the language we use affects our thinking to a considerable extent. They say that a lack of words for some types of concepts makes it hard to express those concepts, and thus decreases the likelihood that we will learn much about them. If this is so, then once a language has begun to grow and be used, it would seem reasonable to suspect that the language also affects the evolution of the new concepts to be expressed in that language.
Apparently there are counter-arguments to this; e.g., if a concept needs to be used often but its expression is difficult, then the language will evolve to ease the situation. However, the studies of the past decade into what are called "self-organizing" systems seem to be revealing that subtle relationships among its interacting elements can significantly influence the course of evolution of such a system. If this is true, and if language is (as it seems to be) a part of a selforganizing system, then it seems probable that the state of a language at a given time strongly affects its own evolution to a succeeding state.
For our conceptual framework, we tend to favor the view that a language does exert a force in its own evolution. We observe that the shift over the last few centuries in matters that are of daily concern to the individual has necessarily been forced into the framework of the language existing at the time, with alterations generally limited to new uses for old words, or the coining of new words. The English language since Shakespeare has undergone no alteration comparable to the alteration in the cultural environment; if it had, Shakespeare would no longer be accessible to us. Under such evolutionary conditions, it would seem unlikely that the language we now use provides the best possible service to our minds in pursuing comprehension and solving problems. It seems very likely that a more useful language form can be devised.
The Whorfian hypothesis states that the world view of a culture is limited by the structure of the language which that culture uses. But there seems to be another factor to consider in the evolution of language and human reasoning ability. We offer the following hypothesis, which is related to the Whorfian hypothesis: Both the language used by a culture, and the capability for effective intellectual activity are directly affected during their evolution by the means by which individuals control the external manipulation of symbols. (For identification, we will refer to this as the Neo-Whorfian hypothesis.)
If the Neo-Whorfian hypothesis could be proved readily, and if we could see how our means of externally manipulating symbols influence both our language and our way of thinking, then we would have a valuable instrument for studying human-augmentation possibilities. For the sake of discussion, let us assume the Neo-Whorfian hypothesis to be true, and see what relevant deductions can be made.
If the means evolved for an individual's external manipulation of his thinking-aid symbols indeed directly affect the way in which he thinks, then the original Whorfian hypothesis would offer an added effect. The direct effect of the external-symbol-manipulation means upon language would produce an indirect effect upon the way of thinking via the Whorfian-hypothesis linkage. There would then be two ways for the manner in which our external symbol manipulation was done to affect our thinking.
One way of viewing the H-LAM/T system changes that we contemplate--specifically, integrating the capabilities of a digital computer into the intellectual activity of individual humans--is that we are introducing new and extremely advanced means for externally manipulating symbols. We then want to determine the useful modifications in the language and in the way of thinking that could result. This suggests a fourth stage to the evolution of our individual-human intellectual capability:
- 4. Automated external symbol manipulation--In this stage, symbols with which the human represents the concepts he is manipulating can be arranged before his eyes, moved, stored, recalled, operated upon according to extremely complex rules--all in very rapid response to a minimum amount of information supplied by the human, by means of special cooperative technological devices. In the limit of what we might now imagine, this could be a computer, with which we could communicate rapidly and easily, coupled to a three-dimensional color display within which it could construct extremely sophisticated images--with the computer being able to execute a wide variety of processes upon parts or all of these images in automatic response to human direction. The displays and processes could provide helpful services--we could imagine both simple and exotic varieties--and could involve concepts that we have never yet imagined (as the pregraphic thinker of Stage 2 would be unable to predict the bar graph, the process of long division, or a card file system).
These hypotheses imply great richness in the new evolutionary spaces opened by progressing from Stage 3 to Stage 4. We would like to study the hypotheses further, examining their possible manifestations in our experience, ways of demonstrating their validity, and possible deductions relative to going to Stage 4.
In search of some simple ways to determine what the Neo-Whorfian hypothesis might imply, we could imagine some relatively straightforward means of increasing our external symbol-manipulation capability and try to picture the consequent changes that could evolve in our language and methods of thinking. Actually, it turned out to be simpler to invert the problem and consider a change that would reduce our capability for external symbol manipulation. This allowed an empirical approach which proved both simple and effective. We thus performed the following experiment.
Brains of power equal to ours could have evolved in an environment where the combination of artifact materials and muscle strengths were so scaled that the neatest scribing tool (equivalent to a pencil, possible had a shape and mass as manageable as a brick would be to us-assuming that our muscles were not specially conditioned to deal with it. We fastened a pencil to a brick and experimented. Figure 2 shows the results, compared with typewriting and ordinary pencil writing. With the brick pencil, we are slower and less precise. If we want to hurry the writing, we have to make it larger. Also, writing the passage twice with the brick-pencil tires the untrained hand and arm.
How would our civilization have matured if this had been the only manual means for us to use in graphical manipulation of symbols? For one thing, the record keeping that enables the organization of commerce and government would probably have taken a form so different from what we know that our social structure would undoubtedly have evolved differently. Also, the effort in doing calculations and writing down extensive and carefully reasoned argument would dampen individualexperimentation with sophisticated new concepts, to lower the rate of learning and the rate of useful output, and perhaps to discourage a good many people from even working at extending understanding. The concepts that would evolve within our culture would thus be different, and very likely the symbology to represent them would be different--much more economical of motion in their writing It thus seems very likely that our thoughts and our language would be rather directly affected by the particular means used by our culture for externally manipulating symbols, which gives little intuitive substantiation to our Neo-Whorfian hypothesis.
Fig. 2: Experimental Results of Tying a Brick to a Pencil to "De-Augment" the Individual
To reflect further upon the implications of this hypothesis, the following hypothetical artifact development can be considered, representing a diiferent type of external symbol manipulation that could have had considerable effect. Suppose that our young technology of a few generations ago had developed an artifact that was essentially a high speed, semi-automatic table-lookup device--cheap enough for almost everyone to afford and small and light enough to be carried on the person. Assume that the individual cartridges sold by manufacturers (publishers) contained the look-up information, that one cartridge could hold the equivalent of an unabridged dictionary, and that a one-paragraph definition could always be located and displayed on the face of the device by the average practised individual in less than three seconds. The fortunes of technological invention, commercial interest, and public acceptance just might have evolved something like this.
If it were so very easy to look things up, how would our vocabulary develop, how would our habits of exploring the intellectual domains of others shift, how might the sophistication of practical organization mature (if each person can so quickly and easily look up applicable rules), how would our education system change to take advantage of this new external symbol-manipulation capability of students and teachers (and administrators)?
The significance to our study of the discussion in this section lies in the perspective it gives to the ways in which human intellectual effectiveness can be affected by the particular means used by individuals for their external symbol manipulation. It seems reasonable to consider the development of automated external symbol manipulation means as a next stage in the evolution of our intellectual power.
Capability Repertoire Hierarchy
The concept of our H-LAM/T system possessing a repertoire of capabilities that is structured in the form of a hierarchy is most useful in our study. We shall use it in the following to tie together a number of considerations and concepts. There are two points of focus in considering the design of new repertoire hierarchies: the materials with which we have to work, and the principles by which new capability is constructed from these basic materials.
a. Basic Capabilities
"Materials" in this context are those capabilities in the human and in the artifact domains from which all other capabilities in the repertoire hierarchy must be constructed. Each such basic capability represents a type of functional component with which the system can be built, and a thorough job of redesigning the system calls for making an inventory of the basic capabilities available. Because we are exploring for perspective, and not yet recommending research activities, we are free to discuss and define in more detail what we mean by "basic capability", without regard to the amount of research involved in making an actual inventory.
The two domains, human and artifact, can be explored separately for their basic capabilities, In each we can isolate two classes of basic capability; these classes are distinguished according to whether or not the capability has been put to use within out augmentation means. The first class (those in use) can be found in a methodical manner by analyzing present capability hierarchies. For example, select a given capability, at any level in the hierarchy, and ask yourself if it can be usefully changed by any means that can be given consideration in the augmentation research contemplated, If it can, then it is not basic but it can be decomposed into an eventual set of basic capabilities. As you proceed down through the hierarchy, you will begin to encounter capabilities that cannot be usefully changed, and these will make up your inventory of basic capabilities. Ultimately, every such recursive decomposition of a given capability in the hierarchy will find every one of its branching paths terminated by basic capabilities. Beginning such decomposition search with different capabilities in the hierarchy will eventually uncover all of those basic capabilities used within that hierarchy or augmentation system. Many of the branching paths in the decomposition of a given higher-order capability will terminate in the same basic capability, since a given basic capability will often be used within many different higher-order capabilities.
Determining the class of basic capabilities not already utilized within existing augmentation systems requires a different exploration method. Examples of this method occur in technological research, where analytically oriented researchers search for new understandings of phenomena that can add to the research engineer's list of things to be used in the synthesis of better artifacts.
Before this inventorying task can be pursued in any specific instance, some criteria must be established as to what possible changes within the H-LAM/T system can be given serious consideration. For instance, some research situations might have to disallow changes which require extensive retraining, or which require undignified behavior by the human. Other situations might admit changes requiring years of special training, very expensive equipment, or the use of special drugs.
The capability for performing a certain finger action, for example, may not be basic in our sense of the word. Being able to extend the finger a certain distance would be basic but the strength and speed of a particular finger motion and its coordination with higher actions generally are usefully changeable and therefore do not represent basic capabilities. What would be basic in this case would perhaps be the processes whereby strength could be increased and coordinated movement patterns learned, as well as the basic movement range established by the mechanical-limit loci of the muscle-tendon-bone system. Similar capability breakdowns will occur for sensory and cognitive capabilities.
b Structure Types
The fundamental principle used in building sophisticated capabilities from the basic capabilities is structuring--the special type of structuring (which we have termed synergetic) in which the organization of a group of elements produces an effect greater than the mere addition of their individual effects. Perhaps "purposeful" structuring (or organization) would serve us as well, but since we aren't sure yet how the structuring concept must mature for our needs, we shall tentatively stick with the special modifier, "synergetic." We are developing a growing awareness of the significant and pervasive nature of such structure within every physical and conceptual thing we inspect, where the hierarchical form seems almost universally present as stemming from successive levels of such organization.
The fundamental entities that are being structured in each and every case seems to be what we could call processes, where the most basic of physical processes (involving fields, charges, and momenta associated with the dynamics of fundamental particles) appear to be the hierarchical base. There are dynamic electro-optical-mechanical processes associated with the function of our artifacts (as well as metabolic, sensory, motor) and cognitive processes of the human, which we find to be relatively fundamental components within the structure of our H-LAM/T system--and each of these seems truly to be ultimately based (to our degree of understanding) upon the above mentioned basic physical processes. The elements that are organized to give fixed structural form to our physical objects--e.g., the "element" of tensile strength of a material-are also derived from what we could call synergetic structuring of the most basic physical processes.
But at the level of the capability hierarchy where we wish to work, it seems useful to us to distinguish several different types of structuring--even though each type is fundamentally a structuring of the basic physical processes. Tentatively we have isolated five such types--although we are not sure how many we shall ultimately want to use in considering the problem of augmenting the human intellect, nor how we might divide and subdivide these different manifestations of physical-process structuring. We use the terms "mental structuring", "concept structuring", "symbol structuring", "process structuring," and "physical structuring."
2) Mental Structuring
Mental structuring is what we call the internal organization of conscious and unconscious mental images, associations, or concepts (or whatever it is that is organized within the human mind) that somehow manages to provide the human with understanding and the basis for such as judgment, intuition, inference, and meaningful action with respect to his environment. There is a term used in psychology, cognitive structure, which so far seems to represent just what we want for our concept of mental structure, but we will not adopt it until we become more sure of what the accepted psychological meaning is and of what we want for our conceptual framework.
For our present purpose, it is irrelevant to worry over what the fundamental mental "things" being structured are, or what mechanisms are accomplishing the structuring or making use of what has been structured. We feel reasonably safe in assuming that learning involves some kind of meaningful organization within the brain, and that whatever is so organized or structured represents the operating model of the individual's universe to the mental mechanisms that derive his behavior. And further, our assumption is that when the human in our H/LAM system makes the key decision or action that leads to the solution of a complex problem, it will stem from the state of his mental structure at that time. In this view then, the basic purpose of the system's activity on that problem up to that point has been to develop his mental structure to the state from which the mental mechanisms could derive the key action.
Our school systems attest that there are specific experiences that can be given to a human that will result in development of his mental structure to the point where the behavior derived there from by his mental mechanisms shows us that he has gained new comprehension--in other words, we can do a certain amount from outside the human toward developing his mental structure. Independent students and researchers also attest that internally directed behavior on the part of an individual can directly aid his structure-building process.
We don't know whether a mental structure is developed in a manner analogous to (a) development of a garden, where one provides a good environment, plants the seeds, keeps competing weeds and injurious pests out, but otherwise has to let natural processes take their course, or to (b) development of a basketball team, where much exercise of skills, patterns, and strategies must be provided so that natural processes can slowly knit together an integration, or to (c) development of a machine, where carefully formed elements are assembled in a precise, planned manner so that natural phenomena can immediately yield planned function. We don't know the processes, but we can and have developed empirical relationships between the experiences given a human and the associated manifestations of developing comprehension and capability, and we see the near-future course of the research toward augmenting the human's intellect as depending entirely upon empirical findings (past and future) for the development of better means to serve the development and use of mental structuring in the human.
We don't mean to imply by this that we renounce theories of mental processes. What we mean to emphasize is that pursuit of our objective need not wait upon the understanding of the mental processes that accomplish (what we call) mental structuring and that derive behavior therefrom. It would be to ignore the emphases of our own conceptual framework not to make fullest use of any theory that provided a working explanation for a group of empirical data. What's more, our entire conceptual framework represents the first pass at a "theoretical model with which to organize our thinking and action."
3) Concept Structuring
Within our framework we have developed the working assumption that the manner in which we seem to be able to provide experiences that favor the development of our mental structures is based upon concepts as a "medium of exchange." We view a concept as a tool that can be grasped and used by the mental mechanisms, that can be composed, interpreted, and used by the natural mental substances and processes. The grasping and handling done by these mechanisms can often be facilitated if the concept is given an explicit "handle" in the form of a representative symbol. Somehow the mental mechanisms can learn to manipulate images (or something) of symbols in a meaningful way and remain calmly confident that the associated conceptual manipulations are within call.
Concepts seem to be structurable, in that a new concept can be composed of an organization of established concepts. For present purposes, we can view a concept structure as something which we might try to develop on paper for ourselves or work with by conscious thought processes, or as something which we try to communicate to one another in serious discussion. We assume that, for a given unit of comprehension to be imparted, there is a concept structure (which can be consciously developed and displayed) that can be presented to an individual in such a way that it is mapped into a corresponding mental structure which provides the basis for that individual's "comprehending" behavior. Our working assumption also considers that some concept structures would be better for this purpose than others, in that they would be more easily mapped by the individual into workable mental structures, or in that the resulting mental structures enable a higher degree of comprehension and better solutions to problems, or both.
A concept structure often grows as part of a cultural evolution--either on a large scale within a large segment of society, or on a small scale within the activity domain of an individual. But it is also something that can be directly designed or modified, and a basic hypothesis of our study is that better concept structures can be developed-- structures that when mapped into a human's mental structure will significantly improve his capability to comprehend and to find solutions within his complex-problem situations.
A natural language provides its user with a readymade structure of concepts that establishes a basic mental structure, and that allows relatively flexible, general-purpose concept structuring. Our concept of language as one of the basic means for augmenting the human intellect embraces all of the concept structuring which the human may make use of.
4) Symbol Structuring
The other important part of our "language" is the way in which concepts are represented--the symbols and symbol structures. Words structured into phrases, sentences, paragraphs, monographs--charts, lists, diagrams, tables, etc. A given structure of concepts can be represented by any of an infinite number of different symbol structures, some of which would be much better than others for enabling the human perceptual and cognitive apparatus to search out and comprehend the conceptual matter of significance and/or interest to the human. For instance, a concept structure involving many numerical data would generally be much better represented with Arabic rather than Roman numerals and quite likely a graphic structure would be better than a tabular structure.
But it is not only the form of a symbol structure that is important. A problem solver is involved in a stream of conceptual activity whose course serves his mental needs of the moment. The sequence and nature of these needs are quite variable, and yet for each need he may benefit significantly from a form of symbol structuring that is uniquely efficient for that need.
Therefore, besides the forms of symbol structures that can be constructed and portrayed, we are very much concerned with the speed and flexibility with which one form can be transfcrmed into another, and with which new material can be located and portrayed.
We are generally used to thinking of our symbol structures as a pattern of marks on a sheet of paper. When we want a different symbol-structure view, we think of shifting our point of attention on the sheet, or moving a new sheet into position. But another kind of view might be obtained by extracting and ordering all statements in the local text that bear upon consideration A of the argument--or by replacing all occurrences of specified esoteric words by one's own definitions. This sort of "view generation" becomes quite feasible with a computer-controlled display system, and represents a very significant capability to build upon.
With a computer manipulating our symbols and generating their portrayals to us on a display, we no longer need think of our looking at the symbol structure which is stored--as we think of looking at the symbol structures stored in notebooks, memos, and books. What the computer actually stores need be none of our concern, assuming that it can portray symbol structures to us that are consistent with the form in which we think our information is structured.
A given concept structure can be represented with a symbol structure that is completely compatible with the computer's internal way of handling symbols, with all sorts of characteristics and relationships given explicit identifications that the user may never directly see. In fact, this structuring has immensely greater potential for accurately mapping a complex concept structure than does a structure an individual would find it practical to construct or use on paper.
The computer can transform back and forth between the two-dimensional portrayal on the screen, of some limited view of the total structure, and the aspect of the n-dimensional internal image that represents this "view". If the human adds to or modifies such a "view," the computer integrates the change into the internal-image symbol structure (in terms of the computer's favored symbols and structuring) and thereby automatically detects a certain proportion of his possible conceptual inconsistencies.
Thus, inside this instrument (the computer) there is an internal-image, computer-symbol structure whose convolutions and multi-dimensionality we can learn to shape to represent to hitherto unattainable accuracy the concept structure we might be building or working with. This interna1 structure may have a form that is nearly incomprehensible to the direct inspection of a human (except in minute chunks).
But let the human specify to the instrument his particular conceptual need of the moment, relative to this internal image. Without disrupting its own internal reference structure in the slightest, the computer will effectively stretch, bend, fold, extract, and cut as it may need in order to assemble an internal substructure that is its respons, structured in its own internal way. With the set of standard translation rules appropriate to the situation, it portrays to the human via its display a symbol structure designed for his quick and accurate perception and comprehension of the conceptual matter pertinent to this internally composed substructure.
No longer does the human work on stiff and limited symbol structures, where much of the conceptual content can only be implicitly designated in an indirect and distributed fashion. These new ways of working are basically available with today's technology--we have but to free ourselves from some of our limiting views and begin experimenting with compatible sets of structure forms and processes for human concepts, human symbols, and machine symbols.
5) Process Structuring
Essentially everything that goes on within the H-LAM/T system and that is of direct interest here involves the manipulation of concept and symbol structures in service to the mental structure. Therefore, the processes within the H-LAM/T system that we are most interested in developing are those that provide for the manipulation of all three types of structure. This brings us to the fourth category of structuring, process structuring.
As we are currently using it, the term
includes the organization, study, modification, and execution of processes and process structures. Whereas concept structuring and symbol structuring together represent the language component of our augmentation means, process structuring represents the methodology component (plus a little more, actually). There has been enough previous discussion of process structures that we need not describe the notion here, beyond perhaps an example or two. The individual processes (or actions) of my hands and fingers have to be cooperatively organized if the typewriter is to do my bidding. My successive actions throughout my working day are meant to cooperate toward a certain over-all professional goal.
Many of the process structures are applied to the task of organizing, executing, supervising, and evaluating other process structures. Many of them are applied to the formation and manipulation of symbol structures (the purpose of which will often be to support the conceptual labor involved in process structuring).
6) Physical Structuring
Physical structuring, the last of the five types which we currently use in our conceptual framework, is nearly self-explanatory. It pretty well represents the artifact component of our augmentation means, insofar as their actual physical construction is concerned.
7) Interdependence and Regeneration
A very important feature to be noted from the discussion in this section bears upon the interdependence among the various types of structuring which are involved in the H-LAM/T system, where the capability for doing each type of structuring is dependent upon the capability for doing one or more of the other types of structuring. (Assuming that the physical structuring of the system remains basically unchanged during the system's operation, we exclude its dependence upon other factors in this discussion.)
This interdependence actually has a cyclic, regenerative nature to it which is very significant to us. We have seen how the capability for mental structuring is finally dependent, down the chain, upon the process structuring (human, artifact, composite) that enables symbol-structure manipulation. But it also is evident that the process structuring is dependent not only upon basic human and artifact process capabilities, but upon the ability of the human to learn how to execute processes--and no less important, upon the ability of the human to select, organize, and modify processes from his repertoire to structure a higher-order process that he can execute. Thus, a capability for structuring and executing processes is partially dependent upon the human's mental structuring, which in turn is partially dependent upon his process structuring (through concept and symbol structuring), which is partially dependent upon his mental structuring, etc.
All of this means that a significant improvement in symbol-structure manipulation through better process structuring (initially perhaps through much better artifacts) should enable us to develop improvements in concept and mental-structure manipulations that can in turn enable us to organize and execute symbol-manipulation processes of increased power. To most people who initially consider the possibilities for computer-like devices augmenting the human intellect, it is only the one-pass improvement that comes to mind, which presents a picture that is relatively barren compared to that which emerges when one considers this regenerative interaction.
We can confidently expect the development of much more powerful concepts pertaining to the manner in which symbol structures can be manipulated and portrayed, and correspondingly more complex manipulation processes that in the first pass would have been beyond the human's power to organize and execute without the better symbol, concept, and mental structuring which his augmented system provided him. These new concepts and processes, beyond our present capabilities to use and thus never developed, will provide a tremendous increased-capability payoff in the future development of our augmentation means.
c. Roles and Levels
In the repertoire hierarchy of capabilities possessed by the H-LAM/T system, the human contributes many types of capability that represent a wide variety of roles. At one time or another he will be the policy maker, the goal setter, the performance supervisor, the work scheduler, the professional specialist, the clerk, the janitor, the entrepreneur, and the proprietor (or at least a major stockholder) of the system. In the midst of some complex process, in fact, he may well be playing several roles concurrently--or at least have the responsibility of the roles. For instance, usually he must be aware of his progress toward a goal (supervisor), he must be alert to the possibilities for changing the goal (policy maker, planner), and he must keep records for these and other roles (clerk).
Consider a given capability (Capability 1) at some level in the repertoire hierarchy. There seems to be a sort of standard grouping of lower-order capabilities from which this is composed, and these exist in two classes--what we might call the executive class and what we might call the direct-contributive class. In the executive class of capabilities we find those used for comprehending, planning, and executing the process represented by Capability 1. In the direct-contributive class we find the capabilities organized by the executive class toward the direct realization of Capability 1. For example, when my telephone rings, I execute the direct-contributive processes of picking up the receiver and saying hello. It was the executive processes that comprehended the situation, directed a lower-order executive-process that the receiver be picked up and, when the receiver was in place (first process accomplished), directed the next process, the saying hello. That represents the composition of my capability for answering the phone.
For a low-level capability, such as that of writing a word with a pencil, both the executive and the direct-contributive subprocesses during actual execution would be automatic. This type of automatic capability need only be summoned by a higher executive process in order for trained automatic responses to execute it.
At a little higher level of capability, more of the conscious conceptual and executive capabilities become involved. To call someone on the telephone, I must consciously comprehend the need for this process and how I can execute it, I must consciously pick up the directory and search for the name and telephone number, and I must consciously direct the dialing of the number.
At a still higher level of capability, the executive capabilities must have a degree of power that unaided mental capabilities cannot provide. In such a case, one might make a list of steps and check each item off as it is executed. For an even more complex process, comprehending the particular situation in which it is to be executed, even before beginning to plan the execution, may take months of labor and a very complex organization of the system's capabilities.
Imagining a process as complex as the last example brings us to the realization that at any particular moment the H-LAM/T system may be in the middle of executing a great number of processes. Assume that the human is in the middle of the process of making a telephone call. That telephone call is a subprocess in the middle of the process of calling a committee meeting. But calling a committee meeting is a subprocess in the middle of the process of determining a budgetary policy, which is in turn but a subprocess in the middle of the process of estimating manpower needs, and so on.
Not only does the human need to play various roles (sometimes concurrently) in the execution of any given process, but he is playing these roles for the many concurrent processes that are being executed at different levels. This situation is typical for any of us engaged in reasonably demanding types of professional pursuits, and yet we have never received explicit training in optimum ways of carrying out any but a very few of the roles at a very few of the levels. A well-designed H-LAM/T system would provide explicit and effective concepts, terms, equipment, and methods for all these roles, and for their dynamic coordination.
d. Model of Executive Superstructure
It is the repertoire hierarchy of process capabilities upon which the ultimate capability of the H-LAM/T system rests. This repertoire hierarchy is rather like a mountain of white-collar talent that sits atop and controls the talents of the workers. We can illustrate this executive superstructure by considering it as though it were a network of contractors and subcontractors in which each capability in the repertoire hierarchy is represented by an independent contractor whose mode of operation is to do the planning, make up specifications, subcontract the actual work, and supervise the performance of his subcontractors. This means that each subcontractor does the same thing in his turn. At the bottom of this hierarchy are those independent contractors who do actual "production work."
If by some magical process the production workers could still know just what to do and when to do it even though the superstructure of contractors was removed from above them, no one would know the difference. The executive superstructure is necessary because humans do not operate by magic, but even a necessary superstructure is a burden. We can readily recognize that there are many ways to organize and manage such a superstructure, resulting in vastly different degrees of efficiency in the application of the workers' talents.
Suppose that the activity of the production workers was of the same nature as the activity of the different contractors, and that this activity consisted of gaining comprehension and solving problems. And suppose that there was only so much applicable talent available to the total system. The question now becomes how to distribute that talent between superstructure and workers to get the most total production. The efficiency of organization within the superstructure is now doubly important so that a minimum of talent in the superstructure produces a maximum of organizational efficiency in directing the productivity of the remaining talent.
In the situation where talent is limited, we find a close parallel to our H-LAM/T system in its pursuit of comprehension and problem solutions. We obtain an even closer parallel if we say that the thinking, planning, supervising, record keeping, etc., for each contractor is actually done by a single individual for the whole superstructure, time- sharing his attention and talents over these many tasks. Today this individual cannot be depended upon to have any special training for many of these roles; he is likely to have learned them by cut and try and by indirect imitation.
A complex process is often executed by the H-LAM/T system in a multi-pass fashion (i.e., cut and try). In really complex situations, comprehension and problem solutions do not stand waiting at the end of a straightforward path; instead, possibilities open up and plans shift as comprehension grows. In the model using a network of contractors, this type of procedure would entail a great deal of extra work within the superstructure--each contractor involved in the process would have the specifications upon which he bid continually changed, and would continually have to respond to the changes by restudying the situation, changing his plans, changing the specifications to his subcontractors, and changing his records. This is a terrific additional burden, but it allows a freedom of action that has tremendous importance to the effectiveness the system exhibits to the outside world.
We could expect significant gains from automating the H- LAM/T system if a computer could do nothing more than increase the effectiveness of the executive processes. More human time, energy, and productive thought could be allocated to direct-contributive processes, which would be coordinated in a more sophisticated, flexible and efficient manner. But there is every reason to believe that the possibilities for much-improved symbol and process structuring that would stem from this automation will directly provide improvements in both the executive and direct-contributive processes in the system.
e. Flexibility in the Executive Role
The executive superstructure is a necessary component in the H-LAM/T system, and there is finite human capability which must be divided between executive and direct-contributive activities. An important aspect of the multi-role activity of the human in the system is the development and manipulation of the symbol structures associated with both his direct-contributive roles and his executive roles.
When the system encounters a complex situation in which comprehension and problem solutions are being pursued, the direct-contributive roles require the development of symbol structures that portray the concepts involved within the situation. But executive roles in a complex problem situation also require conceptual activity--e.g., comprehension, selection, supervision--that can benefit from well-designed symbol structures and fast, flexible means for manipulating and displaying them. For complex processes, the executive problem posed to the human (of gaining the necessary comprehension and making a good plan) may be tougher than the problem he faced in the role of direct-contributive worker. If the flexibility desired for the process hierarchies (to make room for human cut-and-try methods) is not to be degraded or abandoned, the executive activity will have to be provided with fast and flexible symbol-structuring techniques.
The means available to humans today for developing and manipulating these symbol structures are both laborious and inflexible. It is hard enough to develop an initial structure of diagrams and text, but the amount of effort required to make changes is often prohibitively great; one settles for inflexibility. Also, the kind of generous flexibility that would be truly helpful calls for added symbol structuring just to keep track of the trials, branches, and reasoning thereto that are involved in the development of the subject structure; our present symbol-manipulation means would very soon bog down completely among the complexities that are involved in being more than just a little bit flexible.
We find that the humans in our H-LAM/T systems are essential working continuously within a symbol structure of some sort, shifting their attention from one structure to another as they guide and execute the processes that ultimately provide them with the comprehension and the problem solutions that they seek. This view increases our respect for the essential importance of the basic capability of composing and modifying efficient symbol structures. Such a capability depends heavily upon the particular concepts that are isolated and manipulated as entities, upon the symbology used to represent them, upon the artifacts that help to manipulate and display the symbols, and upon the methodology for developing and using symbol structures. In other words, this capability depends heavily upon proper language, artifacts, and methodology, our basic augmentation means.
When the course of action must respond to new comprehension, new insights and new intuitive flashes of possible explanations or solutions, it will not be an orderly process. Existing means of composing and working with symbol structures penalize disorderly processes very heavily, and it is part of the real promise in the automated H-LAM/T systems of tomorrow that the human can have the freedom and power of disorderly processes.
f. Compound Effects
Since many processes in many levels of the hierarchy are involved in the execution of a single higher-level process of the system, any factor that influences process execution in general will have a highly compounded total effect upon the system's performance. There are several such factors which merit special attention.
Basic human cognitive powers, such as memory intelligence, or pattern perception can have such a compounded effect. The augmentation means employed today have generally evolved among large statistical populations, and no attempt has been made to fit them to individual needs and abilities. Each individual tends to evolve his own variations, but there is not enough mutation and selection activity, nor enough selection feedback, to permit very significant changes. A good, automated H-LAM/T system should provide the opportunity for a significant adaptation of the augmentation means to individual characteristics. The compounding effect of fundamental human cognitive powers suggests further that systems designed for maximum effectiveness would require that these powers be developed as fully as possible--by training, special mental tricks, improved language, new methodology.
In the automated system that we contemplate, the human should be able to draw on explicit-artifact process capability at many levels in the repertoire hierarchy; today, artifacts are involved explicitly in only the lower-order capabilities. In the future systems, for instance, it should be possible to have computer processes provide direct and significant help in his processes at many levels. We thus expect the effect of the computer in the system to be very much compounded. A great deal of richness in the future possibilities for automated H-LAM/T systems is implied here--considerably more than many people realize who would picture the computer as just helping them do the things they do now. This type of compounding is related to the reverberating waves of change discussed in Section II-A.
Another factor can exert this type of compound effect upon over-all system performance: the human's unconscious processes. Clinical psychology seems to provide clear evidence that a large proportion of a human's everyday activity is significantly mediated or basically prompted by unconscious mental processes that, although "natura" in a functional sense, are not rational. The observable mechanisms of these processes (observable by another, trained person) includes masking of the irrationality of the human's actions which are so affected, so that few of us will admit that our actions might be irrational, and most of us can construct satisfying rationales for any action that may be challenged.
Anything that might have so general an effect upon our mental actions as is implied here, is certainly a candidate for ultimate consideration in the continuing development of our intellectual effectiveness. It may be that the first stages of research on augmenting the human intellect will have to proceed without being able to do anything about this problem except accommodate to it as well as possible. This may be one of the very significant problems whose solution awaits our development of increased intellectual effectiveness.