Guohua Bai (Ph.D. Guest Professor)



My reading notes



Notes of Social-technical Psychology Study


Norman, Donald (19++)

Cognitive Engineering

"Psychological variables differ from physical variables. The persone starts with goals and intention. These are psychological variables. They exist in the mind of the person and they relate directly to the needs and concerns of the person. However, the task is to be performed on a physical system, which phtsical mechanisms to be manipulated, resulting in changes to the physical variables and system state. Thus the person must interpret the physical variables into terms relevant to psychological goals and must translate the psychological intentions into physical actions upon the mechanisms. This means that there must be a stage of interpretation that relates physical and psychological variables, as well as functions that relate the manipulation of the physical variables to the resulting change in physical state." (p33) (related to activity theory model in levels of : activity-motive; action-goal; operation-condition) Consider an example of writing a letter by using word-processing program, activity, actions, operations levels up and down .

The difference between desired goal and current state gives rise to an intention, again stated in psychological terms. This must get translated into an action sequence, the specification of what physical acts will be performed upon the mechanisms of the system.

The gulf of execution and evaluation: Goals and system state differ significantly in form and content, creating the Gulfs that need to be bridged if the system can be used. The gulfs can be bridged from both sides of physical world (by starting from systems interface)or psychological world (by starting from users' intention) (in activity theory, this is not bridged, but mediated by tools)

Norman uses user's model, designer's model, and system image to describe the process of interaction.

"mental models seem a pervasive property of humans.... These models provide predictive and explanatory power for understanding interaction. Mental models evolve naturally through interaction with the world and with the particular system under consideration. These models are highly affected by the nature of the interaction, coupled with the person's prior knowledge and understanding. The models are neither complete nor accurate, but nonetheless they function to guide much human behavior." (p46).

User's model results from the way the user interprets the system image. Thus in many ways, the primary task of the designer is to construct an appropriate system image.

In general, the more specialized the tool, the higher the level at which a system operates, the easier the task. Spreadsheets are relatively straightforward. Whenever there is one single task and one set of users, the tak of developing the conceptual model is much simplified. When the system is general purpose, with a relatively unlimited set of users and power, then the task becomes complex, perhaps undoable. (48)

Any real system is the result of a series of tradeoffs that balance one design decision against another, that take into account time, effort, and expense. for instance, "it is often stated that current computer systems do not provide beginning users with sufficient information. However, the long, informative displays or sequence of questions, options, or menus that may make a system usable by beginner are disruptive to the experienced user who knows exactly what action is to be specified and wishes to minimize the time and mental effort required to do the specification."(p57).

"It takes at least three kinds of special knowledge to design an interface: first, knowledge of design, of programming and of the technology; second, knowledge of people, of the principles of mental computation of communication, and of interaction; and third, expert knowledge of the task that is to be accomplished . Most programmers and designersof computer systems have the first kind of knowledge, but not the second or third. Most psychologists have the second, but not the first or third. And potential user is apt to have the third, but not the first or second. As a result, if a computer system is to be constructured with truly user- centered design, it will have to be done in collaboration with people trained in all these areas. (p60)."

Bai's summary from readings

Cognitive Approach to Design of Human-Computer Interface

Key Words: Knowledge-representation, Design activities, Metaphors, prototyping, analogical reasoning, man-machine interfaces.


In the area of human-computer interface design, much have been tried to make the machine, a black-box, as transparent as possible to the end users. Metaphor and prototyping approaches have become prominent in the effort since some of declared advantages from cognitive aspects. However, the effort on design, even trying to take into account the human aspects, has primarily taken the form of either designer´s own reasonings and/or laboratory tests. (Krarup,1994). This paper suggests a work-oriented cognitive approach to the discipline of human-computer interface design. In the first part, a theoretical understanding of human perception, recognition, memory, and analogical reasoning based on cognitive psychology. In the second part of the paper, a concrete example is provided to supplement the approach. Logical positivism maintained that theories were only to be justified by an appeal to observed facts, and that theoretical constructs were meaningful only to the extent that they could be observed. In this approach, the emphasis was on the relationship between observable stimuli and observable response (input-output). in the 20th-centry, karl popper argued that observation was not as objective as it seemed, but was acrually rather theory driven. no one ever observes without some idea what it is they are looking for. Scientific observation is always driven by hypotheses and theories, and what you observe depends in part on what you expect to see.

Thomas Kuhn (1977) argued that science has two modes (normal and revolutionary science), and that the transition from one mode to the other involves an intellectural crisis in a discipline. During periods of normal science, a discipline is dominated by a particular theoretical orientation which Kuhn called a paradigm. This paradigm is generally accepted by scientists in the field, even though it may not account for all the available evidence brought to light by experimentation. However, the paradigm does account for the phenomena which scientists regard as being central to the field. The major activity of normal science is to perform experiments which test and elaborate the details of the paradigm. Periods of revolutionary science occur during which the old paradigm is overthrown as a consequence of the build-up of failed predictions and of gaps in its explanatory completeness. What is interesting about such crisis points is that adherents of the old paradigm resist change as long as posible until they can no longer hold out against the onslaught. When the change occurs and revolution takes place, most of the evidence accounted by the old paradigm is reinterpreted and integrated into the new paradigm. at that point, peace breaks out and normal science resumes. Example of this are the Copernican revolution, where the old Ptolemaic view that the planets and the sun revolved around the earth was replaced by our present view that the earth and the other planets revolve around the sun. In a similar fashion, Einstein's relative theory involved a re-interpretation of Newtonian mechanics, which had been the dominant view for several centuries.

Donald Broadbent (1958) argued that one could begin to understand phenomena such as perception, attention, and short-term memory by constructing an information procesing theory in which information flowed through a cognitive system. In other words, instead of looking at perception, attention, and short-term memory separately, he conidered them all as interdependent ingredients in a single cognitive system.

Psycholodists and philosopherss have always had a fondness for using the most advanced technological developments as metaphors for aspects of human functioning. Like the computer, man was viewed as an information processor. At the same time, with the growing influence of the discipline of artificial intelligence, the relationship between the mind and the computer became close. now psychologists could use computer ideas as metaphors for human thinking.

in fact, while processing is substantially affected by the nature of presented stimuli, it is also affected crucially by the individual's past experience, expectations, and so on. Information processing did not really provide a grand theory about the nature of cognition. Instead, it provided a loose set of ideas which could be used to construct theories of cognition, and so it can best be regard as a framework. Herb Simon (1980,p45) expressed "It might have been necessary a decade ago to argue for the commonality of the information process that are employed by such disparate systems as computers and human nervous systems. The evidence for that commonality is now overwhelming." But information- processing approach tends to be rather limited bacause the cognitive system is considered in isolation from motivational and emotional influences. it is also limited in that individual differences in cognitive functioning are often ignored.

Semantic Networks:

All knowledge is in the form of associations. Three main principles of association have been proposed: (1) contiguity: Two things become associated because they occurred together in time; (2) similarity: Two things become associated because they are alike; and (3) contrast: Two things become associated because they are opposite.

A semantic network consists of concepts as nodes, links as relations, and activation strengths linked each nodes. Learning is seen to change the activation values of the links between nodes. Connectionist networks can to some extent program themselves in that they can "learn" to produce specific output when certain input are given to them. When the network has learned to produce a particular response at the output layer following the presentation of a particular stimulus at the input layer, it can exhibit behaviour which looks like a rule being applied (like the production rule like If such-and such is the case Then do so-and-so.

Feedback learning, or called backward propagation or Backprop, describs the process of learning from random pattern to explicit rule patern. At the begining of the learning period, the network is set up with random connections among the units. During the early stages of learning, after the input pattern has been presented, the output units often produce a responce that is not the required output pattern. What backprop does is to compare this imperfect pattern with the known required response, noting the errors that occur. It then back-propagates activation through the network so that the units are adjusted in such a way that they will tend to produce the required pattern on the next learning cycle. This process is repeated with a particular stimulus pattern until the network produces the required response patern Thus the model can be made to learn the behaviour with which the cognitive scientist is concerned, rather than being explicitly programmed to produce it.

The three main computational modelling techniques used by cognitive scientists are semantic network, production systems, and connectionist networks.

A framework (or approach, paradigm, and metatheory) is a general set of ideas which is drawn upon by theorists within a particular discipline. The important thing about framework is that they should be regarded as useful or not useful rather than correct or incorrect. The reason for this is that they consist of high-level assumptions which cannot be tested directly at an experimental level. In contrast, theories should be stated in terms which permit researchers to determine whether they are correct or incorrect. theories are more constrained than frameworks, and often provide precise accounts of the underlying mechanisms and influences which give rise to a set of phenomena. Theories are too general to make predictions about specific situation. However, a model is a particular instantiation of a theory which relates that theory to a specific situation. This than makes it possible to make relatively detailed predictions.

There has been a proliferation of theories, but it is often not clear how these theoies relate to each other. A cognitive architecture is intended to provide the missing theoretical integration, and is thus far broader in its scope than most theories.

A major factor in the emergence of cognitive psychology was the development of computer. Cognitive psychologists espoused the information-processing framework which bore little resemblance to what had happened before.

(Visual) Perception according to Bartley (1989) is the immediate discriminatory response of the organism to energyactivating sense organs...To discriminate is to make a choice reaction in which contextual conditions play a deciding role." in contrast, roth (1986, p81) provided a much more cognitive and contemporary definition: "The term perception refers to the means by which information acquired from the environment via the sense organs is transformed into experiences of objects, events, sounds, tastes, etc." Even though it is not known in detail how pattern recognition occurs, it is clear at a very general level that it involves matching information extracted from the visual stimulus with information stored within the memory system.

Prototype theories: The subjects used information from the various patterns presented initially in order to construct prototypes that were then stored in long-term memory. This prototype knowledge was then used to classify and identify new stimuli, with recognition being simply a function of the extent to which any given pattern matched the stored prototype. Pattern recognition involves comparing stimuli to prototypes, which are abstract forms representing the basic or most crucial elements of set of stimuli. Pattern recognition, which is concerned with assigning meaning to visual stimuli, is a complex business. The appearance of objects changes constantly as we move through the environment, but this rarely produces any apparent difficulty in identifying objects or in making sense of our environment. Even when confronted by a visual stimulus we have never seen before, we are usually confident that we know the category to which it belongs.

memory theory: working memory system is concerned with both active processing and transient storage of information.

Recall: the subject involves a search or retrieval process, which is followed by a decision or recognition process based on the apparent appropriateness of the retrieved information. Recognition involves only the second of the these two processes. the to-be-remebered information is presented along with irrelevant information, and the subject has to decide whether each piece of information has been presented before.

two-process theory therefore claims that recall involves two fallible stages, whereas recognition involves only a single fallible stage. it is for this reason that recognition is superior to recall(Bahrick,1970)

Incidental learning & intentional learning: Intent to learn is not of crucial importance; rather, it is the nature of the processing activity that determines how much is remembered subsequently (hyde and jenkis,1973).

A representation is any notation or sign or set of symbols which "re-presents" something to us. it stands for some thing in the absence of that thing. A representation can be first cllasified as external representations we use in everyday life and internal, mental representations. external representations can be split into pictorial and linguistic. mental representatios can be considered from two main perspective: as analogical (images, mental models) and as propositional (propositions). All representation only represent some apects of the world.

When two things have the same structure, but differ in other respects, we usually call them analogous. Pictures and diagrams are called analogical because their structure resembles that of the world, but linguistic representation does not.

(Above from book cognitive psychology)

(following is from book löwgren, Jonas (1993) human-computer interaction, Sweden,Lund) Two kinds of performances in user's work, routine and decision based. Routine performance is based on diredt memories and used in highly familiar situations. He does dot have to think consciously. This is generally how learning proceeds: by automating lower-level tasks, the user can spend his mental energy thinking on a higher level. Decision-based performance is used in situations that generally familiar but not automatic. A schema is retrieved from memory based on the characteristics of the situation. It is easier to remember things if you have generated them yourself. Pictures are remebered better than words. Our memory is also amaxingly good at recognizing pictures we have seen before. In a experiment, the subject were showen 2560 pictures over a period of four days, they were then able to recognize an average of 2300 of these pictures (löwgren, 1993)

Recognition easier than recall, e.g., for not skilful users, mouse invoke functions by icons or menus is much faster than keyboard shortcuts invoke functions, because the former involves only recognition, but the later has a recall process which takes user's conscious thinking which key to hit. for skilful users, to hit which key board to invoke a specific function becomes routine and involve no recall process.

Gestalt effect of proximity and closedness indicates that when we design user interfaces, the perception of patterns would, for example, mean that the user expects buttons that are close togrther to be logically related in some way.

Research on cognitive aspects typically reports statistical results on average people, but as you know , nobody is average.

Users and designers are good at different things. Users are good at the work they are supposed to carry out. Designers, on the other hand, have a solid knowledge of computers. This affect designers attitude towards a computer system, your vacabulary, your expectations of computers, the goals you form, and a lot of other things. In a normal project, designers don't know too much about the work situation of the users. In the same way, the users do not share a lot of your computer knowledge. You might get endless surprises when you test a prototype with your users. The only failsafe rule seems to be that where you expect problems, everything works fine, but parts of the prototype that you expect to work perfectly are fundamentally flawed.

A story of Bruce Tognazzini(On Interface, MA: Addison-Wesley,1992,pp87-89) about to design a program to find out if the user is working on a color monitor. he concluded that : no matter how many engineers we had crowded into a room to discuss with what areas uers were or were not going to have trouble, we would have never hit upon this as the major problem of the application. Had we not tested, we would have had a disater on our hands: Instead of users having a wonderful first experience, they would have walked away thinking both they and our computer were awfully stupid."

In the 70s and early 80s, most of the work in HCI was based on experimental labortory research. Two roles of psychology, one was to produce general descriptions or theories, as analytical model or as general design guidelines. The other major role was to verify the usability of a interactive system. Around the mid-80s, the interest started to shift from general theories to more practical use of HCI in software development. User-centered systems development bagan to emerge. This typically involved studying real users and their work, designing prototypes and then testing the prototypes on the uers to find out how to improve the design. however, many prototyping approaches have been turned into a very expensive form of work analysis, instead of doing a straightforward study where the developers find out how the users work, they tended to ask the users what they wanted in the new systems. In the end, we could often hear designers complain about stupid users who couldn't make up their minds and didn't know what they want. When we talk about metaphors in user interface, the term unfortunately means a lot of different things. Most often, it is used to describe an analogy to real-world things that the users are already fmiliar with. A conversational model is one where the system acts as a communication partner and talk with the user about the domain objects; a model-world system is one where a black-box is represented and functions are triggered by users input. most systems contain both of these models. analogical metaphors, e.g., design the interface looks like a room with filing cabinets around the walls and a desk in the middle of the room. Metaphor can activate too much background knowledge in the user's mind. but metaphor design can not help users in the particular way to the computer systems, e.g, sort, search, browse. Feedback, user control and undo support. the main motivation for using menus is the principle "recognition is easier than recall"

The same symbol can mean different things to different people, epecially if they come from different cultures Even if you think you know the cultural implications of your symbols, you have to test them on the users to make sure that you haven't overlooked any obvious misinterpretations. one of the most important conclusions about participatory design is that the prototyping environment must be equally accessible to the users and the developers.

the original latin sense of information "informare" referred to the idea of giving shape and form to our mental state (i.e., to form and shape a subject, a person). Through the influence of natural sciences and technological development in the 29th century 'information' lost this aspect of subjectivity and developed into information equalling free, fragmented bits of information, unattached to any person or any other subject. By stressing the importance that information must involve a subject in this context, cognitive features in human beings become very relevant.

The following is form: Krarup,G. (1994) cognitive Aspects as Design criteria for Human- Computer Interface. Human Factors in Organizational Design and management-IV, Elsevier Science B.V. Metaphors can transfer knowledge about one conceptual dimension to another. The introduction of metaphors creats similarities of new kind. But those similarities are not "objective"; they are related to the subject's knowledge in the two domains. The problematic use of metaphor arises where designers are unaware or disregard the fact that some metaphors do not fit in with the user's experience and life-world. Meaning is often represented as geometric or spatial constructions (theory of cognitive semantics), e.g., Mark Johnson, image schemas, in cognitive psychology, which are those well defined, simple, and are used over and over again to conceptualize structures in action, reflection, experience, and reality, such as a container schema, . (Engerström trangle in social psychology could be developed into a social semantics)

Geon theory: when human beings are dealing with objects in the world, a number of primitive figure, or shapes, and combinations hereof are sufficient to determine the category of the object, examples are: a cylinder, a block, a funnel, they are observer invariant. This means that objects can be viewed from any angle and still be recognized. Biederman (1987) has compiled an "inventory" of 36 different primitives-geons. like words which can be formed by combination of alphabets, objects can be prototyped by those geons and recognized. (interface should first introduce geons which users know well)Human beings do not recognize an object as a whole, rather they seperate it into parts which are all geons connected by means of various relations, such as "next to", 'on top of', 'inside'.

Dillon, R.F (1983)

Human Factors in User-computer interaction: An introduction.

Behavior Research Methods & Instrumentation. Vol.15(2), pp195-199

Research has been directed towards two distinct populations of computer users. In the first population, the users are computer professionals: programmers, computing scientists, software engineers, systems engineers, and so on. For this population, the computer is not a means to an end, but rather the end itself. .... The second population of computer users for whom human-factors research is oriented is the non- computer professional. For this user, the computer is not a central focus but is merely a means to an end.

an artifical Buffer has been placed (media-designer ? trained professional operators ) between the user and computer . "This buffer of specialists has made it possible for users to obtain desired results, typically using an application package without knowing very much about the computer system. Clearly, the user-computer interface has always been poorly human engineered, but large amounts of money to support specialists have made it tolerable. (p196)"

With the appearence of inexpensive microcomputer, this buffer of pecialist has crumbled to the ground. The users themself have to be operator, ... is on their own. And they are in trouble, and they are unique in terms of training, motivation to use computer and diversity, heterogeneity. most of them have no formal computer training.

The misleading concept of "user friendly". The right concept should be "usability".

Physical aspect of interface, as keyboard layout, display legibility, lighting, fatigue and stress, noise, etc.

Cognitive aspects: aspects as easy to learn and use; good documantation, including a technical reference manual, a user's guide, and a system to provide quick answers to routine questions (on line help, e.g.); system reliability (user can never bomb or crash" the system (beginers are afraid to harm the system); error handling, incorporate good error prevention, detection,reporting and recovery; appropriate I/O, e.g., CRT terminal, keyboard, button, switches, joysticks, light pens, mouse, voice recognition unit, etc. command lines, question-and-answer, form filling, and menu selection can all add to or detract from usability, depending on the application and the users; respond speed; Cognitive organization: "The system must be conceptually organized in a comprehensible form that corresponds to the cognitive abilities of the users for whom the application is intended.(p198)" (e.g., in IBM DOS operative system and applications are hierarchically designed which make a user move up and down in order to finish a task which need several application, this make system hard to understand by the users and find out where they are. Compare Mac. system)

let user concentrate on application task, work as end, not computer (computer is a means to an end). to make opplication be transparent to user.

Return to Reading Notes