Describing gestures: boundaries, scales and perspectives

 

Paul Bouissac (University of Toronto)

 

1. Introduction: scientific standards and epistemological handicaps.

 

            All observations, even the most casual ones, imply a theoretical framework that is usually provided by commonsense and determines the range of phenomenological features and events which are potentially worthy of note from this vantage point.  Scientific observations require that the theoretical framework be made explicit and adequately supported by a reliable apparatus in the form of methodic procedures and recording or measuring devices. This allows the critical assessment of scientific observations since, following the standard practice of modern research, theories, methods and instruments are disclosed to the scientific community at the same time as the results of these observations and the conclusions which have been reached. At any given time, consensual agreement among the specialists in a particular domain determines the current state of scientific knowledge. Whenever new theories and models acquire this status, this creates a novel epistemological context which usually triggers adjustments, even sometimes deep changes, in other areas of research since disciplines and their specialties are construed as mere windows framing small parts of the larger, assumedly consistent but elusive landscape which is posited by the ideology of modern science (Psillos 1999).   

            The study of gestures should be expected to meet the standards of scientific research. However, the information provided by the articulated parts of the human body in the course of social interactions is so relevant to the conduct of every day’s life that some primal mechanisms appear to have evolved for the vital management of this information. In addition, commonsense has built, in the context of every culture, a “natural” theory of gestures which implicitly specifies the range of potentially relevant features and events that can be observed in the movements of the limbs and faces of interacting agents. As a consequence, the students of gestures, which  necessarily are already members of a particular community of Homo sapiens,  are confronted with a double epistemological handicap: a probable behavioral genetic endowment [see note 1] and a tacit cultural knowledge which are bound to bias their observations. Therefore, if the study is not going to be a mere explicitation of the implicit --  a sort of literary exercise --  the question that they must address is: how to avoid taking too much for granted and how to position themselves so that their research will yield true information rather than tautologies.  

            Of course, the sort of knowledge yielded by the methodic explicitations of gestures in the form of phenomenological descriptions and categorizations is not useless. It helps refine our awareness of the richness and diversity  of dynamic face-to-face social interactions and provides means for better managing of these interactions notably in the context of multicultural, multiethnic civil societies. At a more abstract level, it constitutes an important part of comparative semiotics. But if this knowledge is assessed from a more general epistemological point of view, it appears to remain within the pragmatic sphere, somewhat on the same level as the minimal metalinguistic knowledge required for teaching second languages. Symptomatically, the systematic study of gestures is often justified on the ground that gestures are actually an essential part of both verbal and nonverbal communications. The large number of gesture studies which have been completed to date in the context of descriptive semiotics have for the most part contributed to develop observational and managing skills rather than theoretical knowledge.

            The purpose of this article is to critically assess some of the methods which have been followed so far and to tentatively explore some possible directions towards the construction of a broader theoretical perspective which would not be restricted to the pragmatic sphere. Naturally, any advance in the theoretical understanding of the nature and functioning of the class of behaviors to which gestures belong, would, like in other realms of scientific inquiry, open the way to countless, unforeseeable practical applications.

 

 

2. The lexicon fallacy

 

            Semioticians have shown a predilection for the lexicon approach in their investigations of gestures. They have endeavored to identify normalized patterns of movements that form units conventionally associated to particular meanings. These “lexical” entities have been conceptualized on the model of linguistic morphemes and classified with respect to their degree of semiotic autonomy, that is, whether they can stand by themselves or must enter into combination with other morphemes which can be linguistic or gestural in order to achieve a communicative goal. Standard classifications of gestures mirror the treatment of words which a long tradition of grammarians and lexicographers  have ordered into ironclad categories such as “nouns”, “adjectives”, “verbs”, “adverbs”, etc., which appear to the naive mind as “natural” as any “natural kinds” in spite of the controversial basis of their fuzzy linguistic boundaries and hierarchical ontology. The very notion of “word” itself remains the focus of inconclusive definitional debates. Expectedly, the lexicon approach to gestures has lead to the search for gesture etymons and to the creation of gesture dictionaries. [see note 2] However, these projects are characterized by a tendency to take for granted the existence of “natural boundaries”, so to speak, that would cut out gestures to the naked eye as neatly as words appear to be partitioned in traditional descriptive grammars.     

            The first problem usually addressed by semiotic research on gestures in indeed the problem of spatial and temporal boundaries. Where does a gesture start and end? What geometrical space does it delineate? How to circumscribe it? This seems to be a primary condition for making observation and description possible. But this presupposition fallaciously reverses the problem by assuming a state of affairs, namely that gestures have boundaries, instead of asking whether gestures have boundaries, and, if so, what is the nature of these boundaries. It may be that natural languages usually provide generic terms that designate a class of movements carrying symbolic meanings, as well as nomenclatures listing a variable number of particular movements belonging to this class. This is at least the case in English in which “gesture” or “sign” encompass “shrugging”, “nodding”, “waving”, “pointing” and the like. The lexicon approach to gestures tends to use these nomenclatures as a primary resource for the segmentation of the flow of movements. They develop methods for making visually explicit and technically describable the referents of the gesture lexicon of one or several languages. If the language net that is thus cast upon dynamic interactions appears to be too loose, neologisms are coined in order to enrich both gestural and lexical dictionary.

            The modern pioneers of semiotic research on gestures such as Efron (1941), Birdwhistell (1970), Eibl-Eibesfeltd (1972), Morris et al. (1979), and Ekman & Friesen (1969), have relied more or less explicitly on the lexical resources of their respective languages to cut out, so to speak, their observational frame. The numerous results they have produced and inspired in the form of monographs, repertories and dictionaries such as those by Calbris & Montredon (1987), Pezzato & Poggi (forthcoming), Posner et al. (forthcoming) typically follow the encyclopedic pattern of a succession of entries, each being devoted to a particular gesture or a “family” of gestures which are claimed to be variants of a prototype.

            The entries usually combine graphic representations and descriptions in a natural language. The former are characterized by a set of devices which purport to achieve three main goals: (i) the gesture is individuated, that is, it is construed as a whole comprising a limited number of parts such as limbs, hands and faces shown in a few successive positions, thus freezing an ephemeral configuration or a trajectory indicated by a series of arrows and dotted lines; (ii) the representation is intended to be neutral in as much as all other information regarding the “support” or “producer” of the gesture is carefully eliminated; markers that would reveal the age, class, mood, clothing, or situation of the represented “gesturer” are absent; alternation  of gender appears sometimes in more recent examples as a concession to political correctness; when gestures are shown performed by live actors, the performers, usually impassible, are dressed in a manner as neutral as possible in the belief that any supplementary information would interfere with the neutral individuation of the gesture; (iii) finally, the represented gesture is translatable, that is, a verbal equivalent is inscribed in its vicinity like in a bilingual dictionary; in some cases the expression exists in the natural language used in the community which produces this particular gesture; in other cases, a paraphrase must be created.

            Verbal descriptions constitute a necessary part of gesture entries. Several strategies are used to achieved a discursive representation corresponding to the graphic one. They most often take the form of embedded micro-narratives. On the one hand, movable parts are construed as actors tracing various paths toward their goals and, on the other hand, the gesturing agent is positioned in a hypothetical standard situation. In general, coordinated movements of micro-actors bridge the distant points of  “intended” trajectories and these movements are encapsulated into  minimal narratives which purport to represent the meaning of the gesture in the context of its current use. Furthermore, an etymon is often provided by relating the gesture to an assumed deeper history, a macro-narrative through which concrete, practical actions evolve toward more abstract, metaphorical or ritualistic patterned behavior.    

            If the above characterization of the lexicographic approach to gesture research is accurate, this method raises several problems. First, one of the major difficulties encountered by gesture dictionary projects is the selection of the lemmas. How will the entries be ordered so as to be retrievable? If the names which refer to particular gestures in the language of the researcher are selected, the dictionary will be incomplete in two respects: not only is there no certainty that all gestures have a lexical equivalent or that these equivalents, when they exist, are not ambiguous, but also various traditions and languages treat differently some spheres of activity or domains of interactions. Important distortions and shortcomings can be predicted. The solution that would consist of using only neologisms as lemmas appears to be self-defeating as far as the purpose of a dictionary is concerned. Secondly, the method relies on the definition of gesture prototypes based on the selection of a few parameters. How are these prototypes constructed?  There is a set of assumptions regarding the communicative nature of gestures and their intended articulation. The relevant medium is uncritically considered to be planar Euclidean geometry. Plans are supposed to intersect when the third dimension is involved. The point of view from which they are ideally perceived is frontal, facing a virtual observer. All these features are quite arbitrarily taken for granted. Thirdly, the reconstruction of situations and etymons is highly questionable. Both rely upon imagined stories that provide ad hoc contexts to which a particular type of gesture is assumed to react either as expressing an attitude or as attempting to intervene and modify the state of affairs. This strategy offers a clear case of circularity since the parameters of the situations which are thus selected in constructing the examples are precisely those which can be predicted from the assumed meaning of the gestures which the entries purport to illustrate. This also applies to the reconstruction of etymons which claim to identify some ancient  practical or technical behaviors bearing some resemblance with the symbolic gestures concerned. A story is then provided to explain how the latter derived from the former, usually without the shade of any historical evidence. At best, such accounts are plausible.

Finally, the purpose that gesture dictionaries are intended to serve should be raised. It seems that, once again, the lexicon fallacy is determining. Dictionaries and encyclopedias list terms in alphabetical order in order to facilitate access to semantic and pragmatic information regarding these terms and the topics they designate. Which user can be imagined for a gesture dictionary? Looking for a word that makes reference to a meaningful movement in a particular language? Looking for the unknown meaning of a gesture that has been observed and which has no current denomination? If a reference work is meant to afford easy access to relevant information, these rhetorical questions point to the difficulty of conceiving such projects as anything more than the result of the lexicon fallacy.   

 

3. The phenomenological fallacy: a matter of scale 

 

            The semiotic study of gestures has by and large relied on naked eyes and natural languages for its basic descriptive accounts of meaningful postures and movements. Recordings in the form of photographs and films have simply played the role of visual relays that were destined to be processed on the same phenomenological level as direct observations, albeit with more attention to details and with some added possibility for comparing data. The argument which can be presented in support of this method is that gestures fulfill their communicative functions precisely on this level and, therefore, systematically applying scientific scrutiny to gestures as we perceive them should be expected to yield relevant information concerning their structure and significance. However, this is to forget that many cognitive illusions are bred by direct visual observation or fallacious natural reasoning (Piatelli-Palmarini 1994) and that most scientific knowledge is constructed through the mediation of investigative instruments which afford access to levels of resolution well beyond the range of natural phenomenology, both below and above the thresholds of human perception. Microprocesses and long-duration morphological dynamics equally elude human natural awareness and require technological mediations and explicit logical reasoning using mathematical tools. The belief that gestures, because of their assumed immediacy, are wholesome phenomena which should be treated differently is a fallacy which has impacted negatively on the advancement of knowledge in this domain and is responsible for the mostly trivial character of the discourse yielded to date by the semiotic study of gestures.

            The problem of scale is a crucial part in the acquisition of non trivial knowledge. Like in other domains of experience, the limits of the visible and the conceivable must be pushed back in both directions by scientific inquiry in order to achieve an understanding of the processes of dynamic interactions from which gesture patterns are arbitrarily isolated for the sake of semiotic convenience. There are at least three boundary issues for determining the scale of description of limb movements and body postures, which are always combined both in practical actions and pragmatic interactions.

             First, the range of possible levels of resolution of the observation spans several orders of magnitude, from neurological events, at times involving single neurons, to synchronized macro-patterns encompassing two or more interacting agents as well as environmental and ecological factors. Moreover, gesture units, even if they are conceived as combining a set of complementary movements, are cut off from the temporal continuum to which they belong. Both distal and proximal events are obviously linked to any gesture instances, not only as memories, skills, and immediate dynamic contiguity, but also as proactive behavior, projections and predictions. Segmenting too narrowly the spatial and temporal dimensions, which incidentally cannot be truly distinguished from each other, creates research artifacts which are mostly irrelevant to the flow of human visual interactions.       .

            Secondly, phenomenological descriptions, even in their more thorough and detailed versions, are constrained by the limits of conscious awareness. This awareness is conditioned by several factors:  the natural language which is used and its perceptual and semantic categorizations; the personal and social biases which further interfere with the latter; the saliency of deviant patterns over normalcy which is usually taken for granted or even appears invisible; the restricted temporal window of conscious attention spans;  the unconscious and automatic processing of visual information which has been adaptively engineered by evolution with respect to individual organisms’ survival; the threshold which applies to the quality and quantity of information which can be consciously processed. Scientific inquiry attempts to overcome these constraints by pushing back the  perceptive thresholds through various artificial devices, by distributing the observation process  among a community of observers spread in space and time, by assessing and compounding the information acquired and by elaborating conceptual and mathematical models which transcend the natural boundaries of the life world.

            Thirdly, classifications and explanations of gestures which are based upon direct phenomenological observations are necessarily reductive. From this point of view, gestures can indeed be conceived only as communicative, thus entailing psycholinguistic models of interpretation limited by a relative small set of functions. Furthermore, the data considered in this theoretical framework are bound to be selected a priori, following  phenomenological intuitions regarding what is and what is not communicative. Empirical research along these lines typically take the form of recording and observing individuals belonging to a particular culture or subculture who are asked, sometimes in exchange of payment, to perform predetermined gestures. This is done for the sake of clarity, but any reliance on actors, professional or amateurs, for the acting out of gestures is bound to yield data which are reduced to a few redundant features produced in view of the investigative situation which can hardly qualify as a natural occurrence of social interaction. When two or more persons are asked to interact verbally in a controlled environment on preset topics and in presence of recording instruments, the artificiality of the context and the self-awareness of the subjects considerably constrain their multimodal interactions. Practically, it condemns the experiment to tautological results as it consistently end up with mere confirmations of the premisses 

 

4. Perspectives and prospects for the semiotics of gestures 

 

            All gestures are body movements produced in constant interactions with a fluctuating environment. The sources of this information flow to which gestures react by adjusting or interfering are geophysical, climatic, artifactual,  organismic, social as well as self-generated since all mobility modifies the surroundings of an individual. All these dynamic interactions occur first in the brain where information is processed and represented and where movements are planned and initiated, perceived and understood (Blakemore & Decety 2001). It is also in the brain that they are monitored, corrected, rehearsed or repeated, and that their effects are assessed and stored in memory systems. A very small part of body movements, including gestures, are consciously implemented. Most have a genetic and developmental basis, others are skills which have been learned by spontaneous imitation or deliberate training What semiotics usually calls a gesture is the very small tip of a iceberg whose visible part emerges from complex neuroarchitectures rooted in the depth of evolutionary time. It is in the same region that the neuroarchitectures which support motricity are entangled or interface with those which make language and meaning possible. It is now established that the perception of movements is not a purely visual matter since it also involves neuronal firings in areas which control motricity (Rizzolati et al. 2001). The understanding of a gesture cannot be complete if it does not comprise the representation of a complex set of microprocesses linking its source to the effects of its impact.

            In all gestures, time is of essence. This is generally taken for granted, as if the temporal dimension was a sort of neutral substrate. Graphic representations of particular gestures, drawn according to the principles discussed in section 2 of this article, include sometimes the trajectories of various limbs but practically never indicate velocity, acceleration, and other precise timing information in relation to the temporal structures of the body and its immediate environment. Neuroprocesses proceed in time according to rhythms governed by biological clocks which regulate all body functions (Buijs and Kalsbeek 2001). The time frame of gestures is not neutral but involves a set of variable constraints which must be taken into consideration in any serious study of gestures. Enough knowledge has now been acquired regarding  the role of biological clocks in the management of physiological functions and movements for it to be usefully integrated in research on body dynamic. Moreover technological means for achieving precise measurements in milliseconds, which is the only proper level of temporal resolution for the representation of gestures, have been available for quite some time in psychological research (e.g., Mair 1978) .

            But no gesture can be conceived, let alone observed, in a vacuum. Too few semiotic studies of significant body motricity have considered interacting dyads as coupled systems to be described on the microlevel suggested above. In particular, multipolar interacting processes have been sorely neglected by researchers or have been treated casually as mere generalizations of individual observations. All too often, the receivers or addressees of gestures are taken for granted, as simple decoding poles. It should seem obvious that what semioticians call a gesture is only a small portion of a complex and extended apparatus comprising not only the dyadic microprocesses relating to the motor and semiotic completion of the event, but also a bundle of informational constraints which includes, among other parameters, immediate surroundings, social situation, memories and expectations, and, most importantly, the multimodal messages which overlap in both directions with the production of any gestural instance. It is unfortunate, and ironical, that so much gesture research which is done in laboratory conditions  does not focus on the specificity of this particular type of situation. Instead, researchers tend to claim for their results some degree of morphological and semiotic universality. The complexity of the factorization which is advocated here undoubtedly will require that researchers rely more decisively on mathematical language if they want to be in a position to adequately represent their object of study.

            But there is more. What about the big picture? Except for a minority of semioticians who seem to believe that the positivist account of the communication arc, in the forms variously given to this basic model by Charles Morris, Karl  Buehler, Roman Jakobson and their epigones, constitutes a sort of ultimate theory for the semiotics of gestures, there is a consensus that a serious theoretical basis is lacking and that this lack explains the limitations which are constantly encountered in empirical investigations. The theoretical horizon of an inquiry determines the range of data that can be observed and recorded, and provides blueprints for experiments that may yield true information and consistent explanations. The reductive construction of gestures as messages modeled on a particular brand of functionalist linguistic may have help produce catalogues of drawnings portraying stereotypic  movements and postures accompanied by anecdotal evidence of their culture-specific meanings, but these results fall short of providing answers to the how and why of gestures. Only some explorations of the interface between hand gestures and verbal utterances have opened promising theoretical perspectives, albeit within the limited sphere of the multimodal nature and origin of language (McNeill 1992, 2000).

            Many other explorations could be undertaken. Let us tentatively list a few. On the empirical side, longitudinal studies of primate groups and other social mammals, which were undertaken during the last three decades, have revealed remarkable patterns of dynamic symbolic interactions. For long, humans have seemed to be off-limit for such inquiries. However, contemporary forms of televised  quasi-exhaustive recording of longitudinal, often conflictual interactions over sizeable periods of time provide now unique observational opportunities. More importantly, Reality TV as a genre affects such investigative endeavors of a new coefficient of acceptability providing that adequate rewards and reliable safeguards are available. [see note 3] Only such long-term, continuous observations would make possible the correlation of the many parameters comprising situations and their evolution.

            Another direction of gesture research that can be realistically considered is the meta-analysis of neurological, more particularly clinical literature which is available and keeps growing as imaging techniques provide increasingly detailed representations of behavior-brain correlations. Achieving a theoretical integration of the micro-processes underlying the dynamic of limb movements and the macro-percepts of interactive gestures is a challenge for semioticians who want to emancipate themselves from the low-information yield of current gesturology and construct more complex objects of inquiry. Furthermore, formalizing and mathematizing these new objects through algorithmic and statistical models would undoubtedly facilitate the theorizing of this new range of data in a broader evolutionary framework, thus opening new realms of observation, hypotheses and methods. [see note 4]. The task lying ahead is daunting, but far more exciting and promising than compiling vignettes and anecdotes for unsearchable dictionaries.    

                                   

5. Conclusions: challenges and opportunities for semiotics

           

            The semiotic movement has undergone some noticeable changes during the last two decades: several specialties have branched out and acquired organizational autonomy.The  semiotics of music, the semiotics of law, the semiotics of space and visual semiotics, for instance, have developed as semi-independent subcultures with their own formal associations, congresses and publications. The semiotics of gestures is the latest domain to reach this stage. Although the term “semiotics” tends to be played down in its identity chart, the association was initiated by core semioticians as well as by other researchers long associated with semiotics. While such spawnings can be expected to facilitate the development of knowledge in their respective areas, there is a risk inherent to all subcultures, scientific or otherwise: the eventual creation of a closed mindset comprising unquestionable assumptions and exemplary works, and determining what should count as legitimate research with respect to these standards. The main challenge for the semiotics of gestures will be to be able to question its models and implicit theories.

            This article has attempted to point out the epistemological fragility of some of the current theoretical grounds upon which the gesture research methods are based. It seems indeed that this virtual paradigm must overcome a load of uncritical assumptions and should expand its focus beyond the fairly narrow goal of gesture description, coding and characterization. It should endeavor to construct an object of inquiry whose boundaries, scales and perspectives would be more inclusive and would afford the possibility of producing new knowledge rather than simply verifying mostly trivial hypotheses. Naturally, such a strategic move would require that researchers rely more on a multiplicity of special disciplines which may appear impenetrable in as much as they are scientific domain-specific subcultures each, with its own memory, language and traditions. But the cutting edge of the knowledge they keep accumulating, the problems they are trying to solve and their own genuine needs to expand their focus, are resources which are available in the form of publications, conferences and personal communications. Interfacing strategies may be dangerous because there is always some risk to uncritically appropriate information  and construe fallacious knowledge based on partial data taken out of context without being fully aware of the controversies which surround them within their own scientific paradigms. Often, science popularizers simplify and embellish the results of a variety of research made by others in the course of building their argument, and these results are further simplified and distorted when they are introduced in semiotic or philosophical discourse. In the 1970s, research on the lateralization of some brain functions thus provided fodder for speculations on right-left dichotomies that delighted humanist but that no neuroscientists could underwrite. The recent discovery of so-called mirror-neurons, while it is obviously of utmost relevance to the understanding of the microprocesses which underlie the perception and imitation of gestures (Rizzolatti and al. 2001), nevertheless appears to unfortunately inspire similar uncritical extrapolations. This happens mostly when scientists use figurative language to designate an object, feature or process they have discovered, thus unwittingly giving rise to a metaphoric momentum that ripples across speculative disciplines and is difficult to stop. However, safeguards can be devised to ensure that interfacing strategies do not lead to such pitfalls. Monitoring a domain of inquiry over a period of time, taking controversies into account, arranging face-to-face multi-disciplinary encounters to test the relevance of particular generalizations, are some of the cautionary tactics that should be in order. 

            Research on gestures constitute a challenge for semiotics because, as it was emphasized in section 4, much remains to be discovered in a domain which provided probably the  earliest incentive for conceiving, and speculating on, signs: the power at a distance of some body movements, a kind of symbolic ballistic that achieves intended results at lesser energetic cost and with a statistically significant rate of success. Replacing the gesturing body in the long duration of Darwinian dynamics, in the context for instance of the emergence of individuality and cooperation (Michod 1999), could open new theoretical vistas encompassing the many levels on which gestures can be scientifically approached.

 

Notes:

 

1. The future discovery of hard-wired mapping of some stereotyped gestures in the visual cortex, notably those gestures which relate to courtship, social ranking, and agonistic behavior, is probable. There are at least two reasons which support this expectation: first, it is already established that some facial patterns, that is, visual configurations resulting from synergic muscular contractions, are both produced and decoded independently of learning experience (e.g., Ekman 1994); secondly, the exploration of other sensory domains have revealed surprising hard-wiring such as, for instance, the existence of stereotyped olfactory maps in the olfactory cortex of mice (Zhihua Zou et al. 2001). Since it has been shown that, in humans, the expression of disgust is under the control of specific neuronal circuitry, it can be reasonably hypothesized that so vital an information is the identification (and signaling) of dangerous smell that appropriate sensory-motor systems not mediated by learning experience may have evolved (Calder et al. 2001). Future research is likely to uncover a range of gestural stereotypes involving genetically determined dynamic patterns in specific social contexts which cultures norms regulate rather than generate, a momentum that can be traced back to Darwin (Ekman 1973). For instance, the white patterns (sclera and teeth) generated on the human face by spontaneous muscular contractions in the context of social interactions could be good candidates for hard-wired signaling systems (Bouissac 2001).

     

2 . Dictionaries are artifacts that western literate cultures take for granted. Their principle is intimately linked with a particular writing system. They are metaphorically equated with the idea of totality and completedness. A dictionary, or any other form of exhaustive compilation following the alphabetical order, thus suggests that the knowledge pertaining to a domain of expertise is complete and consistent. The powerful image of an absolute reference seems irrestible to researchers who strive to impose their theoretical views as final. But the alphabetical algorithm, as an artifice used to give an appearance of order to a selective set of notions, is a mere rhetorical device. Even within the field of lexicography dictionaries are problematic and require a number of choices at all stages of their completion, such as the degree of comprehensiveness of their entries and whether the written forms used as lemmas will tend to refer notions to names (onomasiological entries) or names to notions (semasiological entries).    

Decisions regarding the lexically relevant units are also a thorny issue since most lexemes are sets of morphemes which usually comprise sets of sememes (units of content). Principles of segmentation always involve some degree of arbitrariness and true consistency is practically impossible to achieve. Proper names raise another difficulty because of their often ambiguous status. Finally, the formula which structure the entries and the macrostructure of the work as a whole are rife with pitfalls: synonyms, homographs, slangs, taboos, figurative expression, linguistic creativity, to name only a few problems, generally receive ad hoc treatments. For a summary review of lexicography and lexicology see Malmkjaer (1991 :291-305)

 

3. In the 1970s, the psychology of group dynamic inspired experiments which, although they were not specifically aimed at exploring the role of gestures in human interactions over long periods of time, could have yielded interesting data relevant to multimodal communication if they had been also monitored from this latter point of view. Philip Zimbardo, the Stanford University psychologist who had designed and run the experiment, and interrupted it when it was clear that the participants were becoming so involved in their role playing that abuses were being committed, reportedly expressed concern that Reality TV could lead to similar excesses (Shouse 2001). Nevertheless, there has been in recent years several continuous filmings whose object was not the recording of gestures but the outcome of stressful situations. This footage of many days of constant relatively candid camera should provide raw material for the observation of dynamic interactions within their extended contexts. The fact that gestures were not the explicit focus of the filming might ensure a high level of genuineness in spite of the fact that the presence of the cameras must have generated some degree of self-awareness and acting out.

 

4. A common assumption is that gestures are the outcome of algorithms, of sets of step by step instructions which implement various motor programs in view of specific contexts and situations. The problem which is entailed by this assumption is that it requires another assumption: a source which selects and initiates programs as tools toward goals. This leads to the infinite regress of the Russian dolls version of the homonculus. The notions of signs in semiotics or of symbols in the cognitive sciences are conceived as the tools or currency of these processes. They suppose the problem solved right at the outset by equating the input and the output, and by skipping whatever might be between them. But, as some dissenters would say, such units are epistemological phantasies, thought experiments, because all what there is to be observed in the networks from which behavior emerges consists of configurations of excitations and inhibitions (e.g. Rumelhart and McClelland 1986). Signals do not convey meaning. They generate meaningful patterns based on statistical rather than algorithmic models. The semiotics of gestures cannot avoid to raise the issue of deciding which kind of mathematical model is most appropriate to construct its objects in view of the multiplicity of levels which are involved. It cannot proceed with any chance of success on the basis of commonsense phenomenology and folk psychology, and keep ignoring decades of advances in numerous relevant domains of scientific inquiry.          

 

References: