SEMS == IMMUTABLE MOBILES
Constructive Review of Bruno Latour’s Visualization and Cognition
This 32 page PDF doc (Visualization and Cognition: Drawing Things Together) was a very exciting read, from which I took notes. I did not study the doc to the depth of every detail; I focused on how close it related to some of my insights and that it appears to have been a (if not the) seminal doc motivating Alan to initiate YWorlds. I won’t attempt to summarize Latour here. A full critique would create a doc longer than Latour’s.
I quickly realized that what Latour called “Immutable Mobiles” (IM) I had called “Semiotic Structures” which I later abbreviated as SEMS. However, Latour and I viewed this identity from different (but consistently related) contexts. IM=SEM is a core concept for Alan’s YWorld project and Larry/nuets’ UPLIFT Project.
SEM = a human created pattern in physical space that when perceived generates “meaning” in the perceiver. “Meaning” will remain a primitive for this discussion. For this discussion restrict sems to visual displays of distinct symbols on a plane (paper or screen) – texts, diagrams, graphs, maps. Different persons can agree that the patterns (the SEM) each view are identical, even when neither can interpret the pattern. Although machines can prove the identities of copies of a SEM, it remains important that human perceivers also agree on the identity of the SEM (again, not needing to agree on its “meaning”). Latour refers to this as “optical consistency”.
SEMS can be nested: larger sems comprised of smaller sems. Sems can be linked to other sems with different types of relationships. A finite web of sems is also a sem and could be labeled/represented by a unique sem (icon). Individual sems in the web may have links also to other sems outside the web. Operations can be performed on sems giving each sem a set of values, relative to the frame of the operations. Each sem can have meta-sems attached that give “coordinates” of that sem in a network. A given sem can have more than one set of “coordinates”, depending on frames (or alternative clusterings).
Icons for sems can be displayed on a map, arranged so as to be perceived as a recognizable pattern. These maps can be guides for navigating among sems. The same sem can result in different “meanings” depending what sems were perceived before it. It is important to distinguish a sem, like text which can be studied and generate meaning, from an icon that labels that sem as a node in a network graph. I sense that both Latour and YWorlds confuse this distinction.
Although a SEM or IM could be as small as a character I would consider the smallest useful SEM be what could be examined in the specious present, or while all perceptions remain in working memory.
Single words perceived generate meaning, and can be viewed as sems. Many words in a cluster or a linear string of words also generate meaning. An icon representing another screenful of symbols could also generate meaning, if the icon is recognized. However, having icons to uniquely represent many thousands of significant sems would run into the same problem as for Chinese ideographic languages.
Latour posits that all relevant sems can be simultaneously displayed for synoptical perception – “to be presented at once”. To me this is demonstrable impossible – even for sems related to a sub-sub-discipline. It may be theoretically possible for a vast network of all “relevant” sems to be created by machines+humans. But only very small parts can be displayed visually for synoptical examination. However, an enhanced way of displaying multiple sems – as I believe is what is proposed for YWorlds – can be very, very useful. Individuals will probably remain limited by Miller’s 7+/-2 law (of holding in mind independent ideas), teams viewing larger displays may learn to collaborate as collective minds.
Since this will be dynamic, older versions of sems must be available as well as alternative ways to display parts of the web. Think of how a graph of many points can present in a clear, useful pattern by selecting specific axes and coordinate scales.
Published in 1983 (based on years of work by Latour). I was fortunate not to have read it before I composed and presented my own doc at a General Systems conference at Asilomar in 1994: “The Fundamental Reality of Text“. I didn’t begin to call Latour’s Immutable Mobiles “SEMS” until after 1994. A rough draft doc in QuickDoc in 2008 introduces the “SEM: One Feature of Colab Scaffolding“. Glisten, you commented in this which shows how long we have been connected. Look at the comments in the QuickDoc. It appears that my other writings about SEMS are embedded in other docs.
Latour’s focus on the mobility of Immutable Mobiles (IM) was not a major focus for SEMS, whereas the replicability of SEMS was my focus, but also mentioned by Latour. In spite of the excellence of Latour’s doc, he gets carried away with a few aspects of this idea and leaves major gaps. This is typical of all single docs that attempt to cover vast territory, no matter who the author. Many of his gaps are not relevant to this discussion.
Latour argues correctly that advances in visualization, such as perspective, and printing which distributed duplicates of Immutable Mobiles, greatly accelerated the spread and advancement of science during the last five centuries. However, when he argues that science itself is dependent on visualization; I don’t feel he is successful. Could a population who are all sightless but with enhanced other senses never develop science? What if the camera preceded the artist’s discovery of perspective? This is not to deny the very high significance of visuals as the dominant sense for constructing reality.
I am pleased to see that my assertion that Science is the study of patterns in SEMS (data and reports) and not the study of an objective reality is also developed by Latour (my first encounter with another who shares this view). It’s been a while since 1983, and except for a few exploratory minds, everyone still talks as if corporations, nations, peoples, wars are perceivable things – a point also made my Latour. I take this as a very important meme. All that we know about a world beyond our immediate perceivable environments are gained from SEMS (including video reports as sems).
Latour makes a good case for the importance of files and their organization and that real power resides with those who are skilled in navigating and using the files. Power appears to be an important theme for French philosophers, Foucault and Latour. I sense that YWorlds also seeks power – which is OK if it is power to create, but power-over-others should be avoided unless absolutely necessary, as for survival.
Latour appears to ignore the cognitive diversity of humans, as do most philosophers and scientists through the ages. Some sems may not be perceived or comprehended by some persons no matter how hard they try. Some persons can have experiences that they report in a sem; but there may be others who can’t have that type of experience, so they can’t evaluate the claim in the sem. The time and study required to equip a person with competencies to gain “meaning” from all different categories of sems may mean that everyone will have domains of sems incomprehensible.
Everyone using such a system will have to trust others whose expertise is different from theirs. Everyone will have their domains of expertise.
I am not familiar with his other writings, so I speak only of “a Latour” – author of this doc. He may imply that there is only one comprehensible system of SEMS that represents a practical reality for humans today. There are some who would question this assumption and who believe risking the future of Humanity/Gaia on this assumption may not be wise.
There may be more than one “reality” in complementarity, as in the field/particle complementarity. Physics may have its sems all fit into a single, logically consistent set; but I am not sure of biological sems and even less sure of psychological and sociological/cultural sems.
In speculation, who can say with confidence that the (superficial) Red and Blue clusters of sems and corresponding inner woven worlds of believers isn’t a case of “multiple universes” in interference.
More critically, if the intent of this analysis is to create a new technology to bring some constituency and synergy in humankind’s distributed, collective knowledge, so we can plot a route out of our Crisis-of-Crises – then we must be clear about the sems that are most important.
Ever since the first Earth Day where I challenged my panel members whether we had to fix what we had done to the biosphere or whether we had to fix the structure of human societies that both led to the destruction and may be incompetent to do what is needed – has been an ignored sem. IMHO only when the distribution of cognitive/emotive/performance competencies of the global population have been uplifted significantly and the primary structures/processes of Civilization are replaced, can we come to resolution of our Climate Change and other crises.
Some research should be conducted as to what might be done for Gaia, but the bulk of our energy should be on better comprehending and changing human systems.
Latour’s focus on power reflects the French concern, but also implies an eventual engagement with the old power order (which I feel we grossly underestimate its real power and high resistance to change) and ignore what might be a much more feasible route, replacement (via societal metamorphosis) instead of transformation (or a mix). Until these two alternatives are studied and evaluated it is impossible to prove which is more viable and which might be a dead end.
The above review may seem quite superficial compared with the more indepth analyses I discovered by a brief search online, some of which I provide links below. I have yet to examine these.
Citations to Visualization and Cognition
Chart for New Theory of Text
Distributed Cognition: Where the Cognitive and Social Merge
Word, Pictures, and the Unity of Knowledge
Data Visualization and Defunct Metaphors
Discipline and the Material Form of Images: An Analysis of Scientific Visibility
Culture and Cognition: The Boundary of Literacy and Scientific Inquiry