New Media Futures

Daniel Faltesek

Oregon State University

Corvallis, OR

New Media Futures by Daniel Faltesek is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, except where otherwise noted.

Download for free at https://open.oregonstate.education/new-media-futures/

Publication and on-going maintenance of this textbook is possible due to grant support from  Oregon State University  Ecampus .

Suggest a correction

This book was produced with Pressbooks.

Contents

- Cover

- How to use this book

- Section 1: Theorizing the Future

- Section 2: Where Change is Unlikely

- Section 3: Things That Are Likely to Change

- Section 4: Methods

- Section 5: Provocations

- Glossary

- Creative Commons License

- Recommended Citations

- Versioning

1

2

How to use this book

H.1 What is this book?

This book is intended for use in a large introductory class in new media in a program that covers the “full-stack” including critical/cultural studies, media management, diffusion of innovation, and synthetic media production. The first half of this basic sequence covered new media and democracy, finance, intellectual property law, basic games, and transmedia. The second half of the sequence covers many topics related to aesthetics, design, technology, and methodology.

To that end, this book needed to be written so that it would be helpful for many different professors and trajectories of study. This book is in neither engineering, social science, nor the humanities, but also all of those. At the same time, this is a program in the Communication Studies and Media Studies traditions of the United States and that texture will come across.

From another perspective, this book is intended to prevent another book from being written. There is no major textbook in the area of communication futures, and texts regarding production are typically quite expensive. This book should offer a free alternative to what would likely be an expensive textbook.

H.2 How can you use this book?

If you teach a course in media futures, this book likely has some resonance for you, if you do not teach in this area, the book can be useful in a number of different ways. I see it as a supplemental text for courses that are moving into new territory.

You may notice considerable overlap between sections two and three. Ideally, the descriptions in section two concern the physical base of a thing while section three deals in the context of an enfolding. Ideas are thus covered twice.

H.3 This book doesn’t make a lot of predictions

For many good reasons. Predictions are often wrong. Why would I take that risk?

H.3.1 Here Are Some Predictions

To avoid quickly dating this book, I avoid making very specific predictions. Instead, the space for conclusion and prediction is left open for the lectures and class activities. There is a full list of potential activities included to help you plan.

If you need predictions from me, here are a few:

- Increasing television resolution is a game with diminishing returns

- Fully immersive body suits will not be widely adopted

- We will be much better at getting information out of the brain than directly putting it in

- Anything like a singularity will be hacked

- Big Data is a bubble that will pop

- AI discourse will revitalize humanism

- Content will continue to be King

- Governments will be very good with technology, much better than independent technologists

- Technologists will continue to falsely believe that they are more capable than the state

- Simulations will never be satisfying

- Ethics will continue to hinge on aesthetics

- Nationalism will only grow deeper

- We will continue to invent discourses that excuse or explain the fundamentally political nature of disaster

- The future won’t be as great as was promised

- Nothing will taste quite as good as it does in your memory

- The same old intertextual references will be made

H.3.2 About the Provocations

I view the provocations as ways to conclude units or ideas, I also view them as Futures takes on major problems in communication, in their essence:

1. The Truly Alien: how do we account for post-semiotic futures?

2. Torn Enfoldings: are any sensations outside of symbolic interaction?

3. The Impossibility of Alterity: what about all the things we don’t say?

4. The End of Work: what if foundation cultural forms like the importance of “work” disappear in the future?

5. The Nightmare of Satisfaction: how would we move forward if desire was actually satisfied?

6. Hacking the Mind: are the benefits of new communication technologies worth the cost of lost agency?

7. Without Plurality: how can we overcome the gravity well of groups and nations?

8. Infinite text: should we continue to assume that most thoughts and discourses are intrinsically worth circulation?

9. Me and My Bots: how does the world change if we flatten the ontology of speaking?

10. As Good As It Gets: what if the present is the best possible allocation of experience?

The provocations can also be conversation starters and offer chances to discuss collective action, ethics, and aesthetics. Many of the discussions of these questions are in contradiction.

H.4 Problems with this Book

It is likely that this book has problems, all textbooks do. This is intended for use in a frosh-sophomore level course. As I often tell colleagues, students take more than one class. This book is not intended as a multi-level text that could be used across an entire major. As such the descriptions in this book could easily be seen as superficial and unimportant. I plead guilty. This text is an ambitious attempt to make the grounds for an emerging variant of the communication discipline coherent.

There are three problem areas that could be useful for teaching against this book:

1. This book presumes that academic production practices are treated as roughly interchangeable. Design and argumentation are equal. You could easily teach against this book in such a way that you elevate your disciplinary practice. Be that argumentation, dialogic listening, or design. Defend your discipline while you use the conceptual resources of this volume.

2. Like things are not together. It is likely that you will want to assign pieces from different sections on the same day. The chunks in the book are intentionally small so that you can deploy them like academic Legos.

3. Construction may trump deconstruction. You likely notice flavors of Derrida, Lacan, and Kirkegaard in this book. I can easily see objections to this framing from Hegelians on the right and the left. If your version of positivist social science resolves the problems of passion and desire, teach against this text from that angle. I hope I have provided a formidable shadow to box. If you are a humanist who finds too much of the outside in this text, I hope you find a wily social scientist playing hermeneutic games that trouble the class.

Furthermore, I have intentionally avoided writing this book in Law & Order, “ripped from the headlines” style. There are likely many examples I have omitted of cool new stuff that happened while I was writing. There will be many more. This is why you prepare for class by adding new examples and extending the text.

If there are major issues, or if the science really shifts, I will post an update on my website and change the next edition. It is also possible that the book will expand in the future.

Section 1 – Theorizing the Future

1.1 Where is my jetpack?

Sound familiar? This was supposed to be the time of ubiquitous flight and mars travel, cured cancers and realistic virtual reality. Sure, there were major changes, the cost of communication services dropped dramatically, computer parts became impossibly cheap, and the barriers that once imposed scarcity onto communication have come crashing down. But the future looks a lot more like the past than not. Predictions are hard, especially when they are about the future.

What we understand as the past, to be studied as history, as largely collected through traces that are unreliable. What do we actually know about what voters thought in the 1928 election? For that matter, what do we know about voters in the 2016 election? Contemporary communication technologies allow ubiquitous access to information and widely distributed contact with individuals. New models are trumpeted as offering access to additional information and possibility, ecommerce technologies that would promise to know consumers better than they know themselves. At the same time, these technologies seem incapable of actually resolving the problem of public opinion formation and collective action. The underlying problem of the public sphere is not one of technology, but one of passion. So, you don’t have a jetpack and the forces of hate seem to be gaining strength.

Consider the rise of the everything store, once known as the Sears catalog, now Amazon.com.[1] There is nothing particularly remarkable about the idea that people would like access to a wide array of goods, delivered to their homes, at reasonable prices. Jobbers delivered an assortment of goods with non-fixed prices, department stores developed additional inventory and fixed prices (a great improvement over constant negotiation).[2] Grocery stores and discounters appeared not as a paradigm shift, but as a continuation of the same trend. At times, firms might opt for smaller assortments, but this is merely the play of strategy as the desires of the population are neatly mapped by business operations. The future of retail is also the past.

What continued for all of this time is the desire of the public to be warm, housed, and fed. It is not remarkable that the desires of the public are continuous over time. Deep structures like hunger continue. What varies is the vast array of symbolic expressions of these needs, which come through fashions, cuisines, and other cultural codes. Which is not to say that codes do not become an end in themselves, the rich intertextual life of the public is just as real as the physical life. Manuel Delanda recognized this when he juxtaposed two distinct gradients for the legitimation of a state: symbolic and material.[3] A system that provides sustenance with no meaning or intersubjective investment is objectionable, just as a system that has a vast symbolic life with little effect would be an utter failure. Failed symbolic legitimacy can overwhelm physical plenty.

This textbook is not an account of the reasons why jetpack development has been so slow. For the most part these would be effectively explained by elementary physics and engineering. In an article on the topic of jetpacks in The Guardian, Cardiff University Lecturer and science columnist Dean Burnett laid out the key issues with jetpack technology: gravity is a substantial force and most flying machines use properties like aerodynamic lift (wings) to generate enough lift, managing the size and deployment of the engine itself.[4] After all, it is not enough to build an engine that might lift a human successfully, but you need to attach it to a human being. The Evergreen Aviation museum, a world leader in obscure craft which includes the Spruce Goose, features a number of single user escape helicopters. James Bond, eat your heart out. Attaching a thruster to a human body is tricky and we haven’t even gotten into special issues in buckle development. Burnett’s argument did not hinge on the difficulty in building the machine, but in the fact that it is not desirable. Jetpacks are far more dangerous than bicycles, people would make terrible choices with them, and they would produce vast emissions of greenhouse gasses. You don’t want a jetpack – notice that word: want.

This is the inflection point for our studies of the future. In communication, the question is rarely if something is possible, but if it is probable or desirable. The title is New Media Futures, our subject matter: what are the possible and probable future technologies for the creation of meaning.

1.2 Disciplinary Context

This book was written for use in a broad New Media program based in the Communication Studies tradition. There are a number of disciplinary threads that tie our traditions into their current administrative alignments. The rhetorical wing of communication traces their origins to a walkout from a writing conference. Mass communication research on the other hand would emphasize the role of social research and media studies from the outset of communication.[5] Others would choose moments that resonate with their particular moment of transdisciplinary contact, from art history to sociology. Communication is not an unusual academic field in this sense.

All academic fields depend on a largely arbitrary disciplinary moment, a point where some critical ontological or epistemological choice was made that determines the answers to many subsequent questions. What does that mean? Economists often begin from a disciplinary fable about the rationalization of barter. Sociologists may reduce interactions to the result of a social force. Artists explore the moment of genius where creative energy seemingly appears from nowhere. Psychologists find the core of all behavior in the cognitive structure of the individual with a lurking basis in the brain.

The tighter the story, the more likely your discipline is to have prestige. It is not that these stories are entirely wrong, but that they always necessarily tell part of the story.

Communication is a great field because it is organized around a number of weak stories. At the same time, this is a curse when dealing with organization of the university system. Is communication a point to organize around or a virus that is withering the marrow of disciplinary rigor?

The critical moments for the study of communication would be decisions about people and context (meaning networks and objects), proximal and distant.

People

Context

Proximal

Interpersonal and small group

Biological and Technological determinism

Distant

Rhetoric and public culture

Infrastructures

This is not to say that scholars may not connect multiple areas of research, but that most research tends to fall into these slots. The truth is likely between all: it is not that the infrastructure is enough to cause the movement, but that the movement surely would not have formed without it. Academics select an angle because it provides explanatory leverage, that when placed in conversation with other perspectives, can provide rich understanding of the world.

Beyond communication, this book is situated with regards to Futures studies. This is only one of many possible names for this academic trajectory, along with foresight and many others. In his 1932 call for aid, “Wanted – Professors of Foresight!” H.G. Wells called for the development of a field of foresight, this new field would deal with unanticipated consequences that accompanied the development of new technology. The question for Wells: why are publics so reckless when confronting technologies that vastly increase the speed or range of processes? What additional skills could be brought to bear to more effectively engage with these problems:

There are no Professors of Foresight as yet, but I am by way of being an amateur. Let me draw a plain conclusion from tonight’s audition. Either we must make peace throughout the world, make one worldstate, one world-pax, with one money, one police, one speech and one brotherhood, however hard that task may seem, or we must prepare to live with the voice of the stranger in our ears, with the eyes of the stranger in our homes, with the knife of the stranger always at our throats, in fear and in danger of death, enemy-neighbours with the rest of our species. Distance was protection, was safety, though it meant also ignorance and indifference and a narrow, unstimulated life. For good or evil, distance has been done away with. This problem of communications rushes upon us today – it rushes upon us like Jehu the son of Nimshi. It drives furiously. And it evokes the same question: is it peace?

Because if it is not to be peace foreseen and planned and established, then it will be disaster and death. Will there be no Foresight until those bombs begin to rain upon us?[6]

This is a conservative idea, Wells calls for a futurism that could imagine a peace that could be created with existing technology. Distance is gone, the question becomes how to deal with closeness in the name of peace. The default condition lacked contact, now that contact has been established, how do we deal with it?

In this same time period, the Futurist movement in Italy took the opposite approach: instead of preserving or creating piece, conflict was desirable. For the futurist, nostalgia is the problem, an oppressive force that prevents the technologies of acceleration from transforming society in new profound ways. Consider this excerpt from a futurist work by Martinetti:

This is how we deny the obsessing splendor of the dead centuries and collaborate with victorious Mechanics, the force that grips the earth in its network of speed.

We are collaborating with mechanics in destroying the old poetry of distance and wild solitudes, the exquisite nostalgia of departure, and in its place we urge the tragic lyricism of ubiquity and omnipresent speed.

Our Futurist sensibility, in fact, is no longer moved by the dark mystery of an unexplored valley, of a mountain pass that we, in spite of ourselves, picture as crossed by the elegant (and almost Parisian) ribbon of a white road, where an automobile gleaming with progress and full of cultured voices abruptly pulls up, sputtering; a boulevard corner camped in the middle of solitude.

Every pine woods madly in love with the moon has a Futurist road that crosses it from end to end. The simple, doleful reign of endlessly soliloquizing vegetation is over.

With us begins the reign of the man whose roots are cut, the multiplied man who merges himself with iron, is fed by electricity, and no longer understands anything except the sensual delight of danger and quotidian heroism.[7]

The sensibility here should remind you of the ideology of contemporary technology conglomerates. It isn’t that technology makes things better, but that technology transforms all of life, and those ways that came before are not simply obsolete but regressive. We should consider this not to celebrate futurism, but to see how this set of ideas about speed and destruction recur. Schumpeter did not invent creative destruction – it was baked into the aesthetics of this movement.[8] Martinetti pushes us toward an anti-romantic view of the world. At the same time the masculine ideal of this movement is exclusionary, the celebration of rootlessness costs the stability of the tree. When Mark Zuckerberg promoted the slogan “move fast and break things,” it was intended to exemplify the challenge to the status quo.[9] This was a new kind of organization that wouldn’t follow rules. Now, a decade later, we can see that rules of political communication and media ethics were hard won and necessary. Zuckerberg wasn’t new, the Futurists understood the appeal of destruction and the power of novelty.

In 1967, the American Academy of Arts and Sciences published “Toward the year 2000: Work in Progress.”[10] Unlike the romantic appeal for peace of Wells or the anti-romantic zeal of Martinetti, Daniel Bell’s position comes closest to ours in this book (and likely your course):

Time, said St. Augustine, is a three-fold present: the present as we experience it, the past as a present memory, and the future as a present expectation. By that criterion, the world of the year 2000 has already arrived, for the decisions we make now, in the way we design our environment and thus sketch the lines of constraints, the future is committed. Just as the gridiron pattern of city streets in the nineteenth century shaped the linear growth of cities in the twentieth, so the new networks of highways, the location of new towns, the reordering of graduate-school curricula, the decision to create or not create a computer utility as a single system, and the like will frame the tectonics of the twenty-first century. The future is not an overarching leap into the distance; it begins in the present.[11]

Bell sees change as occurring in systems. The contributors to the project span the social sciences with a range of prescient insights about the power of computer systems to transform decision making and emerging technologies. This model of future study depends on systems theory and a clinical detachment from what a future would be like. The account in this book differs in that we are not concerned with abstract visions, such as Daniel Patrick Moynihan’s thoughts on the near future of non-educational transfers in cooperative versus wedding cake federalism, but in the creation of things and experiences.[12]

This is not an exhaustive list of works on the future, there are many more that could easily fall within the purview of a course on the History of the Future and that in a course on the future of futures would be relevant. It is entirely possible that your instructor will include a great many more futurists for your consideration during lecture or in other readings. What I want to pull out of these three works in particular are three themes:

- Wells sees the risk of collapsed distances and accelerating systems. He calls to the fore assumptions about the conditions that stabilize systems that could be eroding. Future thinking can lead to peace.

- Martinetti sees the future as a chance for a new aesthetic, a chance to throw off the restrains of the old order. Future thinking can lead to more productive conflict.

- Bell sees the future as a logistical reality. Future thinking emphasizes the conditions of possibility of the present and probabilistic models of what is to come.

1.3 Key Concepts

In this section, I will be describing a number of key concepts in the study of future media. These theoretical interventions will be helpful as they allow you to effectively sort a large number of ideas and see the development of arguments over time. At first this might appear to be jargon — overly technical and specific. Experts use specialized vocabulary for the sake of efficiency. There is no reason why we should say a paragraph when a well-established word would do. By reaching agreement about key discourses and ideas, further ideas can be developed more quickly and with greater depth. Spending time considering the theoretical structure is important in this sense as it allows us to have a discussion about key concepts and classes of ideas without laboriously naming every concept. Intentionally obfuscating ideas is a problem, but we should also be reticent of the idea that all non-technical ideas should be reduced to sound bites or simple binary oppositions. These concepts need to be interesting enough to get at the debates of this time, but not confusing.

1.3.1 Continuity and Rupture

One of the most important tropes in this culture involves the term “modern.” What it is to be modern is to be current and enlightened, modern is new and smart. That which came before is backward. Bruno Latour developed this idea in his book We Have Never Been Modern, where the assumption that the rift between prior practices and new practices is called starkly into question.[13] By pushing off the old to the pre-modern false novelty provides an illusion of knowledge, a distinction without a difference. This discourse appears in many forms.

James Carey and John Quick described the idea of the electrical overturning of social structure as the “electrical sublime.”[14] The utopian hope that electrification would transform social relations has been an ongoing theme. Carey and Quick describe the Innis-McLuhan exchange, where Canadian theorist Harold Innis argued that electrification would only continue existing power relations, and McLuhan took the position that electrification would enable new modes of life that would restore our everyday space.[15] Innis was not opposed to technology, you likely hear the oppositional voice on a nearly daily basis. The point is that ubiquitous technology is neither the key to utopia or the gateway to despair. Questions of value and structure exist independently of the technical details of society. The ubiquity of electricity transformed society, but not in the mythic dimension of producing an entirely new human.

Vincent Mosco made a similar case in his critique of the digital sublime: the ostensibly new digital world had entirely different rules and marked a transformation in the ways that things are done.[16] It is not the single online video that transforms televisuality, but the Netflix platform delivered through multiple devices. Cultural theorizing that relied on observations of first adopters would miss the actual interactions of the multitudes of users who had not yet arrived online. Instead of online interaction leading to a destabilization of identity, the identifiers of the users were amplified and the editorial function of prior bottlenecks decreased.

The meta-analysis of prediction offers further warnings. Philip Tetlock argues in his classic, Expert Political Judgement, that predictions by experts can be scored for relative accuracy.[17] The general outcome of the study suggests that well-designed formal models do an excellent job predicting the future.[18] People with open minds and the liberal arts sensibility, my way of describing “foxes,” do reasonably well. The more deeply entrenched in a particular world-view, the worst the predictive accuracy, with undergraduates coming in last. Perhaps this is why we have curriculum committees and professional advisors. The best predictions would seem to come from people who have a fox-like cognitive style, with reduced hindsight bias, a higher propensity for integrative thinking and cautious probability judgements, with few attempts to invoke belief defense mechanisms.[19] Tetlock’s advice was in my mind throughout the planning and execution of this textbook:

We often learn we have gone too far in one direction only after it is too late to pull back. Executing this balancing act requires cognitive skills of a high order: the capacity to monitor our own though processes for telltale signs of excessive closed or open-mindedness and to strike a reflective equilibrium to cultivate the art of self-overhearing, to learn how to eavesdrop on the mental conversations we have with ourselves as we struggle to strike the right balance between preserving our existing worldview and rethinking our core assumptions. This is no easy art of master. If we listen to ourselves carefully, we will often not like what we hear.[20]

You should also keep this in mind through your life, especially the part about not changing your world-view too quickly. This is not a call to change your mind quickly, but to really think about how you think.

For the most part there is a great deal of continuity – people and their desires remain fairly similar over time. There are also moments of rupture. Theorizing these moments is far more interesting, often taking far more energy and attention that theorizing the continuation of the present. It is critical to balance our theorizing of that which changes and does not change.

At the same time, Nassim Taleb has argued that the approach to theorizing from continuity is backward. The optimal theory for the future in this view would depend on the analysis of structures from the perspective of “black swan” events and more complex dynamics that come with non-linear systems.[21] Antifragility is a critical contribution by Taleb for our theoretical approach – instead of assuming that systems evolve toward some harmonious order, he proposes a rigorous accounting of structures, forces, or ideas that thrive on disorder.[22]

Changes are real and theorizing the big ones is important. In the context of a future oriented media studies be careful not to confuse the possibility of rupture with the likely continuation of the status quo.  At the same time, beware of the convenient continuation of the status quo – it can change.

1.3.2 Convergence and Emergence

Among the most interesting and important features of any theory are those that explain the relationships between micro and macro factors. Process development often hinges on factors that are difficult to observe, exist between levels of explanation, or are paradoxically hidden by the very constructs that would make them meaningful in the first place.

When we describe emergence, it is not that some media are “emerging” but that some ideas appear as constellations that then are recognized only once they are in effect in the world. Emergence and convergence are not opposites. It is important to note that this book is written from the perspective that emergence is not unobservable or unknown, but along the lines proposed by Mario Bunge where emergence is a combination that produces novelty:

In other words, we explain the emergence, behavior, and dismantling of systems in terms not only of their composition and environment, but also of their total (internal and external) structure. Nor is this enough: we should also know something about the system’s mechanism or modus operandi: that is what process makes it behave – or cease to behave – the way it does.[23]

A certain structural functional logic can guide our analysis of emergence in media systems. Our emergent combinations are not the mystical combination of parts which make more than their sum, but are embedded in complex assemblages that are already designed to incorporate the possibility of desire. These models also have assumptions and rhetorical frames, generally the social designs that are supposed are biological or mechanical. Despite this oversight, there are important lessons to draw from systems theory, one that is particularly pithy and for Bunge useful: don’t skip levels. A theoretical explanation needs to account for the micro, mezzo, and macro, even if just in a cursory way.

Convergence on the other hand implies that two things are merged together. This can be more or less intentional. Sociology and anthropology converge at the cultural dimension of meaning and the model of structures. These modes of convergence do not produce novelty, instead they are ways of arriving at particular structural functions or changes through combination. Convergence tends then to describe the ways that we discuss financial structures that allow a large conglomerate to function or the sort of devices that will provide us with an infinite supply of reruns.

Scalable, planned interventions collide with the everyday knowledge of the field. Michael deCerteau famously framed this as the distinction between the strategic and the tactical.[24] James C. Scott used an analog of this insight in his critique of high modern social planning. Plans fail because the way that planners see for scale makes it almost impossible to comprehend the situation in the same way that people on the ground do. When we think about successful convergence, as expressed as a transmedia property, the result is the opposite of novelty. Exposition of an already existent novel story system is the most effective way to generate a return.

This is not to say that convergence cannot produce novel results, but that if the overt design of a system is to produce more of the same it seems unlikely that the conditions for novelty will be truly present.

Figure 1.1: A picture I took on Kings in Corvallis.

What do you notice in this picture? A poorly placed sidewalk. The users of this environment have a clear preference to walk directly ahead, down the sidewalk that was once placed in this location. Now the sidewalk has been moved slightly people continue to walk where they want.

Desire lines exist in many places, you likely know of locations where the sidewalks were laid out as a grid where people would clearly prefer curves or angles. Robert Moor, reviewing the problem of desire lines, noted that the policy of Central Park in New York City had been to pave the desire lines: to use them as a guide to where sidewalks should go.[25] If they had followed this approach the park would have been filled with sidewalks. Purely emergent sidewalk design also fails. The question for designers: how can we balance the factors, honoring the desire of the users of a space without destroying the experience of it?

1.3.3 The Conditions of Possibility

A. The Simple Conditions

There is an important distinction to begin with between necessary and sufficient. Consider the development of a fire, it is necessary for fuel, a source of oxygen, and heat to be present for a fire to ignite. Remove any one of these three necessary elements and there are no longer are sufficient conditions for fire. Warmth and air are a summer day.

Developed by Immanuel Kant in book The Critique of Pure Reason, the conditions of possibility argument provide important resources for media research as it avoids the search for pure forms.[26] Instead of a metaphysical position that treats human sensory experience as secondary, Kant produced a system that allows sensation to be the primary focus of philosophy. Within the world of experience, Kant considers some experiences to be special, those that lift us up out of our normal perception – Slavoj Zizek proposes that these experiences are double, they are both sublime and disappointing as they remind us that we exist in a world of perception. Excavation of the conditions of possibility for the media present is an essential task for future studies. The following are two examples of this sort of analysis:

First, in his 2005 classic, Convergence Culture, Professor Henry Jenkins argues that new participatory cultures will be enabled by the convergence of media technologies.[27] When students encounter the book they are often quizzical: they live in a nearly completely converged world, the idea of medium specificity or a rigid break between the internet and the television is alien. Convergence is a fact of their lives, it did not have the positive and progressive implications described in the book.

What readers miss in the account of convergence culture is that the underlying drive would be that of a robust culture encountering lower barriers for interaction online. It was not the convergence of the devices that would have transformed social life, but the changing culture. Accounting for the forces within the convergence story is the reason why we assign this book to this day.

Second, it would be folly to say that all of the implications of convergence culture would have been possible if the culture had simply tried harder. Digital Non-Linear Editing software transformed the workflows of the contemporary media producer. Rapid, ubiquitous time-axis manipulation of video is remarkable and definitely necessary for the development of our current media culture. DNLE did not cause social change alone – it was merely a critical part.

In more concrete terms, the conditions of possibility for a thing are all the things that must be true for it to exist. A house with wooden studs requires a timber industry to produce materials, the entire chain of material operations necessary to make the house are required, but are not necessarily apparent in the consideration of the style of the windows. The conditions of possibility are often invisible and taken for granted.

Distinguishing between necessary and sufficient in this case assumes causation. Although metaphysical speculation is interesting and occasionally useful, for our purposes we can assume that there are causes and effects in this world. Causation is special, and mere correlation is blocked from taking on the power of a cause.

B. Episteme

This analysis of the procession of ostensibly invisible forms is apparent in discourse as well. In his remarkable book, The Order of Things, Michael Foucault describes an episteme, which investigates the discursive conditions of possibility for the present.[28] The layering of ideas and the progress of those ideas can also be excavated for analysis, this task is called genealogy. Foucauldian analysis asks the reader to consider the history of an idea and to take seriously the idea that one system of ideas can inflect another.

A powerful effect of this shift is the “death of the author.”[29] Roland Barthes criticized the romantic genius and the way that the idea of the author allows a search for a “secret, an ultimate meaning, to the text.”[30] This insight has been found in other communication fields as well, Ed Black in the critique of neoaristoelianism: we should judge speeches on the basis of their effect in circulation, not in the intention of the speaker.[31] The horizon of meaning must exceed individual intent.

Michael Foucault goes further to attack the institution of authorship and the privilege of the subject in producing text. This post-structural provocation is powerful, as many of the technologies considered in this book, and in communication research today, involve autopoesis – texts produced by automation. Think of the authorship of a Facebook feed – the means by which the feed you view was produced is the selective production filtered by relevance and recency of content created by a number of other people. The website/app load you experience is untouched by human hands: there was no author as such. This does not mean that the assumptions that were used to produce the program building are somehow non-human. At this point we tend to infuse the creators of systems like this with the romantic genius quality of the author. Characters like Mark Zuckerberg or Bill Gates replace Shakespeare.

Staging the larger debate about the role of structure and human agency is critical. Some scholars emphasize the profoundly human dimension of communication, framing research through the stories of people. This anthropological strand of communication research is important and stands in juxtaposition to the sociological strands that would focus on the mathematics of diffusion, or the critical/cultural which would decenter the story of the actor and the network for a genealogy of the discourses which made sense of both the network and the actors. The disciplinary matrix of Communication will be explored at length in another section of this book.

How we understand human agency is a profoundly important episteme.

C. Modal Logic

Necessity requires that something be not possibly false. Contingency would allow a conjecture that would be possibly true and false. Those claims which are truly necessary would be limited in the sense that that they would not include the conjectural information. Necessity is boring. Analytic results in general are powerful as they are restricted to simple qualities. The associated theory of positivism depends on the elimination of ambiguous or multiple signs. This presents an important limit on the use of analytic propositions for the study of communication.

If a connection to formal logic is desirable for you as a learner (or instructor) the theory proposed in this introduction would require a rigorous modal logic. Keep in mind, that we are not looking at single qualifiers, but hundreds of nested and reflexive logical structures. At some point, it will be necessary to suspend the expansion of the formation, this choice points toward the concern with infinite regression.

D. Modes of Proof

Mode of Reason

Description

Deduction

Working down from principles

Induction

Working up from examples

Reduction to the absurd

Working until the results are obviously wrong

Abduction

Working with the probability that a claim is true

It is important to consider the kind of proof you are employing. For the most part, you use reduction and abduction in everyday life. Deduction and induction are useful for mathematical processes but are difficult to find in the real world. Much of contemporary argumentation theory offers ways of theorizing the various logical leaps that are made with abductive reason (more on this later).

1.3.4 Time and Temporality

When are we? I ask this question often of students, there are many satisfying answers. Some answers conceive of time as an objective thing. As the agents of the Federation Bureau of Temporal Investigations explain to a befuddled Captain Sisko, “time is what keeps everything from happening all at once.”[32] There is a powerful truth here: time as we perceive it is real (Time is a condition of possibility – it surely exists, and to consider what it is would be fully speculative) and some events are path dependent. You cannot have microcomputers without transistors. This is the time of chronos: when we are and how events process. At the same time, without the performative dimension of the initial public offering, the moment of the microcomputer revolution would be unmarked.

Kairos positions time as a point, this is the moment of now.[33] The means by which the moment is produced are central to communication theory as a whole. Time as a moment is inherently synthetic. Chronos continues to proceed even if we ignore it. Time telescopes as you get older, facts that you once knew that seemed fresh and important can become painfully dated. Consider the way that people talk about electricity generation. In the early nineties, it was a meaningful thing to say that the “technology isn’t ready yet.” If this is baked into your conception of the current moment you have missed decades of innovation. Underlying truisms about how electricity moves. Gone are the days of Enron engineered blackouts, which made sense as California would be an energy importer.[34] California now has been known to send solar energy out to avoid over-supply.[35] Publics often remain in moments long after the clock has moved on.

Figure 1.2: US Central Intelligence Agency, 2012 https://commons.wikimedia.org/wiki/File:World_Time_Zones_Map.png

24/7, 365. A cliché intended to express that someone is continuously engaged or about an idea. The problem: there are more than twenty-four hours demarcated at any given moment on earth. Assuming that it is 1500 pacific daylight time on a Thursday, it is 1200 Friday on Teraina Island, and 1000 Thursday on Baker Island.[36] These times are GMT -12 and +14 respectively.

The standardization of time is a political process featuring precise engineering, not a scientific truism.

1.3.5 Probability

For the most part, students encountering this book will have been educated in a fairly standard model of frequentist probability. The basic dynamic presented is that of a random generator, usually presented as a fair coin, which then produces a string of results that are totally independent. Does a string of wins or losses mean anything in this context? No.

Instead of theorizing the probable in terms of a random device, increasingly the public is presented with dynamic scores that consider the probability of an event, given information, anterior and posterior to a new event that provides information. In an election, there is a likely winner and a likely loser. Adding new information like scandals or policy proposals can change the current probability of a win or loss.

The model for understanding the event of the upcoming election depends on an understanding of the relative durability of our assumptions. Adding new information, like a new poll, to the model would not change how we think of the race from a default position of 50/50, instead based on chronological time and the impending kairotic event, a score could be derived.

Simple probability of course remains, but is not particularly helpful in resolving the implications of multiple factors on a single relationship. If you are expecting probability to be handled in this book as the consideration of a pair of discrete outcomes, you will be lost. More on the mathematics and the Bayesian shift in Section four.

1.3.6 Speculative

A common adjunct term used in the discussion of the future is speculative. Dunne and Raby, in their book Speculative Everything, pose that speculative methods for design are intended to provide a grounded opportunity for the evaluation of potential worlds. Existing within the clear boundaries of the possible, speculation is intended to break the linkage between the probable and the plausible, allowing the consideration of the preferable.[37] Fantasy (and metaphysical speculation) are not particularly useful for speculative research.

Dunne and Raby take direction from Ricard Barbrook’s (known for his development of the Californian Ideology) position on future imagination, which begins with a consideration of the way that the image of the future has often been forgotten.[38] A limitless future had been promised repeatedly, none of the promises of current boosters are particularly unique.[39] In rhetorical studies, this appears in the use of the phrase “future anterior,” invoked as a device where a utopian future potential is leveraged against the present. This is in an important sense the inverse of “tarrying with the negative” where the scales of evaluation are shifted by the application of a nebulous negative value. Although Barbrook does not use this phrase in his writing, the critique comes through clearly in his descriptions of Bell Labs and the wildly optimistic presentations of participatory culture. The utopian promise of technology is always just around the corner, whether that is Walter Cronkite promising a world without hunger or swarms of robots making labor obsolete.

Among the central problems in the existing regime of design thought is the sort of vision employed by these organizations. Design has become a quick gloss for looking to aestheticize their products or plans. Those organizations have a high-modern sensibility which is intrinsically strategic – this idea already appears in the discussion of desire and communication. What speculative storytelling often does is emphasize the everyday dimension, the sort of experience that high modern imagination loses.

Speculative design is intended as a political program that can unmoor the tools of design as an academic pursuit from the rough docks of problem solving methods. Instead of a design theory that finds answers to questions posed by powerful institutions, speculation allows designers to find their own questions and to design for society, rather than a particular client. Problem solving is only one of a number of epistemic possibilities, speculation as much as it enables argumentation and debate, is an academic technology that can produce new knowledge in fruitful ways. It is not a new insight that design and argument are deeply linked, what is fascinating are the manifold of discourses presented to justify the lack of creativity in the design process itself.

Where the perspective by communication researchers differs from designers is that we generally are very interested in the ways that discourses would need to change to arrive at a possible future. It would be reasonable to conclude that communication is slightly more conservative in disciplinary outlook then design or architecture. It would also be reasonable to see this as a reflection of larger disciplinary coordinates, as communication is not locked into a problem-solving epistemology.

We should consider some of the methods for speculation proposed by Dunne and Raby:[40]

- Fictional worlds – literary and artistic contributions can challenge the stability of signs and promote new combinations

- Utopia/Dystopia – work through the ideas to either of the two extreme conclusions: the juxtapositions are productive

- Extrapolation – follow the dreams that lead to existing designs, let the dreams play out all the way to their conclusions

- Idea Stories – writing concepts as narratives; they use the example of red plenty (a new technological planned Soviet economy); use the narrative and look for resonances

- Thought Experiments – collide ideas in a non-narrative form, work with the abstraction of the formula

- Reduction to the Absurd – take the idea to the point that it fails and literalize it

- Counterfactuals – flip one of the actually flippable switches at a moment in history and suppose how that specific change would have affected the present

- What-ifs – flip one of the switches for the conditions of the present and work forward

The most important point: this is already how people work and think in design, the real reality is bracketed behind a discourse of problem-solving and reality that is itself a discourse. Speculation is powerful because it allows us to retake the imaginative language of design without being loaded into a static concept of reality. Within these categories play with the dimensions of narrative (concrete)/non-narrative (abstract), present experience/past memory/future expectation.

1.3.7 Virtual

Virtual does not refer to a device, be that goggles or a suit, but to the prospect of a synthetic perception. Brian Massumi, Canadian Communication professor and specialist in sensation and communication research, has argued that the virtual only exists in the combination of position and moment, as an effect of an endless loop of sensation, “When its effects are multiple, the virtual fleetingly appears. Its fleeting is in the cracks between and the surfaces around the images.”[41] Contemporary affect theory in communication has linked the physical, textual, and relational, “Affects are virtual synthetic perspectives anchored in (functionally limited by) the actually existing particular things that embody them.”[42] What does this mean? How you actually feel when you experience something matters. Your body and perspective are not barriers to understanding the world, they are the world. Andrew Murphie describes this as an enfolding, the multiple faces of what is ultimately a single surface.[43]

The virtual as a form of synthetic perception is deeply connected to the imagination. Once we establish a theory of virtuality that exceeds the sum of parts and perception, the analysis of the virtual comes to include physical and discursive considerations. Virtual worlds are then the worlds we inhabit as well as the imaginary worlds that we feel into possible existence. This book has an expansive orientation toward text, sensation, and technology because it is necessary.

1.3.8 Ideology

Ideology is a commonly used word, generally referring to a system of ideas that provide a coherence to thinking that exceeds the basic descriptive facts of the world. In this sense, everyone is ideological. If you were to remind someone that their world view was in a sense ideological they would likely be offended, there is a connotation in the term which supposes that an ideology is artificial. To consider what ideology is and why it is important, we should consider a few practical ideas.

How do we deal with people who have wrong perspectives? A straight-forward case here would be the consideration of individuals opposed to the vaccination of children. Vaccines are a safe and effective way to decrease the prevalence of infectious disease. The solution to non-vaccination would seem to be to challenge the ideology of the individuals, telling them that experts have determined that vaccination is safe. It must be that some bad piece of information is blocking their mind from arriving at the truth. Remove the bad block and they will think correctly. This doesn’t work. Brendan Nyhan and Jason Reifler’s research has been exemplary in demonstrating boomerang effects where seemingly ideology solving messages actually backfire, increasing the underlying belief.[44] What the seemingly crude theory of ideology misses is the idea that expertise, a form of backing that would be taken very seriously in some ideological frames, is negative in the conspiratorial frame of the anti-vaccine movement. Rather than operating as a bad idea that somehow clouds the mind, conspiracy discourse has a much richer symbolic life.

Conspiracy theories are a popular topic for research as they are a wicked problem for moving society forward. Jodi Dean, a leading political theorist, went as far as to ground the conspiracy as one of the foundational units for political analysis today.[45] The underlying structure of conspiracy includes the dominant duped view, which is maintained by a nefarious actor who knows secret information, that secret information would lead to a complete overturning of the dominant discourse. A participant in a conspiracy theory is not passive, they are actively working to reveal to you, the revealer the degree to which you are duped by ideology. Elizabeth Anker has argued that the dominant affective position of American politics is melodrama, of which conspiracy is a key form.[46] Conspiracy theories are both satisfying and practical. Qanon conspiracy discourses allow supporters of the Trump administration to incorporate bad news into their framework by inverting the roles of other characters in the drama.

If one wears a Q shirt to a rally and demands the release of the OIG report, they likely are aware of the controversy that is the Qanon conspiracy theory and have considered it as such. This is where Dean, citing Zizek and Sloterdjik, have formulated ideology as “enlightened false consciousness.”[47] People know that there are inconsistencies in their beliefs but they choose to continue. Ideology is not something that happens to people like a nightmare where they should be woken up, instead ideology is something people do for themselves to make their worlds. Ideological critique has languished in recent years as the mere identification of an ideology means very little and the application of new information likely doesn’t lead to attitude change.

This is not to say that attitude change is impossible, we have decreased tobacco and increased condom use, but that the underlying relationships around belief are not linear or based on simple delusion. Joshua Kalla and David Brockman have found that persuasion related to social issues, such as gay marriage, is possible but most effective when not tied to an impending political measure.[48] People are willing to have interesting conversations, as long as they are not motivated as such.

It also becomes clear why marketplace of ideas models fail – ideas are often mislabeled, mishandled, and the buyers are often also sellers already coming to market with strict shopping lists. Changing attitudes depends on affective change, a virtual dimension, that is much more interesting for our consideration of potential futures.

Figure 1.3: person filling up soda cup at soft drink dispenser by Franchise Opportunities, 2017. www.franchiseopportunities.com/www.franchiseopportunities.com/

In the May 2012, New York Mayor Bloomberg proposed a ban on giant cups of soda. The reasoning: the consumption of sweetened beverages is a public health problem, if people were made to “double-fist” their nectar they might drink less. You could still buy a ton of soda, it would just be less convenient. The push back was intense: limiting people to 16 ounce cups was a major loss of freedom. Eventually, the regulation was struck down as exceeding the authority of the department of Health and Mental Hygiene.

Behavior change is important, especially in the context of population health. If a substance is truly dangerous it is highly regulated. Tobacco and alcohol are good examples here. What do we do with times when the case for regulation isn’t so clear or when an overt ban would be heavy handed? Cass Sunstein (a law professor) and Richard Thaler (a behavioral economist) proposed a theory called libertarian paternalism where instead of overt strong prohibitions on conduct a series of small changes in design called nudges could be employed to subtly change behavior.[49] Changes in “choice architecture” could lead to different results by manipulating: defaults, expected error, action mapping, feedback mechanisms, layouts, and incentives.[50] If one were to simply change the context around the individual, they would make the “right” choice. What is striking here is the resonance between this position and the crude theory of ideology. There are many times when better designed systems can produce better results, but those situations will rarely align with practical politics or the leverage of the state.

1.3.9 Accelerationism

Just as the futurist aesthetic challenges the provincialism of slowness, vegetation, and romanticism, the accelerationists now challenge the axiological assumptions about slowness, stillness, embodiment, mindfulness (and many others) of contemporary theory. The accelerationist turn calls for the assessment of the choice to decelerate and the theoretical constructions associated with traditional humanistic critique. A key reference point in the literature on accelerationism is the Marxian claim that capitalism collapses because of its own internal contradictions, the dependence of accelerationism on this foundation is also contested.[51] Why would this point matter? If we have some predictable end point to social process, it would seem reasonable that if we could engineer that process to accelerate could be beneficial. The inevitable collapse narrative is convenient, but misses the key point made by Friedrich Pollock that as market systems strain under their own contradictions they tend to become authoritarian fusions that he calls “state capitalism.”[52] Depending on which core classes you have taken, you can see a lurking debate about the nature of social theory developing here.

Without the broader consideration of social theory, we could also see accelerationism as the choice to embrace contemporary technology. Once the choice is made to embrace technology, the study of the internal structure of capitalism as it is, abstraction, and acceleration, the range of possibilities for both research and politics dramatically increase. Williams and Strnick’s “#Accelerate – Manifesto for an accelerationist politics,” poses a different accelerationist future. The traditional points for the critique of the human sciences as romantic returns to what is increasingly a fantasy world. The future is taken by those who actively deploy the tools of modernity toward their own ends:

To do so, the Left must take advantage of every technological and scientific advance made possible by capitalist society. We declare that quantification is not an evil to be eliminated, but a tool to be used in the most effective manner possible.

…

The tools found in social network analysis, agent-based modeling, big data analytics, and non-equilibrium economic models, are necessary cognitive mediators for understanding complex systems like the modern economy. The accelerationist Left must become literate in these technical fields.[53]

The left in the context of this extended block quote can just as easily mean humanists or social scientists or artists. For many years, it was fashionable to critique computational means of thought production and then a necessary defense for meaningful theoretical structures. Today, these moves keep academic debates frozen in time. Among the most important ideas to move beyond is the critique of mastery, the idea that the claim to use technology in a quantitative project was something of a claim to fully represent the world and control that representation.[54] Researchers need to be good with technology. Future media students need to deploy a combination of theoretical, cultural, and technical methods, they don’t need a sophisticated list of excuses for why they don’t know how to do things.

More metaphysically, Nick Land, a key accelerationist theorist, argues that the accelerationist moment is a feedback loop.[55] By unleashing cybernetic power the accelerationist turn could potentially enable the singularity, which is the concept that a powerful enough computer could allow the uploading of all intelligence into a single artificial meta-structure.[56] The question of the desirability of assimilation into a collective is another question entirely. For advocates of singularity, the prospect of brain-computer interfacing is exciting as it transforms the condition of possibility of embodiment. For those opposed, it is embodiment itself that is the heart of the human condition. At the same time, only the artificial intelligence of capable of producing the singularity can truly be said to be sufficient to cause such an event. It is entirely possible that brain-computer interfacing will never reach this level – more on this in section three.

Accelerationism can provide three important insights:

- If we assume that social processes are knowable and predictable, their engineered outcomes, if positive should be hastened. Fatalism is a choice made by humanists and social scientists, not a necessity.

- Humanistic critique often relies on implicit value assumptions that intrinsically conservative, and should be challenged or inverted.

- The possibility of a wildly divergent future with an alternative cosmology.

1.3.10 Simulation

The prospect that reality is an illusion has a long history across many human cultures. Sensation is not satisfying, there must be some other reality out there. Make no mistake, this is not a claim that the physical world does not exist, but that there is no higher essence that could somehow be beyond the world as we understand it.

Jean Baudrillard provocatively claimed that “the Iraq War did not take place.”[57] What he meant by this was not that there was no military conflict in Iraq, but that role of media performance in the war was such that it produced a new reality of war, a virtual world where one experiences the war through the vision of a military system attacking a building. The kinds of wars, and seemingly spectacular yet invisible costs, could dramatically recalibrate the choice to engage in armed conflict. At the highest level, this forms a simulacrum, a symbolic world more real than reality.[58] Escape is not an option, there is no way out of language, the alternative is to critique the most pernicious forms within our simulation. In opposition to the central thesis of accelerationism, that there is an end point that can be approached to history or a system of symbols, Baudrillard reminds us that there is no end point. History is always already in the dustbin as we are continuously remaking it, there is no end point that we are moving toward: just more discourse.

The most popular simulation topic today comes in the form of the simulation argument. Presented in this form by Nick Bostrom, we are asked to consider the possibility that we are currently living in a simulation. The essential premise of this argument is that it is likely that a highly technically advanced civilization would have seemingly infinite computing power.[59] From this point, the prospect that a civilization could run an ancestor simulation (a realistic virtual world that we are a part of) is possible, assuming that the processes by which such a civilization would come to pass would not be entirely self-destructive. Bostrom is thus not arguing directly that we live in a simulation, but that we should consider the conditions of possibility for arriving at the state of post-humanity where we might have seemingly infinite computing power.

Existential risk, the prospect that humanity or any human like civilization could be destroyed, becomes a central concern for the evaluation of possible futures.[60] HG Wells consideration in Professors or Foresight wanted, was the new technologies obliterated distance, it was not that a utopia of infinite communication was coming, but that the new technologies heralded new destructive possibilities. Simulation provides us a framework for considering what the world could and should be.

In a more concrete sense, deepfakes are a profound immanent problem. Deepfakes use neural nets to map images and sounds together.[61] Primarily used for the production of pornography, deep fakes allow the simulation of what would be real material. The status of photographic evidence has already been in decline for many years, the deep fake transitions from the world of the singular fake to the entire moving vivacious simulated fake. The reason why deep fakes are so vexing for the public sphere is their ability to fully break the chain of the indexical trace. Phillip Rosen argued that the fundamental quality of images in the public sphere is their capacity to providing evidence of having been there – that there was something real and evidence of action that could exist.[62] Metaphysical games are fun, pragmatic questions about the status of evidence in court point toward the danger of simulation.

Simulation is important in three ways:

- Philosophy has been concerned with the feeling that this is all an illusion or simulation for thousands of years: this is a foundational idea. These are tightly bound up with questions of the meaning of life, hope/despair, genesis/apocalypse.

- People are often searching for some trace of perception that is a life-line to the “real” world. No such connections exist.

- Simulations can appear to be more real than reality, are quite useful; dangerous.

1. Derek Thompson, “The History of Sears Predicts Nearly Everything Amazon Is Doing,” The Atlantic, September 25, 2017, https://www.theatlantic.com/business/archive/2017/09/sears-predicts-amazon/540888/. ↵

2. Wolfgang Schivelbusch, The Railway Journey: The Industrialization and Perception of Time and Space (University of California Press, 1986). ↵

3. Manuel De Landa, Philosophy and Simulation: The Emergence of Synthetic Reason (London; New York, NY: Continuum, 2011). ↵

4. Dean Burnett, “Jetpacks: Here’s Why You Don’t Have One | Dean Burnett,” The Guardian, September 23, 2014, sec. Science, https://www.theguardian.com/science/brain-flapping/2014/sep/23/jetpacks-science-scientists. ↵

5. John Peters, “Democracy and Mass Communication Theory: Dewey, Lippmann, Lazarsfeld,” Communication 11, no. 3 (1989): 199–220. ↵

6. H G Wells, “Wanted – Professors of Foresight!,” October 1, 2018. ↵

7. F.T. Martinetti, “We Abjure Our Symbolist Masters, The Last Lovers of the Moon,” in Futurism: An Anthology (New Haven: Yale University Press, 2009), 94. ↵

8. Joseph Schumpeter, Capitalism, Socialism and Democracy, Reprint. edition (Harper, 1975). ↵

9. Nick Statt, “Zuckerberg: ‘Move Fast and Break Things’ Isn’t How Facebook Operates Anymore,” CNET, accessed October 2, 2018, https://www.cnet.com/news/zuckerberg-move-fast-and-break-things-isnt-how-we-operate-anymore/. ↵

10. This book is a fascinating achievement that covers much of the same potential ground as this book. Daniel Patrick Moynihan, “The Relationship of Federal to Local Authorities,” in Toward the Year 2000: Work in Progress (Cambridge, MA: MIT Press, 2000). ↵

11. Moynihan, 1. ↵

12. Moynihan, “The Relationship of Federal to Local Authorities.” ↵

13. Bruno Latour, We Have Never Been Modern (Harvard University Press, 2012). ↵

14. James W. Carey and John J. Quirk, “The Mythos of the Electronic Revolution,” The American Scholar 39, no. 3 (1970): 395–424 ↵

15. The critique of McLuhan’s utopianism appears in Bell’s work as well. ↵

16. Mosco, Vincent, The Digital Sublime (Cambridge: MIT Press, 2006). ↵

17. Phillip Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, N.J: Princeton University Press, 2006). ↵

18. Ibid, 76. ↵

19. Ibid, 143. ↵

20. Ibid, 215. ↵

21. Nassim Taleb, The Black Swan: Second Edition: The Impact of the Highly Improbable Fragility", 2nd ed. (New York: Random House, 2010). ↵

22. Nassim Nicholas Taleb, Antifragile: Things That Gain from Disorder (Random House Trade Paperbacks, 2014). The assumptions of order and natality are among the most important reasons for the continuation of psychoanalysis as a field. ↵

23. Mario Bunge, Emergence and Convergence: Qualitative Novelty and the Unity of Knowledge (University of Toronto Press, 2015). ↵

24. De Certeau, Micheal, The Practice of Everyday Life, 2nd ed. (Berkley: University of California Press, 2002). ↵

25. “Tracing (and Erasing) New York’s Lines of Desire | The New Yorker,” accessed October 2, 2018, https://www.newyorker.com/tech/annals-of-technology/tracing-and-erasing-new-yorks-lines-of-desire. ↵

26. Slavoj Žižek, The Sublime Object of Ideology (Verso, 1989), 203. ↵

27. Henry Jenkins, Convergence Culture: Where Old and New Media Collide, n.d. ↵

28. Michel Foucault, The Order of Things (Routledge, 2005). ↵

29. Michael Foucault, “What Is an Author?” (Lecture, 1969). ↵

30. Roland Barthes, Mythologies, trans. Jonathan Cape (Paris: Noonday Press, 1991). ↵

31. Edwin Black, Rhetorical Criticism: A Study in Method (Univ of Wisconsin Press, 1978), 75. ↵

32. “‘Star Trek: Deep Space Nine’ Trials and Tribble-Ations (TV Episode 1996) - IMDb,” accessed October 2, 2018, https://www.imdb.com/title/tt0708655/. ↵

33. John Durham Peters, “Calendar, Clock, Tower” (Media in Transition 6, Massachusetts Institute of Technology, 2009), http://web.mit.edu/comm-forum/mit6/papers/peters.pdf. ↵

34. Julian Borger, “Tapes Reveal Enron’s Secret Role in California’s Power Blackouts,” The Guardian, February 5, 2005, sec. Business, https://www.theguardian.com/business/2005/feb/05/enron.usnews. ↵

35. Ivan Penn, “California Invested Heavily in Solar Power. Now There’s so Much That Other States Are Sometimes Paid to Take It,” www.latimes.com, accessed October 2, 2018, http://www.latimes.com/projects/la-fi-electricity-solar/. ↵

36. Spend some time looking at this time zone map and it will become clear that time zones are political. https://www.timeanddate.com/time/map/ ↵

37. Anthony Dunne and Fiona Raby, Speculative Everything: Design, Fiction, and Social Dreaming (MIT Press, 2013), 5.p 5 ↵

38. Richard Barbrook, Imaginary Futures: From Thinking Machines to the Global Village (Pluto, 2007), 8; Richard Barbrook and Andy Cameron, “The Californian Ideology,” Science as Culture 6, no. 1 (1995): 44–72. imaginary futures, 8 ↵

39. Barbrook, Imaginary Futures, 243. ↵

40. Dunne and Raby, Speculative Everything, 67–88. ↵

41. Brian Massumi, Parables for the Virtual: Movement, Affect, Sensation (Durham, NC: Duke University Press, 2002), 133. 133 ↵

42. Massumi, 35. ↵

43. Andrew Murphie, “Putting the Virtual Back into VR,” in A Shock to Thought: Expression After Deleuze (New York: Routledge, 2002), 188–215. ↵

44. Brendan Nyhan and Jason Reifler, “When Corrections Fail: The Persistence of Political Misperceptions,” Political Behavior 32, no. 2 (June 1, 2010): 303–30, https://doi.org/10.1007/s11109-010-9112-2. ↵

45. Dean, Jodi, Publicity’s Secret (Ithaca: Cornell University Press, 2002). ↵

46. Elisabeth R. Anker, Orgies of Feeling: Melodrama and the Politics of Freedom (Duke University Press, 2014). ↵

47. The key idea in the psychoanalytic critique of ideology is to acknowledge that people are often actively participating in the reproduction of a discourse, they are not duped by ideology, they are manufacturers of it. As a theoretical construct, this compliments the social science research and provides a forward looking sense of the narrative structure around ideology today. Peter Sloterdijk, Critique of Cynical Reason (Minneapolis: University of Minnesota Press, 1987); Žižek, The Sublime Object of Ideology; Dean, Jodi, Publicity’s Secret. ↵

48. Joshua Kalla and David E. Broockman, “The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments,” SSRN Scholarly Paper (Rochester, NY: Social Science Research Network, September 25, 2017), https://papers.ssrn.com/abstract=3042867. ↵

49. Richard H. Thaler and Cass R. Sunstein, Nudge: Improving Decisions About Health, Wealth, and Happiness (Penguin, 2009). ↵

50. Richard H. Thaler, Cass R. Sunstein, and John P. Balz, “Choice Architecture,” SSRN Scholarly Paper (Rochester, NY: Social Science Research Network, April 2, 2010), http://papers.ssrn.com/abstract=1583509. ↵

51. This introduction is an essential resource for understanding accelerationism. Robin Mackay and Armen Avanessian, “Introduction,” in #Accellerate: An Accellerationist Reader (Fairmouth, UK: Urbanomic, 2014). ↵

52. Friedrich Pollock, “State Capitalism: It’s Possibilities and Limitations,” in The Essential Frankfurt School Reader, ed. Arato and Gebhardt (New York: Continuum, 1985). ↵

53. Alex Williams and Nick Srnicek, “#Accellerate,” in #Accellerate: The Accellerationist Reader (Fairmouth, UK: Urbanomic, 2014). ↵

54. Williams and Srnicek, 360. ↵

55. Nick Land, “Teloplexy,” in #Accellerate: The Accellerationist Reader (Fairmouth, UK: Urbanomic, 2014). ↵

56. “The Singularity Is Near » Homepage,” accessed October 2, 2018, http://singularity.com/. ↵

57. Jean Baudrillard, The Gulf War Did Not Take Place (Indiana University Press, 1995). ↵

58. Jean Baudrillard, Simulacra and Simulation (University of Michigan Press, 1994). ↵

59. Nick Bostrom, “Are You Living in a Simulation?,” Philosophical Quarterly 53, no. 211 (2003): 243–55. ↵

60. Nick Bostrom, “Existential Risks: Analyzing Human Extinction Scenarios,” Journal of Evolution and Technology 9, no. 1 (2002), https://nickbostrom.com/existential/risks.html. ↵

61. “How ‘Deep Fakes’ Became Easy — And Why That’s So Scary,” Fortune, September 11, 2018, http://fortune.com/2018/09/11/deep-fakes-obama-video/. ↵

62. Philip Rosen, Change Mummified: Cinema, Historicity, Theory (Minneapolis: University of Minnesota Press, 2001). ↵

Section 2 – Where change is unlikely

Somethings do not change. Consider the humble triangle. For all triangles including a right angle, formula a^2 + b^2 = c^2. For Euclidian geometries, this formula describes the length of the sides of that triangle. Unless there are dramatic changes in the Universe, triangles will continue to have three sides, three angles, and operate according to some very simple rules.

The items in this section are intended to provide a foundation that allows us to understand the conditions of possibility for the discussion of things that can change, or will change. Some of these items may seem bizarre – they are included because of the extensive virtual horizon of media moving forward, this book is intended for classes that consider headsets, bodysuits, and brain-computer interfaces just as seriously, or likely more seriously, than they consider the design of a front page of the paper for the Dodgers winning the World Series.

Each sub-point in this section will identify something that is unlikely to change, some important basic information about it, and the practical consideration of how that unchanging property impacts your world.

2.1 The Electro-Magnetic Spectrum

NASA (the National Aeronautics and Space Administration) provides this handy chart:

Figure 2.1: National Aeronautics and Space Administration, Science Mission Directorate. (2010). Introduction to the Electromagnetic Spectrum. Retrieved January 2nd, 2019, from NASA Science website: http://science.nasa.gov/ems/01_intro

The spectrum extends all the way from the long waves of radio through the short waves of positron imaging and background radiation in the universe. When you screw the coaxial cable into your cable port for your television or modem you are attaching a metal conductor for a radio signal.

All electromagnetic radiation is similar, taking this block quote from NASA:

Electromagnetic radiation can be described in terms of a stream of mass-less particles, called photons, each traveling in a wave-like pattern at the speed of light. Each photon contains a certain amount of energy. The different types of radiation are defined by the amount of energy found in the photons. Radio waves have photons with low energies, microwave photons have a little more energy than radio waves, infrared photons have still more, then visible, ultraviolet, X-rays, and, the most energetic of all, gamma-rays.“[1]

Why do we care about this then? Almost all of our contemporary image media and network driven technologies depend on the spectrum. All optical media are spectrum dependent: light interacting with paint and screens both rely on the same underlying spectrum dynamics of color.

Does that mean that your microwave is dangerous?

No. Your microwave uses a faraday cage to contain its electromagnetic energy. The same cages are useful for computer security – a faraday bag is important for containing suspicious electronic devices, and faraday cages protect electrical equipment.[2] What is the magic here? A web of metallic bars or strands.

This will not change in your lifetime. The fundamental units of all systems will be somewhere on the spectrum.

2.1.1 Scarcity

Consider a car radio, as you drive away from a city the signal from your favorite vaporwave station becomes weaker and weaker as you move away until you can no longer hear it at all. Of course, such a hip station would use the FM method of encoding as it allows greater fidelity to the original (it sounds better). At that point, you might switch to an AM station from that same city which carries broadcasts of stand-up comedy. As you approach the next city you may find the same frequencies on which you enjoyed vaporwave and jokes carry jazz and sports. In any given place, there cannot be two transmissions on the same frequency. The waves cancel each other out: this is a key property of waves.

The principle of scarcity is so important that it is the basis of broadcasting law. In RTNDA v. Red Lion, the Supreme Court made clear that the scarcity of broadcast spectrum allowed the restraint of speech using radio.[3] Deciding what would be carried on a rivalrous channel over the air would be a matter of public concern. In the Tornillo decision, the Supreme Court found the opposite in the case of newspaper: unlike a radio station they are not rivalrous.[4] Economic, rather than electromagnetic, justifications for restriction run afoul of the first amendment.

This is also the basis of policy discussion in the context of net neutrality. Although at the upper end of utilization bandwidth through the internet may have a physical limit, for most intents and purposes it is a non-rivalrous resource. Advocates of net neutrality contend that such an important electromagnetic service should be considered under the same legal regime as telephone service: as a common carrier. Opponents contend that operators of internet systems should be allowed to recoup additional funds from heavy users of the infrastructure and that they may have a first amendment interest in editing the flow of information. More on this will appear in a later section as policy and network shaping technology are highly likely to change, what is unlikely to change is the rivalrous scarcity of bandwidth.

At this university, this course comes after a course on political economy and legal theory of new media, for those of you reading this without such a course this 2×2 matrix can be very helpful.

Rivalrous

Non-Rivalrous

Excludable

Private Goods: single use experiences or things. Food.

Club Goods: traditional cable television, concerts.

Non-Excludable

Common Goods: Goods that can be exhausted but are not easily controlled, open pasture land, oceans.

Public Goods: Ideas and other things that are not used up and are for everyone: ideas, air.

2.1.2 Optics

Light sources emit many photons on many wavelengths in many directions. A bright light source could be more clearly organized by the use of a lens that would organize the light created by the source. The Fresnel Lens is a great example of this. The ridges of the lens allow the light passing through to be organized to flow in a coherent direction.[5] Light can also be focused with a conventional concave lens. The lens on your camera is condenses the light that reaches it into a coherent beam that lands on some kind of sensor or film. When you consider this in the context of the light being electro-magnetic energy, it makes sense why you should not take a picture of the sun. All of the energy that the sun emits is scattered, the camera (or your eye for that matter) uses a lens to focus energy. All sensors in cameras, to be addressed later in this section, are energy collectors.

Polarizing filters use very fine lines to block the flow of unorganized light to a source, this is useful as it can dramatically reduce glare. This is why polarizing sunglasses may not allow you to see your phone, and is how transparent glass 3D systems are able to separate channels for the viewer.

Light can also be split apart into discrete wavelengths. A prism reveals the colors within white light, while mass spectrometer allows you to see which colors are absorbed or reflected by a particular object. This has a number of implications for how we understand the design of cameras, displays, and paints.

There are two ways that we can understand primary colors: additive and subtractive. Most students in this course will be familiar with subtractive theories of color. These theories contend that with a certain number of basic colors (primaries) all other colors can be produced. Typically, these are presented as red, yellow, and blue or more technically as cyan, magenta, and yellow. The base off-set printing process uses CMYK (where K is black).[6] Tiny dots of these colors could then be made and printed in proximity, which is perceived as a field of color. These different inks would absorb some frequencies of light while reflecting others. This is the subtractive primary base, as more colors are added the reflection profile of the material will move toward black.

Additive primaries exist in light itself. These primaries are red, blue, and green. The absence of light is black, while the presence of all lights is white. Each of these theories is meaningful and correct, they simply describe different optical properties of different media as they interact with light.

Figure 2.2: CMYK vs RGB Dungodung, modifications applied by DarkEvil, 2006 https://fr.wikipedia.org/wiki/Fichier:La_Boqueria_RGB_VS_CMYK.jpg

2.2 World Systems

World systems theory sees the collection of countries, geographies, and cultures as an evolving system over time. This approach is helpful for our analysis because it views these large institutions as a continuously evolving system, rather than historically specific units.[7] Studies of the future of media in this state system would call for the evaluation of core-periphery distinction, the flow of materials between countries, shifting frameworks for the interpretation of events and flows.

The inclusion of world systems here is not intended as a conservative note, but one that calls for rigor. Some of the weakest predictions that have come through popular media studies in recent years are those that underestimate the staying power of the state as a form of human organization. We make states because they are useful for doing things that we want with people who may have common cause with us. Some of the hardest won lessons of the last few years have been that new media technologies do not “laugh” at powerful institutions as some authors supposed in the 1990s that they would, but that they are often twisted to support the status quo.[8]

It is also important not to limit our consideration here to the state: large corporations, transnational activist groups, publics/counter-publics, trans-state organizations, militaries, and terrorist groups just to name a few are actors in this system. They are not going away. Facebook will not replace the government. Trade wars, a rising threat at the time of this writing, prove the power of the state over the economy and technology and the power of technology and economy to shape the state. Government should always be in your consideration of the future.

2.2.1 Geography

There will continue to be scarce resources and geographic barriers to the movement of people and things.

As of the time of this writing, one element that may appear to be unlikely to change is the distribution of rare earths, such as neodymium, which is critical for building magnets. It is important to note that rare earths are not actually rare, they are only seen as rare because of the negative externalities of mining and the sure cost of sorting through huge amounts of soil.[9] The rare earths are the lanthanides, which are just above the exciting radioactive actinides section of your handy periodic table. The things we use these technologies for, like MOSFET transistors are highly unlikely to change. How will we find the political will to secure our necessary supplies of these materials in a responsible way?

The ways that the configuration of space on earth shapes possibility should not be understated, many discourses organize the understanding of space and time on the basis of the physical layout of mountains, lakes, oceans, and many other physical forms. Geographic forms are interposed into other political discourses which then become co-productive.[10] What does that mean? Mountains may not change, the ways that we talk about manipulating mountains will.

2.2.1.1 Undersea Cables

Former Alaska Senator Ted Stevens is frequently lambasted for his remark that:

And again, the Internet is not something that you just dump something on. It’s not a big truck. It’s a series of tubes.[11]

This was initially intended as an argument that the internet was rivalrous, but the deeper point is worth thinking about: the internet is a physical thing. Among the most important reminders of this is the undersea network of cables that make up the backbone of the internet.

Edward Snowden revealed in his 2013 document dump with wikileaks that the National Security agency had tapped into the cable landings of other countries. Consider this map:

Figure 2.3:TeleGeography Submarine Cable Map: https://www.submarinecablemap.com

There are a finite number of good points where cables would be connected between countries, sniffing at these key points would allow a state actor to see a great number of messages. With that information, they could see to their national defense or provide their industries an advantage over those in other countries. From the other side, such points present key weaknesses. Cable cutting subs would sever these links in the first moments of a war. In other conflicts, military use of public telecommunications infrastructure has been a critical vulnerability.

2.2.1.2 Satellites

If physical linkages are not possible one important way of linking distant areas involves routing the signal up and away from the surface of the planet. Satellites are already a key form of logistical media, they allow your navigation system to know where it is, and process television signals.

Compared with links to fiber optic or coaxial cables, satellite transmissions require more power, are slower, and more expensive.[12] At one point Facebook commissioned blimps that would fly and produce conductivity across the surface.[13] Access to a nation’s sovereign airspace is no small matter, and the risk the distant service might displace the development of local infrastructures is quite real. At the end of the day, the local spatial fix that could be created through control of the telecommunication infrastructure outweighed the idea of the quick, cheap, foreign access point.

Beyond cable cutting, among the first major acts of the next World War will include the use of anti-satellite weapons, either with ground based systems, other satellites as weapons. The world of ubiquitous satellite access is a result of a lack of major interstate conflict, not technical necessity or inevitability.

2.2.1.3 Other Transmitters

All linkages depend on some interaction with a transmitter. The spectrum is limited, with different bands assigned for different uses.

Figure 2.4: Chart of Spectrum Uses – US Department of Commerce, National Telecommunications and Information Administration, October 2003 https://www.ntia.doc.gov/files/ntia/publications/2003-allochrt.pdf

Radio transmitters and receivers, like those in your cell phone, allow you a great deal of access to information. Transmitters will continue to be critical infrastructure. Any method that does not involve transmitters, would involve shifts in the laws of physics or biology that are well beyond this book, or the society where people would read it.

2.3 Transistors

The digital revolution depends on the transistor – a semi-conductive device that allows the solid state encoding of logic gates (they can also be used as amplifiers). For the most part, the transistors we are interested use a field effect. Before we get into the discussion of operators, it is useful to think about the two kinds of transistors: P and N. In order to augment the electrical properties of the semiconductors already used in transistor, specific materials can be added that change those electrical responses.[14] In the case of P doping, the region in the middle of the potentially conductive zone is treated with a material that will further react when exposed to a proximal electrical field. The P transistor will then turn off an underlying flow. An N doped semiconductor will turn it on.

At first you might think that this is a fairly low level innovation, these seem to be simple switches that express binary logic. What is powerful about these systems is that they can be produced in massive volumes at incredibly low prices. Replacing expensive vacuum tubes, transistors made it possible to build many more processing systems than were possible before. Even better, transistor based processors were so affordable that general purpose processors could replace specially built electronics in many cases. Transistors make software possible.

The design of logical operators is highly unlikely to change. All computationally legible information can be represented using transistor states of A/B. These are described through truth tables.

The following are descriptions of key logic gates:

AND

A

B

Output

0

0

0

0

1

1

1

0

1

1

1

1

In terms of practical transistor design, an AND gate requires two switches aligned in a series. Only if both switches are on will the current flow.

*To read these circuit diagrams, you can see inbound voltages from v, A or B. In these diagrams I see the world as nMOS, meaning that switches are flipped on by A or B, the voltage is aways present with v.

OR

A

B

Output

0

0

0

0

1

1

1

0

1

1

1

1

In terms of practical transistor design, the switches are parallel. If either switch is on the current will flow. The problem, is that we cannot easily distinguish between the values A and B. Either could be on.

As you can see, the operator OR is not particularly revealing, which leads to the gate XOR

A

B

Output

0

0

0

0

1

1

1

0

1

1

1

0

The principle of functional completeness, supposes that there are two gates that are functionally complete: NAND and NOR: all other gates can be constructed using either of these. NAND supposes that two inputs result in one…

A

B

Output

0

0

1

0

1

1

1

0

1

1

1

0

The transistor setup for the NAND gate is quite straight forward as the current would always be flowing across the active region of a P doped transistor. One of the trickier ideas here – how would a NAND gate produce a NOT gate? If both leads of a transistor were lead into a single input, the presence of that input would produce a 0 outbound signal. The leads of the transistor do not necessarily need to run to the same place.

The complement would be a NOR gate…

A

B

Output

0

0

1

0

1

0

1

0

0

1

1

0

Figure 2.9: Diagram of a CMOS NAND [by Dan Faltesek]

Notice the critical idea here: there are four transistors combined into a single circuit which includes both a NAND and a NOR, instead of the binary logic here being on/off the circuit encodes high/low. When both P type transistors block, the flow creating the NAND, the path for the NOR is opened.

Why is this such a powerful technology?

At first it might appear that the reduction of all information to binary might appear difficult and confusing. What is important to understand is that binary states rely on the same processing algorithms that you rely on every day. Consider binary addition, it relies on the same process that you use for adding any other numbers, the difference is that you carry the 1 whenever the sum is more than 1.

Notice that I am not required to carry until bit 3, where 1+1=0; carry the 1. In Bit 6 we see the next step where 1+1=0 carry the 1, but we have already carried a one, thus the result is one. Subtraction follows a similar process. On the level of the transistor, mathematical tasks become simple combinations of on and off.

Transistors and logic gates offer the brute force necessary for the simulation of any specific operation.

The underlying principle of semiconductivity is highly unlikely to change, the alternatives to existing semiconducting technologies in terms of semiconduction still rely on the Boolean logic of transistors, meaning that the core of the idea is likely here for the long haul. Beyond that, it is unlikely that quantum computers, spintronics or other such technologies can replace the signal amplification role of the transistor, after all, the transistor did not replace the vacuum tube.[15] Quantum computing is interesting, and could begin to supplant the transistor binary paradigm, but the technology is far further away than is generally acknowledged.[16] The period of time between the transistor and Facebook was over fifty years.

This should remind you of Gitelman and Pingree’s axiom: new media do not completely replace the old, they resituate and define the use of the others.[17] Moving forward we can use these ideas to understand other technologies and the possible limits of computation in understanding – the magical innovation of the digital is the possibility that all information could be quantized and processed through logic gates.

Summary:

- Transistors are durable, they have no moving parts.

- Moore’s Law: we are always finding ways to put transistors into a smaller space, at affordable prices.

- Uniform fields of transistors enable high-level abstractions. These abstractions are what we call software.

2.3.1 Heat

All circuits produce heat. Electricity is the movement of electrons and that physical motion really exists. Bitcoin miners struggle with adequate supplies of electricity, both for their systems and for their cooling. Integrated circuits include resistors, which intentionally deal with excess voltages as heat. Heat further increases resistance, which can cause other problems as well. Quantum computers, and high precision imaging transistors, must be kept cold. Dissipating heat is the essential task for design of new systems.

2.3.2 Abstractions

Many of our emerging technologies depend on augmenting layers of transistors. Understanding how those systems work and how they translate information into a meaningful form is absolutely essential for understanding the future. The transistor and the qubit mean little if there is no language to implement instructions on them. Even then, there is already vast computing power available. Aside from the calculation of np-hard problems, it is hard to see what new classes of operations will appear in this era. The most likely answer is that the kinds of abstractions that could be processed will go far further. This term will appear in another section, but it is useful to think of it now: the digital refers to the production of fast, useful abstractions.

2.4 Humans and Sensations

This section is not intended to defend the idea of humanism, but to say that the critical subject of this book is people. Those people have sensations. The questions of how those sensations can be produced and reproduced. In this section, we are concerned with understanding basic human physiological structures and the basic physics, chemistry, and biology of perception. Special attention will be paid to the idea of standardization, meaning the question of how we map different physical inputs into a sensation and what result may or may not be registered. This is a particularly powerful element of the analysis of the virtual as we must account for the ways that meaning can be actually produced.

Note that this is limited to a fairly banal conception of five senses. This section of this book is intended to concern that which is highly unlikely to change, so ideas like expanded senses and ambient awareness are in section three of this book.

2.4.1 Hearing

The sense of hearing is the result of the complex processing of vibrations, primarily, in the cochlea of the human ear. This typically responds to a frequency range 20-20,000 hertz.[18] Younger people may hear higher pitches than older people.

Sound as we know it is a longitudinal wave. The mechanics of a wave like this are slightly different than light waves, but can be described in similar terms. An energy source, like a speaker or a voice produces an energy wave that then travels through a medium to reach a reception point. Energy can move around this space and be reflected back at the source.

Figure 2.10: Acoustic Envelope – WikiAudio, 2018 https://www.wikiaudio.org/adsr-envelope/

This is the acoustic envelope. Any sound has an attack, the initial moment when the sound is produced, a time where it is sustained, and the end where it decays and is released. This is an important idea for understanding both sound and sound processing. The envelope represents a sound as an energetic moment, within that moment the elements of the wave can be further manipulated. If an entire envelope is heard in a reflection with a complete attack, it is called an echo. When the wave interacts with the sustain of a prior envelope it is called reverb.

When different waves interact, they can form what is called a standing wave where the energy of the two is merged. It is also possible that a wave could cancel out the other if it is out of phase with the original.

There are two dimensions we should consider: the frequency of a sound and the energy level. The core note of the piano is Middle C with the A of a Viola at 440. Your instructor will likely use a frequency generator to produce some of these tones and harmonics. If they do not, you should be using a program like garage band or just playing around with the random pianos that seem to be around college dorms to think about these concepts.

Figure 2.11: Common Frequencies – Centers for Disease Control, 2015, https://www.cdc.gov/ncbddd/hearingloss/sound.html

The energy of sound is expressed in decibels. This is a logarithmic scale meaning that an increase of ten decibels is one hundred times the energy level. Thus, if an employer were to break the 84 dB threshold for hearing protection in an environment by a few dB it is a massive increase in energy level.

Reflections may be adjusted. Sounds can be absorbed. An anechoic chamber uses large absorptive wedges to eliminate sound. Sound can also be diffused by these wedges. Contact with a surface with a great number of cleavages allows the wave to impact over time. This is critical. Once the wave is broken into many smaller reflections in different time frames the total sound is dramatically reduced.

Reflection can be used strategically in designing spaces that would be advantageous for superior reflection, like a lecture hall. Among the reasons why people enjoy singing in the shower is the propensity for the small space with parallel walls to form standing waves, further the humid air of shower has greater impulse than dry air in the world. This is an important point: sound waves propagate differently depending on the medium.

Objects producing sounds are also limited by their resonance frequencies. If one plucks a string, the wave length is constrained by the total length of the string. Of course, harmonics may operate on different intervals, but the fundamental frequency of the sound will remain the same.

Figure 2.12: Air Force Barber Quartet, Seymour Johnson Air Force Base, 2015. https://www.airforcemedicine.af.mil/News/Photos/igphoto/2001345190/ Vocalists from Tops in Blue perform songs from the last century to show how music has changed and grown, Oct. 10, 2015, at Seymour Johnson Air Force Base, North Carolina. Paying tribute to the music of the 1930s, Tops in Blue performed several songs as a barbershop quartet. (U.S. Air Force photo/Senior Airman Aaron J. Jenne)

A barbershop quartet is a fascinating study in harmonics. If you listen to a group with a few singers you can hear more tones than should be present. Why? Because harmonics form between the frequencies where the singers are.

Figure 2.13: Common Energy Levels – Centers for Disease Control, 2015, https://www.cdc.gov/ncbddd/hearingloss/sound.html

How is this translated into the brain? The human takes in sound through the cochlea, either received as vibration of the creature directly, or through a system of bones in the ear which is then processed in the inner ear by the cochlea. Shera, Guinan, Oxenham describe the role of the inner ear:

The mammalian cochlea acts as an acoustic prism, mechanically separating the frequency components of sound so that they stimulate different populations of sensory cells. As a consequence of this frequency separation, or filtering, each sensory cell within the cochlea responds preferentially to sound energy within a limited frequency range. In its role as a frequency analyzer, the cochlea has been likened to a bank of overlapping bandpass filters, often referred to as “cochlear filters.” The frequency tuning of these filters plays a critical role in our ability to distinguish and perceptually segregate different sounds. For instance, hearing loss is often accompanied by a degradation in cochlear tuning, or a broadening of the cochlear filters. Although quiet sounds can be restored to audibility with appropriate hearing-aid amplification, the loss of cochlear tuning leads to pronounced, and as yet largely uncorrectable, deficits in the ability of hearing-impaired listeners to extract meaningful sounds from background noise.[19]

The research from which this quote was extracted concerns the understanding of the critical band of the cochlea, where the ear may actually interpret signal. This would play a critical role in understanding what kinds of filters and sound modifications could be conducted. It would make little sense to increase the volume of what could not be heard in the first place.

Much of what we would understand to be semantically meaningful activity takes place in the relatively low end of this band. Voices can be understood by slicing out 250-1000 hertz. Once activated a relatively small number of neurons interact with the hair cells, indicating a tone at a particular point. Sound perception is limited to the particular bands where the hairs are capable of processing a vibration.

Sound is standardized through the frequency, envelope, and energy level. Special forms of sound organization that take place over time and through the organization of tones is called music. The schemes by which we organize tones are highly likely to change.

2.4.2 Vision

Vision, in humans, relies on the retina to interpret light focused by the lens. This information is then relayed through optic nerve to the brain, it is processed by a number of regions. Although the exact process by which recognition of particular forms is still unknown, the brain appears to first process edges and then to work among multiple sets of edges to form an image.[20]

On the level of the retina, there are two distinct receptors: rods and cones. Rods are distributed outside the central fovea and have little role in color, they are highly sensitive.[21] Cones are found in the foveal region.

Figure 2.14: Distribution of Rods and Cones, University of Texas Dallas, http://www.utdallas.edu/~tres/integ/sen3/display7_10.html

Cones are far more responsive, detecting color. There are three kinds of cones: S, M, and L which appear to be associated with different perceptions.

Rod sensitivity peaks between S and M. Each cone cell has a single line to the optic nerve, rods[22] It is important to note that there are far more L and M cones than S.[23] The receptors of the retina are thus all receiving some particular color, it is not that the cones see color while the rods do not. The resolution of the retina is roughly 150,000 cones per square millimeter.[24] By contrast, the most sensitive commercially available film cameras have a pixel density of roughly 42,000 per square millimeter. Most display systems operate at much lower resolutions.

Beyond resolution, the eye and the sensor have different refresh rates. Historically the normal flicker detection threshold for the eye was understood to be around 70 Hz (just above the refresh rates of an analog television), although research has indicated that detection of flicker for light with a spatial edge can be perceived at much higher frequencies [25]

The underlying structures of artistic composition make sense in the context of the eye relying on a combination of edge and color detection, from line through variety the basic features of art work with these perception mechanisms. At the same time, there are surely codes that interplay with the nature of perception itself. More on this in the next section.

Depth Perception

Most depth cues are mono-optical.[26]

Cue

Description

Occlusion

One thing is in front of another

Parallax

Moving your head allows you to see a new image

Size

Things in the distance look small

Linear Perspective

Lines appear to converge toward the horizon

Texture

At a distance textures are not visible

Atmosphere

Things in the distance are hazy

Shadows

Light sources produce indications of depth

Convergence

The eyes tend to converge on an image at a moderate distance

Stereopsis (Binocular Convergence)

Each eye sees a different image

As you may have noticed, parallax (mono-optical) depends on the motion of the head. This demonstrates a level of sensory integration, with touch, as the proprioception of the head is involved.[27] Modally, the senses are more integrated than disintegrated.

2.4.2.1 Color

Color is not changing in as much as it will not have a dispositive resolution. What does this mean? Color is ultimately a combination of an object, light, and a perceiver. Where color resides between these sites is an old philosophical debate. What is especially important to understand about color is that it is not separate from vision itself. Commonly in communication and art programs color is taught as something secondary and less than line or form. As you have already read the idea of color and non-color receptors is dubious – rods perceive a sort of blue. This lack of color is known as the coloring book hypothesis: the brain produces a world of outlines which are then colored in. As Chiriuta describes, the research on vision does not bear out this theory: colors are processed simultaneously in edge detection.[28]

At the same time, color dysfunction is extremely common among males. It is best practice to NOT use color as the primary means for encoding information, as many people are unable to detect certain hue differences. Changes on an evolutionary scale are clearly among those in the less likely category for this book.

Attempts at standardizing color hinge on reproduction. Although Pantone is generally known for fun research on color trends, the real products are specialty inks that can be used across product classes. Pantone produces educational materials that can help understand what a particular color is, at least in as much as it can be recreated:

Figure 2-15: Screenshot of Pantone Website

2.4.3 Touch

To begin with there are a few major categories of touch perception: mechanical (pressures and vibrations), temperature, pain, and proprioception (dimensions of the body at present).

In terms of processing, much of the work of touch sensation is accomplished by ganglia, with the majority of touch neural structure devoted to the perceptions of pain and heat.[29] This paragraph is directly informed by Abraira and Ginty’s review for the Journal Neuron. Of the neurons that respond to touch sensations, there are low and high threshold variants. These perceptions are thought to be mapped to the different conduction potentials of the neurons, such as their myelination (being covered with a protective insulator). Hair also plays an important role – hairs are physical mechanisms that produce sensation and depending on hair type. Receptors also display different adaptation rates, meaning that they might continue firing if repeatedly stimulated, these are likely the key to textures. Fast adapting fibers have far more intense reactions. The uses of such fibers would be clear: sometimes you need a really strong touch to tell you to move your arm, but you don’t want that signal repeated too often. Yet another type of receptors, including Pacinian Corpuscles: these are important as they are receptors that can meaningfully transmit vibration frequency. The highest density of these receptors is found in the finger tips. As Abraria and Ginty note in their consideration of the integrative theory of touch:

Our skin, the largest sensory organ that we possess, is well adapted for size, shape, weight, movement, and texture discrimination, and with an estimated 17,000 mechanoreceptors, the human hand, for example, rivals the eye in terms of sensitivity.

They theorize that the Dorsal Horn of the spinal cord is akin to the retina in the perception of touch, serving as an intermediate rendering point for incoming touch information, although some touch information may pass through by another channel.

The processing of touch perception is complex. Berger and Gonzalez-Franco hinge their research on the idea of the “cutaneous rabbit,” an illusion in touch where taps at two points produce a feeling that the touch moved between those points.[30] They use this argument to lead to an important debate: is touch perceived as a connection between particular points on the skin and parts of the brain; or is touch something far more tied into cognitive processes beyond the mapping of the skin? In their experiments, virtual reality equipment (oculus rift) was used to produce an “out-of-body” touch illusion. What is important about their work, is that they were able to produce out-of-body touch illusions without corresponding visual stimuli although enhanced by that additional information. Sensation of touch is deeply tied to other cognitive features, it is not simply a push-button effect on the skin. This line of research can also be explored through the work on affect and touch: in the absence of conscious-reflection touching can produce pro-social outcomes.[31]

As much as experiments like the cutaneous rabbit prove, the problem of haptic translation is difficult. The limits of the interpretation of touch as haptics will be discussed at greater length in the next section, but the question becomes, can we actually deal with the sure number of sensations that would be needed to replicate the world as we know it? Does a vibrating glove match the grain of velvet?

A further consideration is the role of the mapping of the body as it relates to the position of limbs, organs, and the space around the self. Terekov and O’Regan have proposed a model of the perception of space where an agent secures an awareness of space as an unchanging medium, rigid displacements, and relative position.[32] This offers an important insight about the production of spatial awareness both for creatures and AI systems: the foundation of the world can take place without a concept of space itself. In the context of human perception this becomes something of a sixth sense of body position, movement, and force.[33] Tuthill and Azim argue the perception of the body in space then is critical for stability, protection, and locomotion.[34] Most animal motor functions depend on the feedback loop of the perceptual system, with a few notable deceptions. Generally, the feedback information provided by various species proprioception systems are similar, suggesting a common origin.

Heat and pain are less relevant. There are clear media applications for the use of hot and cold, the cases where pain responses would make sense are limited. An example would be the Star Wars experience at Disney World which makes use of haptic feedback vests to help guests perceive the impact of laser hits.[35] Of course this is not true pain, just a gentle tap.

2.4.4 Taste

Taste is a combination of sensations. Some of what we understand as taste is smell, blended with texture, sight, and sound. In terms of the specific sensory differentiation point, the key to taste is the taste bud, which contains specific chemical receptors.

Roper and Chaudhari report in their review of the literature for Nature Reviews: Neuroscience that each taste bud contains three distinct types of receptor cells: Type 1 unknown function with a highly heterogeneous makeup (50%), Type 2 larger and spherical detecting sugars, amino acids, and bitter compounds (33%), Type 3 with the mechanism for the detection of sour distributed in patches around the mouth (2-20%).[36] The taste buds are distributed, along with touch and temperature receptors, in various structures around the mouth called papillae.[37] The review contains more specific information about the synaptic linkages for each type of cell, this is beyond the scope of our analysis for this book.

First, the simplistic four flavor model is not supported by the research. The receptors of the taste buds can recognize many different compounds. Consider the amino acid lysine: it is nearly 70% as sweet as sugar, and one of the primary flavor elements of pork. Second, the idea of regions of the tongue being associated with particular flavors does not hold. Taste receptors are found in many places in the body, which makes sense as they are sophisticated chemical detectors and there are many chemical detection tasks that would seem to be key to human life.

The total number of potential flavors ranges between five: salty, sweet, sour, bitter, umami (glutamates) and twelve. Any number of chemicals can be detected by the taste buds described in the research. Some receptors detect carbon dioxide.[38] Spicy flavors are detected by the VR1 heat receptor (which helps keep you from burning yourself).[39] Thus the common-sense retort that there is no taste of spicy, it is just hot. But hold on – not only are the receptors for hot in your mouth (and not on your forearms or lower back), a specific receptor for heat, and another receptor for the perception of coolness that runs along same set of nerves.[40] Why would flavor be limited to one of the sets of chemical receptors tied to nerves that are enfolded into perception? There also seem to be receptors for calcium, zinc, maltodextrin, histidine (linked to heartiness or kokumi), glycerol sensation may hinge on chain length.[41] It is still unclear how the sense of salty works, yet we know from practical experience that salt can decrease the perception of bitter, and almost everyone enjoys a salty snack.

The hinge of this question is not biochemical, but phenomenological, how do we know when a combination of chemical interactions in the taste bud become an independent flavor? Andrew Smith writing for a New York Times blog noted that flavor scientist questioned how many different flavors should be recognized due to the lack of a firm basis for the idea of flavor.[42] More importantly, the body responds in many ways to taste alone, without actually swallowing the food. This is the challenge then for the role of flavor itself – how do we create a meaningful set of classifiers for such a robust and multidimensional experience? Is it enough to say sour when there are so many other sensations and descriptions? Is the ethnological (wine) solution adequate where experiences are described in a series of other ontologically separate terms? Or does the attempt to encircle the description of a flavor become an endless hermeneutic game?

In a review of the commonalties of mammalian taste, Yamolinsky, Zucker and Ryba note that insects and non-insects have distinct neurological structures for taste.[43] On the other hand, despite extensive differences, insects and mammals seem to have reactions that sort tastes into a somewhat similar framework – things that keep you alive (sugars and salts) and things that poison you (bitters). Innate flavors are present, at least at the start, this is not to say that people may not enjoy other flavors that they first found overwhelming.

Standardization is quite difficult. I learned this working as a short-order cook in college, when many of my classmates who were from Nepal found the food of the North Dakota region to be entirely too bland. They described the experience of eating foods at the extreme limit of spicy as containing a different world of flavors that were not present in other cuisines, which is an idea supported in the spicy adaptation literature.[44] The pH of your saliva can change how sweet something tastes.[45] Absent the ability to truly standardize flavor, or even provide flavors that would not seem injurious to some audience members, it seems unlikely that any meaningful standard for human flavor perception could be created. If such a technology were possible, the implications for human health and industry would be profound.

2.4.5 Smell

Olfactory responses pose a number of fascinating challenges for future media technology. Scents are difficult to reproduce, requiring particular chemistry and delivery. Like other senses described in this book smells are culturally specific. Individual reactions in the olfactory system with chemicals come together to form a coherence experience of an odor. There are cultures devoted to a return to a more “natural” way of living that object to the low odor presentation of many Americans. There are foods that are so offensive that they require active policing, like durian. Functional MRI research confirms that exposure to personally relevant aromas is tied to increased activity in the amygdala.[46] Research on smell remains woefully behind research on vision and hearing.[47] The lack of research on smell even allows organizations and companies to take a lead role in education.

Critical to the operation of the olfactory bulb is the flow of molecules over roughly four-hundred receptors.[48] Dr. Thomas Cleleand describes the process of learning smells thusly:

What we think that the first couple of layers of the olfactory system do is to build odors and define their sort of fuzzy boundaries,” Cleland continues. “You get this messy input, and the perceptual system in your brain tries to match it with what you know already, and based on what you expect the smell to be. The system will suggest that the smell is X and will deliver inhibition back, making it more like X to see if it works. Then we think there are a few loops where it cleans up the signal to say, ‘Yes, we’re confident it’s X.’

The future of smell research involves building massive processing models that might accurately model raw number of connections that could be made between different molecules and the bulb and the processes of memory that then encode the experience if encountering a new smell. It is also clear that this varies from taste or touch as it exists in even greater combinations an in a recirculating cybernetic movement toward sensation. Unlike the media of the instant, smell develops with an extended temporality.

There are many places that require careful olfactory planning. Alfred Taubman, an important developer of shopping centers, was careful to avoid errant wafts of scents from food courts into unwelcome places. Stores looking to build a unique brand, particularly those targeting young people, have been known to heavily perfume their entries. Disney World uses pipes to distribute the aroma of cookies in relevant places.

Expert practices in olfactory development include various approaches to tasting particular products. Meister, the coffee columnist for Serious Eats, reports that coffee tasters smell for enzymatic issues, sugar caramelization (mallard reaction), and dry distillation.[49] Each of these flavors has many subexpressions that circle in on a combination of chemicals and relationships that make the smell meaningful.

The standardization of olfactory experience is perhaps the most difficult. A more formal descriptive language comes from the IFF database, which provides an index of olfactory information much akin to Pantones.[50]

Figure 2.16: Screenshot of IFF compendium

Notice the information provided. An olfactory description, which makes reference to other olfactory experiences, and use cases. What might be most important is the concept of chemical stability. Unlike a sound or an image, there are concerns that a distributed scent could ignite, levels of use, density, and beyond.

Benson Munyan, a professor of somatics at the University of Central Florida, has demonstrated that the use of olfactory cues can increase immersion in an experience, the logistics of using these devices are challenging as errant scents linger.[51] Commercial approaches to smell offer far fewer scents. The initial challenges may appear to come in the delivery of compounds (tubes, stickers, wafters) but on a deeper level there is no unifying substance for the production of different aromas. In the context of vision, there is a limited spectrum of light that the eye can process. Sounds are vibrations in a particular band processed by verily specific elements in the cochlea. In both of these cases there is a unifying form of energy and spectrum. Olfactory response involves a much wider variety of media and receptors.

In a world where a basic chemical synthesis method for olfactory experiences remains elusive, and it likely will be, experience design in these media will remain tied to particular places and to particular chemical compounds that might be strategically released. Although the IFF database provides the beginning of a theory of olfactory information, it is important to recall the reasons for the existence of Pantones in the first place. All color mixing methods have weaknesses in their underlying gamut. All reductive methods for the production of stimuli are problematic by their nature. Until micro-synthetic chemistry systems are available olfactory virtual reality will not exist. The underlie revolution in a chemistry plant on a card will fundamentally change the world. It is not a matter of all the chemicals that we might want to smell for fun, but a system capable of producing such a selection of smells could produce any number of industrial chemicals and drugs on a micro-on-demand basis.

2.5 Desire

People want.

This is the shortest and perhaps most important entry in this section. Bertrand Russell argued in his Nobel Prize Lecture that the problem of the human condition is desire. In the first instance the desire for the things that make survival possible. In the second instance, when survival was assured, desires for power and prestige. Power, for Russell, was almost always a violent drive, as that which would allow one to make others do what they would not, is troubling. Friedrich Nietzsche’s Genealogy of Morals, documents the shift in axiology where even basic concepts of right and wrong are continually recirculated within the event horizon of desire.[52] Placing the symbolic codes by which we evaluate right and wrong as secondary to desire is a provocative and important move to this day

People initially present something that they want, this want could be satisfied, but often the chase for the thing becomes more satisfying than the thing itself.[53] Desire is unquenchable. Hannah Arendt supposed that the desire for meaning could be, in the highest form, expressed through creative action. Natality does not escape desire, but for Arendt it is a desire to make new that is the highest point of the human condition, actions that are often confused with work or labor.[54] For Russell, managing excess desire is the key, to resolve his diagnosis that war is a function of a desire for excitement he proposed:

I think every big town should contain artificial waterfalls that people could descend in very fragile canoes, and they should contain bathing pools full of mechanical sharks. Any person found advocating a preventive war should be condemned to two hours a day with these ingenious monsters. More seriously, pains should be taken to provide constructive outlets for the love of excitement. Nothing in the world is more exciting than a moment of sudden discovery or invention, and many more people are capable of experiencing such moments than is sometimes thought.[55]

Desire will not change.

1. Electromagnetic Spectrum - Introduction,” accessed October 5, 2018, https://imagine.gsfc.nasa.gov/science/toolbox/emspectrum1.html. ↵

2. Eamono Doherty, “The Need for a Faraday Bag,” Forensic Magazine, February 21, 2014, https://www.forensicmag.com/article/2014/02/need-faraday-bag. ↵

3. Red Lion Broadcasting Co., Inc. v. FCC, 395 US 367 (1969). ↵

4. “Miami Herald Publishing Company v. Tornillo,” Oyez, accessed October 5, 2018, https://www.oyez.org/cases/1973/73-797. ↵

5. “Advantages of Fresnel Lenses | Edmund Optics,” accessed October 5, 2018, https://www.edmundoptics.com/resources/application-notes/optics/advantages-of-fresnel-lenses/. ↵

6. “Color Systems - RGB & CMYK,” accessed October 5, 2018, https://www.colormatters.com/color-and-design/color-systems-rgb-and-cmyk. ↵

7. Immanuel Maurice Wallerstein, World-Systems Analysis: An Introduction(Duke University Press, 2004). ↵

8. Larry Downes and Chunka Mui, Unleashing the Killer App: Digital Strategies for Market Dominance(Harvard Business Press, 2000). ↵

9. Sarah Zielinski, “Rare Earth Elements Not Rare, Just Playing Hard to Get,” Smithsonian, accessed October 5, 2018, https://www.smithsonianmag.com/science-nature/rare-earth-elements-not-rare-just-playing-hard-to-get-38812856/. ↵

10. Gearóid Ó Tuathail and Gerard Toal, Critical Geopolitics: The Politics of Writing Global Space(U of Minnesota Press, 1996). ↵

11. Evan Dashevsky, “A Remembrance and Defense of Ted Stevens’ ‘Series of Tubes,’” PCMAG, June 5, 2014, https://www.pcmag.com/article2/0,2817,2458760,00.asp. ↵

12. Broadb et al., “FCC Concludes Satellite Internet Is Good Enough for Rural Broadband,” Broadband Now(blog), February 16, 2018, https://broadbandnow.com/report/satellite-internet-good-enough-rural-broadband/. ↵

13. Daniel Terdiman and Daniel Terdiman, “This Blimp Startup Is Taking on Google’s and Facebook’s Flying Internet Projects,” Fast Company, August 14, 2017, https://www.fastcompany.com/40453521/this-blimp-startup-is-taking-on-googles-and-facebooks-flying-internet-projects. ↵

14. “Doped Semiconductors,” accessed October 5, 2018, http://hyperphysics.phy-astr.gsu.edu/hbase/Solids/dope.html. ↵

15. “Introduction to Spintronics,” accessed October 5, 2018, https://www.physics.umd.edu/rgroups/spin/intro.html. ↵

16. Larry Greenemeier, “How Close Are We--Really--to Building a Quantum Computer?,” Scientific American, accessed October 5, 2018, https://www.scientificamerican.com/article/how-close-are-we-really-to-building-a-quantum-computer/. ↵

17. Lisa Gitelman and Geoffrey Pingree, “What’s New About New Media?,” in New Media 1740-1915 (Cambridge: MIT Press, 2004), xi–xxiii. ↵

18. “Extended High Frequency Online Hearing Test | 8-22 KHz,” accessed October 5, 2018, https://www.audiocheck.net/audiotests_frequencycheckhigh.php. ↵

19. Christopher A. Shera, John J. Guinan, and Andrew J. Oxenham, “Revised Estimates of Human Cochlear Tuning from Otoacoustic and Behavioral Measurements,” Proceedings of the National Academy of Sciences of the United States of America 99, no. 5 (March 5, 2002): 3318–23, https://doi.org/10.1073/pnas.032675099. ↵

20. “How the Brain Recognizes What the Eye Sees - Salk Institute for Biological Studies,” accessed October 5, 2018, https://www.salk.edu/news-release/brain-recognizes-eye-sees/. ↵

21. “Rods & Cones,” accessed October 5, 2018, http://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_9/ch9p1.html. ↵

22. Dale Purves et al., “Cones and Color Vision,” Neuroscience. 2nd Edition, 2001, https://www.ncbi.nlm.nih.gov/books/NBK11059/. ↵

23. “Rods & Cones.” ↵

24. Ibid. ↵

25. James Davis, Yi-Hsuan Hsieh, and Hung-Chi Lee, “Humans Perceive Flicker Artifacts at 500 Hz,” Scientific Reports 5 (February 3, 2015): 7861, https://doi.org/10.1038/srep07861. ↵

26. “Depth Cues in the Human Visual System,” accessed October 5, 2018, https://www.hitl.washington.edu/projects/knowledge_base/virtual-worlds/EVE/III.A.1.c.DepthCues.html. ↵

27. Vito Pettorossi and Marco Schieppati, “Neck Proprioception Shapes Body Orientation and Perception of Motion,” Frontiers in Human Neuroscience, November 2014, https://www.frontiersin.org/articles/10.3389/fnhum.2014.00895/full. ↵

28. M Chiriuta, Outside Color: Perceptual Science and the Puzzle of Color in Philosophy (Cambridge: The MIT Press, 2015). ↵

29. Victoria E. Abraira and David D. Ginty, “The Sensory Neurons of Touch,” Neuron 79, no. 4 (August 21, 2013): 618–39, https://doi.org/10.1016/j.neuron.2013.07.051. ↵

30. Christopher C. Berger and Mar Gonzalez-Franco, “Expanding the Sense of Touch Outside the Body,” in Proceedings of the 15th ACM Symposium on Applied Perception - SAP ’18 (the 15th ACM Symposium, Vancouver, British Columbia, Canada: ACM Press, 2018), 1–9, https://doi.org/10.1145/3225153.3225172. ↵

31. Annett Schirmer et al., “Squeeze Me, but Don’t Tease Me: Human and Mechanical Touch Enhance Visual Attention and Emotion Discrimination,” Social Neuroscience 6 (June 1, 2011): 219–30, https://doi.org/10.1080/17470919.2010.507958. ↵

32. Alexander V. Terekhov and J. Kevin O’Regan, “Space as an Invention of Active Agents,” Frontiers in Robotics and AI 3 (2016), https://doi.org/10.3389/frobt.2016.00004. ↵

33. Joshua Klein et al., “Perception of Arm Position in Three-Dimensional Space,” Frontiers in Human Neuroscience 12 (August 21, 2018), https://doi.org/10.3389/fnhum.2018.00331. ↵

34. John C. Tuthill and Eiman Azim, “Proprioception,” Current Biology 28, no. 5 (March 5, 2018): R194–203, https://doi.org/10.1016/j.cub.2018.01.064. ↵

35. Anthony Levine, “Star Wars Virtual Reality at The Void in Disney World and Disneyland,” USA TODAY, May 22, 2018, https://www.usatoday.com/story/travel/experience/america/theme-parks/2018/05/22/star-wars-secrets-empire-void-virtual-reality/629435002/. ↵

36. Stephen D. Roper and Nirupa Chaudhari, “Taste Buds: Cells, Signals and Synapses,” Nature Reviews. Neuroscience 18, no. 8 (August 2017): 485–97, https://doi.org/10.1038/nrn.2017.68. ↵

37. “How Does Our Sense of Taste Work? - National Library of Medicine - PubMed Health,” accessed October 5, 2018, https://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0072592/. ↵

38. “Scientists Discover Protein Receptor for Carbonation Taste,” National Institutes of Health (NIH), September 27, 2015, https://www.nih.gov/news-events/news-releases/scientists-discover-protein-receptor-carbonation-taste. ↵

39. James Gorman, “A Perk of Our Evolution: Pleasure in Pain of Chilies,” The New York Times, September 20, 2010, sec. Science, https://www.nytimes.com/2010/09/21/science/21peppers.html. ↵

40. “TRPM8: The Cold and Menthol Receptor - TRP Ion Channel Function in Sensory Transduction and Cellular Signaling Cascades - NCBI Bookshelf,” accessed October 5, 2018, https://www.ncbi.nlm.nih.gov/books/NBK5238/. ↵

41. Kokumi and fatty acid citations follow, the general point is that receptors are present for many chemicals. Lisa Bramen, “The Kokumi Sensation,” Smithsonian, accessed October 5, 2018, https://www.smithsonianmag.com/arts-culture/the-kokumi-sensation-78634272/; R. D. Mattes, “Oral Detection of Short-, Medium-, and Long-Chain Free Fatty Acids in Humans,” Chemical Senses 34, no. 2 (September 15, 2008): 145–50, https://doi.org/10.1093/chemse/bjn072. ↵

42. Peter Andrey Smith, “Beyond Salty and Sweet: A Budding Club of Tastes,” Well (blog), July 21, 2014, https://well.blogs.nytimes.com/2014/07/21/a-budding-club-of-tastes/. ↵

43. “Common Sense about Taste: From Mammals to Insects - ScienceDirect,” accessed October 6, 2018, https://www.sciencedirect.com/science/article/pii/S0092867409012495. ↵

44. Agneeta Thacker, “FYI: Are People Born With A Tolerance For Spicy Food?,” Popular Science, June 10, 2013, https://www.popsci.com/science/article/2013-06/fyi-are-people-born-tolerance-spicy-food. ↵

45. Ken-ichi Aoyama et al., “Saliva PH Affects the Sweetness Sense,” Nutrition 35 (March 1, 2017): 51–55, https://doi.org/10.1016/j.nut.2016.10.018. ↵

46. Rachel S. Herz et al., “Neuroimaging Evidence for the Emotional Potency of Odor-Evoked Memory,” Neuropsychologia 42, no. 3 (2004): 371–78. ↵

47. Gordon M. Shepherd, “New Perspectives on Olfactory Processing and Human Smell,” in The Neurobiology of Olfaction, ed. Anna Menini, Frontiers in Neuroscience (Boca Raton (FL): CRC Press/Taylor & Francis, 2010), http://www.ncbi.nlm.nih.gov/books/NBK55977/. ↵

48. “Learning, Memory, and the Sense of Smell,” Text, Cornell Research, May 25, 2016, https://research.cornell.edu/news-features/learning-memory-and-sense-smell. ↵

49. Serious Eats, “Advanced Coffee Tasting: What Your Coffee Smells Like,” accessed October 6, 2018, https://drinks.seriouseats.com/2012/04/coffee-cupping-aroma-what-coffee-smells-like.html. ↵

50. International Flavors and Fragrances, “Bergamal,” accessed October 6, 2018, https://www.iff.com/en/smell/online-compendium. ↵

51. “Why Smells Are So Difficult To Simulate For Virtual Reality,” UploadVR, March 9, 2017, https://uploadvr.com/why-smell-is-so-difficult-to-simulate-in-vr/. ↵

52. Friedrich Nietzsche, On the Genealogy of Morals: A Polemic. By Way of Clarification and Supplement to My Last Book Beyond Good and Evil (Oxford University Press, 2008). ↵

53. ? ↵

54. Arendt, Hannah, The Human Condition (Chicago: University of Chicago Press, 1998). ↵

55. Bertrand Russell, “What Desires Are Politically Important?,” (Nobel Lecture, December 11, 1950). ↵

Section 3: Things that are likely to change

This section covers many ideas similar to those in section two with an important difference, these concepts are likely to change. At the core of each idea are things that won’t change.

3.1 Semiotics

Semiotics refers to the study of symbol systems. This is independent of information theory, which considers the capacity of a channel and the capacity of a message to overcome entropy or noise in the channel.[1] Semiotics is concerned with meaning. We can think of meaning on three levels: semantic (did the communicators reach agreement about meaning), pragmatic (did the meaning coordinate action), and alterity (what meanings were precluded by the presentation of the sign). An important problem comes in the degree to which meaning can be communicated at all. Communication is constitutive.[2] The following statement is rich with mysticism: you are the meaning that you use and are used by. This does not mean that you are dominated by language or that you have no decisions, but that meaning is a central dimension of human existence.

Communication is not sending meaning packages through tubes to each other. Meaning is constantly produced between people. This does not mean that there is no agreement, but that the slippages between potential agreements produce many productive errors. Network research suggests that over-convergence is a major problem, if messages are too similar people distrust them.[3] Further, ongoing message divergence is evidence of error correction, which could itself evidence of an effective system. People make meanings, those meanings are unstable, and this instability is productive.

3.1.1 Signs

The sign is the basic unit of semiotic theory. It is important to understand that signs are not stable. For Charles Sanders Pierce, the sign is triadic:

Figure 3.1: Triadic Model

The object, the intepertant (the sign that is created in the mind of the receiver), and the sign are all in relationship.[4] Although the object is very much involved, but there is no objective basis for the sign, the sign itself and the sign produced by the sign are equally as important. Notice that the idea in the mind that matters in this model is that which exists in the mind of the receiver, not the sender. This model takes the intent of the speaker out of the center of the model. It does not matter what you intended if that cannot be produced as an interpretant.

Sassure’s model has a slightly less complex circulation.[5]

Figure 3.2: Signifier Model

In this context, the thing (signified) is imperfectly represented and in circulation with the signifier (the label). The signified can be almost anything, often including another sign. The signifier is constantly being loaded with additional content.

What should be clear in both models is that meaning is continuously in circulation. The ways that we manage this constantly shifting meaning are many and likely the reasons why you will be employed after college. A key distinction made by Saussure that is helpful: langue and parole. Langue refers to the formal code, while parole refers to everyday speech. This is why we teach multiple methods for determining what meanings are at any given time. There is no encyclopedia or dictionary for symbol systems, corporations are constantly searching for the ways that terms and ideas function at any given time.

3.1.2 Typology of Signs

Pierce has three kinds of signs:[6]

Icon: signs that look like things.

Index: representations of action (such as smoke is a sign for fire).

Symbol: an entirely artificial system (such as the text of this book).

Notice that these categories are not absolutely clear. Smoke may be a probabilistic sign of fire, but it can also be a symbol for something being hot. It is less likely that you thought that smoke was a sign of a smoke monster, a creature made of smoke. It is the probability that the sign is what you were thinking that is what makes it function. You see smoke and reasonably guess fire. This is the core of abductive reasoning, we make a number of probable assumptions and work as if they are confirmed. It should be apparent at this point that there are not easy logical answers or transcendental operators.

These signs can help you make decisions about how particular messages work. An iconic sign that includes an image may function quite differently than a description of that sign. You can see that in each of these contexts all three dimensions of a sign are always subtly shifting. Consider the iconic representation of a telephone, to some degree that is still most likely presented with an older handset on a desk cradle. Younger people may be less familiar with such a phone, however the icon for phone will likely remain the handset and cradle for some time.

Symbolic signs can be incredibly dense. This book is made up of almost entirely symbolic signs in the form of written text. Unless you are already quite a reader, the symbols in this book would be difficult to guess. Feelings and sensations can be exceptionally difficult to represent with a sign, often hinging on multiple signs that encircle what would be described.

3.1.2 Codes

It is important to keep in mind that Saussure contended that meaning existed in opposition. We know what a sign means through it’s relation to other signs.[7] Codes are organized systems of signs. Some codes are more sophisticated than others. People use codes all the time, they are not particularly special. For Roland Barthes the role of codes, as myths, allows the interposition of codes and facts:

In fact, what allows the reader to consume myth innocently is that he does not see it as a semiological system but as an inductive one. Where there is only an equivalence, he sees a kind of causal process: the signifier and the signified have, in his eyes, a natural relationship. This confusion can be expressed otherwise: any semiological system is a system of values; now the myth-consumer takes the signification for a system of facts: myth is read as a factual system, whereas it is but a semiological system.[8]

Symbolic forms have the status of facts, like marriage proposals, names for ships or highways, and statements of financial data, among many other possible codes. This is why semiotic critique is so useful, at each stage of recirculation the products of code become the facts that produce reality. It is not required that one fully establish all of the grounds on which they might argue, this would be boring and wasteful. You operate using the assumptions inherent in a code as if those assumptions were facts.

It would also make sense that the highly iterated symbolic signs would begin to play an increasing role in society. Once someone has gone to all the necessary trouble to learn a sign and all the content that comes with it, the deployment of that sign again in more sophisticated systems would be efficient. In a highly complex system of signs, the ways in which the meanings of certain signs might shift becomes an important topic of negotiation. This takes the form of a kind of meta-talk: you can identify it in many different kinds of communication. In relational communication, it can take the form of the “where do we stand” conversation with a dyadic partner; in political communication the discussion of the “narrative”

3.1.3 Ideograph

Special case of signs that have lost all content but still interpolate the social field. Examples of these special terms include freedom and the people. We all know that you are supposed to love freedom, but the meaning of freedom and any discussion of the internal dynamics of what might make someone more or less free are not up for political contestation. Likewise, the idea of the people is very important in many political cultures. The general will of regular folks is seen as excluded from what would be an elite conversation, which is then claimed and included by elites in the form of the will of “the people.” Measuring the will of the people is imperfect, there is no method to directly combine an understanding of the will of thousands of people. The term ideograph was developed by Michael Calvin McGee, it proposes that certain terms take on this special status where they can stitch together the social totality.[9] The key to studying these systems is to evaluate the use of the term presently (synchronically) and to juxtapose that with the development of the term over time (diachronically). This method can provide you some insight into how the term functions and the role it plays in the code of society. At the highest levels, certain signs can have powerful effects on the structure and meaning of a code, even moderating the function of multiple layers of other signifiers. There are many other signs like an ideograph that are very important – these are the core topics of your course work in Rhetorical Studies (should your program include Rhetoric).

3.1.4 Publics

Public sphere is a mistranslation of publicity which is a verb, not a noun: publicity is a process by which ideas become visible and circulate. Much of the thinking in this field is organized around Jurgen Habermas Structural Transformation of the Public Sphere, a key work which proposed a theory that a rational, critical public sphere could emerge through the circulation of texts. This book is a rich starting point as Habermas is willing to sketch the historical, psychological, and sociological dimensions by which the pubic becomes possible. It is also important to note that even at the time of publication, he was not optimistic about the future of the public sphere.

Nancy Frasier’s classic “Rethinking the Public Sphere,” gets at the key problems with this account, with a particular emphasis on exclusion.[10] Women and minorities were excluded from the bourgeois realm of the public sphere. This did not mean that they did not communicate, but that they formed counter publics all along. The exclusions and rules that come along with the public sphere can become an oppressive ideology on their own. Frasier is clear that these counter publics are a structural element of the theory, they are not intrinsically positive: they simply are. Frasier makes four key points: inequality cannot be wished away, there is not a single public, excluding “private” issues misses much, and better models of the multiple interactions of publics offer a lot for understanding social change.

Along similar lines, Michael Warner has further explored counter publics as they form in discourse and through attention alone.[11] You can become a part of a public without giving a speech or wearing a campaign t-shirt: you join by paying attention. Ron Greene on the other hand has recuperated the centralizing functions of the public sphere as a sort of postal service: the public as an idea allows letters to find their addressees.[12] What if the public sphere is the process by which ideas are moved around, even if that process deeply flawed? The key idea here is that a public forms when those that could be addressed are exposed to a possible message.

This says nothing of the far older debate between theories of publics and masses. Publics are those who are engaged in some kind of deliberation while masses are a herd to be steered. It is also important to notice that publics in this formulation are asked to have an idea of themselves. Publics are a powerful form for the organization of everyday life, but the ways that publics are hailed into existence and organized are always changing. To this point these constructs are in continuous revision and discussion.

3.1.5 Argumentation

How we argue and how we evaluate arguments is constantly changing. Arguments are another special form of code. The idea of the syllogism is thousands of years old, but ultimately the form of the syllogism is only useful in cases where a clear formula in the context of a dialectical regime of truth is possible. What does that mean? The syllogism depends on the idea that there is a truth and that by ascertaining proper premises an accurate statement can be confirmed.

Consider this syllogism:

Dan is a person
People have opinions
Dan has opinions

Or to put in an abstract form:

Dan is a member of group A
All members of A have property B
Thus, Dan has property B

In terms of a structural logic, his operates through a right hand left hand movement that is also used in some computer programming languages. The problem with the syllogism is that it can only handle one operator at a time. What we find in the analysis of complex issues of policy or value is that there may be multiple contingent identities and relationships in any given argument system. You also may have noticed the implicit use of the idea of ‘all’ in the example –a further problem. How do we make arguments when identities are unclear or are in flux?

Argumentation theorists have developed alternative models that can appreciate the complexity and contingency of real speech and reason. Toulmin’s model supposes that an argument has the following parts: claim-warrant-backing-data-qualifier-rebuttal.[13] The key to this model are the warrants, the inferential leaps that connect claims and data, the data can be other claims. The power of this model is that it opens up a lot of space for nested argumentative functions, where one argument contains many others. Not necessarily the best for analysis of value claims, this model is useful for finding the sites where probabilistic claims about the future become stronger and weaker.

What is spoken:

Claim: Deflation is worse than Inflation
Data: Historical financial data for two-hundred years, indicates that depressions were more likely to be correlated with deflation, and that deflation linked events were longer and more severe.

What is unspoken:
Warrant: the use of correlation across examples provides a reasonable account of cause and effect
Backing: long term financial data are the appropriate information for this question, in the discussion of a causal claim a strong account of correlation can be developed as causation especially when definitive causation is impossible
Qualifier: the claim was structured around the idea of worse than
Rebuttal: the argument is designed to respond to the idea that inflation is the paramount fear for economists

This claim about deflation could then be situated as if it were a fact in a larger system of claims which we could call a case. The great strength of this approach for understanding practical discourse are the multiplicity of possible warrant and backing formulations. In the introduction to this book we established the idea of abductive reasoning, the prospect that a claim can be probabilistic and that we might decide between rival probable models of reality or facticity. For many interesting claims of policy, the collection of probabilistic claims is the most important dimension of the argument in the first place. It is not merely that deflation is worse than inflation, but that if the central objection to the development of good social policy is the risk of inflation, avoiding the risk of deflation would be a reasonable consideration. The next line of argument in the debate for this side should be clear at this point: we are closer to deflation than inflation.

Figure 3.3: Toulmin Model Diagram by Emily K.K. Faltesek, 2010

For a full discussion of warrant structures, please take an argument theory class in the communication department.

Just as this model is strong for evaluating systems of probabilistic policy claims, it is unable to account for the multiplicity of value claims. Or if one did use such a model to account for values and aesthetics they might trend toward bizarre conclusions.

Luc Boltanksi and Laurent Thevneot offer a theory better suited for value based on Justifications.[14] Claims of value happen in different worlds of value. In this model, we are not judging policies but values and aesthetics, given their capacity to work with in established regimes of value. This is important as it gives us a way to deal with the empirical dimension of values discussion which is more than the stark utilitarian calculus that accompanies policy. It is in the alignment of the value and the test that a legitimate transfer of value is established. Illegitimate tests are those that attempt to use the wrong values and tests for a situation. It is conceivable that one might argue that something belongs in a different world, thus becoming subject to a different economy of value.

This model of value hinges on the agreement of people in a community to the development of a relevant test. For example, in the world of celebrity, the appropriate test is popularity. The actual means by which the tests function, often aesthetics, are not easily contested and are not of particular interest. You cannot declare: “fallacy, you felt wrong.” Legitimate orders are those that can be justified.

The standards by which we evaluate argument and the ways that ideas are moderated are always changing. When argument moves beyond a single simple claim of fact, argument itself is contested. The power of the code of argument is that we freely move between the levels of dialectical and rhetorical judgement: the ideas of logical validity and desirability continuously play into each other.

In practical terms, a justification must be provided, this must be a claim that declares a world (a setting), with a polity (a form or organization appropriate to that world), who then have a value (economy of worth), which can be judged with an appropriate test.

Example: who is the more important pop star today, Taylor Swift or Cardi B?

This is the world of fame. The polity are those who are a part of the larger manifold of the recording industry. Importance is judged by recent chart performance, the test being Billboard Hot 100 performance. In the last calendar year, Cardi B has out-performed Taylor Swift. Swift’s last album was a commercial disappointment, one of the fastest dropping ever. Furthermore, Bodak Yellow (Money Moves) by Cardi B displaced Swift at the top of the chart. We could thus reasonably say that Cardi B is the more important pop star on the basis of the rules of this community, the economy of worth, and appropriate test.

Value arguments are typically about the alignment and transfer of value. It is important to understand that the ability to access a reliable test decreases asmptopically as you approach the horizion of the individual and that this is a rhetorical rather than dialectical form. You are likely persuading the about the efficacy of the test as much as you are deploying it. At the lowest level, the test may be purely aesthetic.

3.2 Design

Design is a translational practice where insights from a number of distinct fields are applied to the making of something. The evaluation of the things made exists along a number of dimensions including the function and appearance of that thing. Donald Norman’s classic The Design of Everyday Things, applies a number of ideas from social research to make objects less bad, or even possibly good.[15] Celebrations of good design in this context tend of focus on stories of highly successful consumer products. Critical design researchers emphasize that this translational approach to design leaves too many assumptions to those that would commission a design, those in power.[16]

Any design needs a framework. At times this comes in the name “design thinking” which is typically associated with a particular firm.[17] It is not that design thinking is bad, but that there is no one special magical process for making a good design or even evaluating that design. The framework design thinking becomes a buzzword for the consideration of the aesthetics and function of a particular thing. The future oriented design proposed in this book focuses on the ways that we might think of design as enabling a broader conversation and collaboration, not as a mystical replacement for the university or skill development.

3.2.1 Affordance and Signifier

To avoid confusion, we should start by describing the use of signifier in design practice. For designers, signifier refers to a label, like instructions in an elevator. This is a more specific use of the term than one might find in semiotic theory. As Donald Norman describes in his work on complexity and design, often signifiers appear when a design has failed. The non-signified in this sense would refer to a design that would function with a minimum of symbolic or synthetic signifiers.

James Gibson coined the term affordance to refer to the properties of an environment to a creature.[18] Properties can include many things, like stairs.  A door affords you entry, a car affords you transportation. Not all affordances are clearly visible. The idea of affordance is important because it offers a dialectical conception of the purpose of a design of a thing: it is about what becomes possible, not what is intentional or obvious. The provenance of this idea in the psychology of perception should not be forgotten. At stake in the affordance is the idea of the awareness of the user – this is a theory which enhances our awareness of the user by decentering their subjectivity. In other words, you can only truly understand how a person interacts with a system when you take the limits of that person seriously.

Given the preference for seemingly effortless communication, many designers would prefer that awareness of the affordance of a system be as liminal as possible. This is important because it tells you about the way of knowing that is present in design – the idea is that the designer can produce a world of almost effortless meaning. The code of the design invention melds into the thing as if it were an inevitable fact.

3.2.2 Simple, Complicated, and Complex

Following the definitions area derived from Donald Norman:[19]

Simple: in design simplicity refers to the occasion when a system fits with the psychological expectations of the user.

Complicated: the occasion when a system or thing does not fit with the psychological expectations of the user, or is emotionally fraught.

Complex: a thing has a lot of parts.

Simple and complicated have somewhat oppositional meanings. The optimal case for conventional design is when a complex thing is made simple. The biggest concern with simplicity is that designers, unchecked, have a penchant for inserting their own psychological position as that of the user. This is why a central tenant of usability theory (discussed later in this chapter) is “you are not the user” Design is a process of concealing and revealing, strategically, for a particular effect.

Figure 3.4: Picture of a brutalist building I took on campus at the University of Washington

Once a student learns about brutalism, they see it everywhere. Many university buildings of the 1960s and 1970s are wonderful examples. Unfortunately, Oregon State is not lucky enough to have great examples. The repeated use of form that ostensibly reveals the structure of the building, the raw use of concrete. Brutalism is in a real sense the tangible result of high modernism. This was an approach to building that could transcend the everyday condition.

Nikil Saval concluded in the New York Times style magazine that the return of brutalism may be the harbinger of the end of the brutalist ethic:

But the renewed interest in the movement has yet to produce any meaningful change in the culture of what gets built and how. This resurgence has not — not yet anyway — led to any revival of interest in public-minded development. Politics has been divorced from architecture. In fact, love for Brutalism has often led to gentrification. Many social housing projects, such as Erno Goldfinger’s Trellick Tower in London, have become much sought-after private housing. Architecture bookstores sell postcard packs of the greatest hits of Brutalism; you can buy a Trellick Tower mug to sip expensive coffee in your pricey Trellick Tower flat. The aesthetic of Brutalism may at last triumph over its ethic.[20]

Brutalism offers what should be a simple solution to complex problems, yet the legacy of this approach to construction is terribly complicated.

A dualism of simple and complicated resonates with the design of our social networks and experiences in media research. Not all complicated things are bad – some things need to be complicated, others should not be simple. Design is best when use structured prompts to drive a chain of questions that can provoke a rich discussion. Even this starting point will fall away.

3.3 Aesthetics

Aesthetics are much maligned. The surfaces of things are positioned as being less than the things. The ways that we position discourses of beauty and value have profound effects on how the world is understood. Anti-aesthetic discourse is an aesthetic just as much the discussion of a particular appearance. This discourse supposes that one would have a preference for depths and timeless truths, rather than the things of the moment. Beauty is both a cause and an effect.

Aesthetics are a particularly useful point for understanding how desire functions in everyday life. The things that people prefer, the ways that they are valued and how those valuations interact in code are important and reveal much about how people are and want to life in the world. This is a powerful point with regard to both design and ethical theory as themselves depending on some conception of beauty. Aesthetic theory becomes something of a base level for reconstructing studies of design and persuasion.

This book does not contain a full theory of aesthetics, hopefully you will receive this in one of our courses that is dedicated to such concerns. The key point that you need to understand is that appearances matter and aesthetic concerns are not trivial. They are a key aspect of the development of codes and the moderation of desire. Your instructor is likely preparing examples for class that hinge on contemporary aesthetics.

Charlie Tyson, writing in The Chronicle of Higher Education, notes that the aesthetic is making a comeback in academia.[21] The hinge for the aesthetic is quite similar to section two of this text: aesthetic experience when evaluated in an embodied sense is not mere decoration. Aesthetic judgement is real and important. Beauty is not the opposite of truth but something entirely different. It might be strange to think that an important function of college is to help you understand beauty, but it is an essential function.

3.3.1 Clothing

Walter Benjamin, one of the key figures in media studies, focused often on fashion. When people come up with new ideas and new approaches, a case of emergence, the conditions by which that novelty was produced collapse back into the same. This is why Benjamin was so concerned with the cyclical passage of vivacious creative energy into death. Clothing is a particularly useful site for classroom examples in this area, especially logos and novelty brands like Supreme.

Figure 3.5: Normcore sweatshirt. Pngimg, open access image for a “sweater,” http://pngimg.com/download/53252

In 2014, the New York Times fashion section reported that the hottest clothing trend was called “Normcore.”[22] It became apparent that the idea could just as easily be seen as something of a hoax: is it really a fashion trend to report that people are continuing to wear common items? It seems quite likely that people wearing “normal” clothing would be quite happy and comfortable. What is notable here is not so much the object of the sign, but the reflexive uptake of it. The idea of capturing a fresh new trend is a news story in itself. The idea of the trend, the code, and that fashion reporting without a concrete object rely on the idea that the code itself has become a social fact.

3.3.2 Cuisine

It should be apparent that all cuisine is a matter of culinary code. Food is a deeply contested site: strict rules govern the production of ideal food stuffs, ranked by complex mechanisms, scientific hygiene regimes aim to make the food safe, cultural codes with regards to what may be eaten are common.

We have already created the virtual in food. In the sense of advanced technique, we have the absolute peak of modernist cuisine, Myhrvold. In the everyday sense of the same thought Keji Lopez-Alt.[23] The approach to flavor here is to understand the chemical and process dimensions that make a particular food what it is. When embedded in an anthropological context this project is expressed by Alton Brown as Good Eats.[24] This is a model of the virtual and food that is personal, still very food like. Cookbooks are a fascinating code. Issac West argued that the cookbook is one of the most important sites where domestic everyday life interplays with ideology.[25] Some cookbooks are designed to help people learn in difficult situations, such as the Joy of Cooking, the books described this paragraph reinterpret cooking through the world view of techno-science.[26] Instead of a massive compendium of all that one might do with ingredients or a full tabulation of recipes, these scientific cookbooks select particular food products and then isolate the exact semiotic resonances of the sensory experience of the food. Recipes are then redesigned around the maximization of gustatory pleasure based on those codes. Idealized aesthetic visions of the food stuff reimagined around the chemistry and biology of the food itself.

Chemical labs in New Jersey and Minneapolis have already given us virtual cuisine.[27] Foods stabilized and flavored continually. Stabilized seasonality. Consistent flavors. No spoilage. Technologies like McDonalds represent the precision stabilization of that which would be tasty.[28] It becomes possible to think of culinary codes in terms of the production of sensations that were never possible before.

Food is the place where we can see what an enfolding really is – a blending of technology that has remade the earth, makes life possible, and is deeply enjoyable, yet wracked with danger. This short section on food is included as many semiotics texts rely on extended discussions of different foods, such as McDonald’s French Fries. Why the fries? McDonald’s technique for the management of potato moisture was a major advancement, the transcultural mythos of McDonald’s in terms of organizations and branding, and the seeming excellence of the product itself, as testified to by the legendary food writer James Beard.[29] Of course you can’t have these fries, the underlying technology that produced them relied on beef tallow and was retired in 1990.

Cuisine here is a chance to consider the simulacrum: what happens when flavor is more real than reality?

3.4 Recording Devices

For the most part, many media systems require some means of recording perceptible reality. It is possible that a system could produce a synthetic video or something else without a recording in the first instance. Yet, these systems still maintain a recording that can be reproduced as if a recording had existed in the first instance. In section begins with technologies for recording and then moves into the discussion of transport and reproduction. Paper has been common for many years, although the paper of the past may differ from what you are expecting – the ‘rags’ of nineteenth century journalism were literally fabric newspapers. The means of recording have a critical imprint on the text.

3.4.1 Image Recording

Recording devices allow the transmission of information through time, which is the inverse of the momentary transmission of information through space. Among the earliest methods for transmission we have are cave paintings and architectural forms like pyramids. Many of our recording technologies quickly degrade.

Film is an emulsion on a thin, transparent, flexible strip. When light is projected through that strip, an image is recreated. The chemistry of such a strip is an important consideration in the development of a recording system. Early film strips were extremely unstable and flammable. Chemicals used in a system like this would need to be extremely photosensitive and reactive. Some older films have literally decayed into vinegar. The design of these emulsions is important.

Lorna Roth described the problem of film color standardization: getting film developers to produce emulsions that could adequately render all people, with the wide variety of human skin tones was difficult.[30] Even the basic idea of color standardization depended on checking against Shirley Cards, which privilege certain ways of looking at film. Even today, computer filters are unable to fully map the faces of people who do not look a particular way. Image recording technologies, as they are designed to record particular people in a specific way, rely on assumptions about how people look or should look. This is not to say that those assumptions will not change.

Television broadcasts were recorded using film systems, as video tape did not exist. Even the underlying tape technology was not fully developed until after World War 2 when the cellulose tape technology of Europe was introduced in other places.

Images today are recorded as computer files for storage on flash memory. There are two chips, discussed at some length in section two: CCD (charge coupled devices) and CMOS (charged metal oxide semiconductor).

Likely changes in the future of the camera include longer battery lives and more sensitive chips, these would work better in lower light.  Things less likely to change are the lenses that organize the light that is focused on the sensor.

Non-lens technology has been developed at CalTech using optical phased array sensors. Cameras in this sense are two-dimensional planes.[31] These sensors are currently limited, but they could present new possibilities.

The biggest change in image recording in the last decade was not the reduction in price of CMOS, but the ubiquity of cellular phones. Each of these devices includes a camera that people can use at any time. Mirrorless cameras with larger sensors will also become ubiquitous.[32] These cameras may lack the sophistication of professional equipment, but for many purposes they are more than enough.

3.4.2 Sound Recording

The basic technology of sound recording is the microphone, which is a fundamentally similar technology to the speaker. Before the digital, the signal passed through a sound system was fundamentally similar, the recorded electrical signal would pass through the wire which then reproduced the sound with no reprocessing. If you played with a crystal radio set as youth, you know that analog radio is magic in as it broadcasts enough energy to receive the signal and a usable signal as well.

The traditional types of microphones include those that produce an electrical signal including:[33]
Condenser: a diaphragm produces a signal in a capacitor
Moving-Coil (dynamic): an external magnetic source producing a signal in a coil that is allowed to vibrate
Ribbon: a ribbon of material instead of a coil
Crystal: certain crystals transduce a signal in response to a vibration

Optical microphones measure vibration using a fiber optic lead or a laser. Changes in the reflection drive the detection of the signal. The key to reproducing a sound is the production of a document with the frequencies of the sounds present at the time of recording. It makes sense then that if someone could simulate those waveforms, thus allowing entirely synthetic sounds to be created with far greater control and precision than those that came before.

Figure 3.6: Soundeditor by Dan Faltesek

Sound is then either recorded as an analog signal or processed using a quantizing chip and stored as a digital audio file.

3.4.4 Recording Experience

This is a horizon for future technology. Our current modes for storing experience are for the most part literary. Walter Benjamin seemed to find that Proust offered something of an attempt to store the experience of a thing through rich description.[34] There is an important idea best expressed through psychoanalysis that can help us understand why experience is so hard to process – when more signifiers are added to a system or code, they retroactively shift the meaning of those that came before. A full representation of a place or space will likely require extensive mapping both of the physical environment but also of the person experiencing it.

3.4.3 Codecs

All information that is recorded is stored in some format. On a film strip, the image was retained as a discrete cell that would move as the strip crossed the light source. In terms of the images processed by digital sensors, the raw volume of data can be striking. Early digital field production cameras used heavy cards (like the P2) that would allow high resolution footage to be captured and stored on an array of flash chips. Those early cards could store gigabytes of data, which in raw meant about ten minutes of footage. At this level, streaming video would be bandwidth prohibitive. For transmission over the internet, the information must be encoded differently. The H.264 codec allowed much lower bit rates, the VO9 codec came later, and more recently the AV1 codec has allowed Netflix to further reduce the total volume of data flowing through the internet.[35]

The drawback to encoding the video differently is the loss of data. Courts may not accept transcoded video, it is too easily faked. When data is transcoded, a great deal of what is lost is the internal structure, color data that may not appear to the viewer but is lost. If one then tries to do substantial editing to the file after it has been transcoded from, for example, Apple Pro Res 422 to H.264, the corrections will not look as good as if they were made on the original file.

Codecs are the medium of the future. New media experiences exist between ultrahigh capacity sensors and display systems.

3.5 Editing systems

The editing of information is one of the essential properties of the production of new media. Editing before the advent of digital technology was often destructive, a film strip would be cut and taped back together to make a coherent strip. With the advent of the digital nonlinear editor the capacity for the production of new media was dramatically increased. There are a few types of digital editors that we need to consider:

3.5.1 Image Editing (Orthogonal Layer Interfaces)

If you are editing a single image you have likely used an orthogonal editing interface, this allows you to create a series of layers that may modify an image or mask parts of that image from modification. Some familiar editors in this space include Adobe Photoshop. The layers dimension is present in many different products, providing a high level of control and simplicity for image development. The most recent developments in this field are those that can produce new content in ways that sample the underlying image, such as a content aware fill. Although this interface metaphor is unlikely to change, the new element here is the likelihood that these elements will appear in other systems.

This model presumes the painters algorithm where the view of an image is composited along a z-axis from the top down. The distinction between raster and vector based editing systems is falling away.

Figure 3.7: Layers by Dan Faltesek

The key to this system is that we must have some way to render a static image element.

3.5.2 Digital Non-Linear Editors

The fourth dimension of video and audio is time. Music, after all, is the organization of tones in time. When we organize video or sound clips the combinations can produce rich results that vastly exceed the sum of the parts. The timeline provides a well-structured way of apprehending the project itself and controlling the dynamic state of the product as the viewer encounters it. Before 2002, this technology was unproven, it did not take long for DNLE to prove that Hollywood films could also use these techniques to produce powerful libraries of clips and rapid plastic changes.

As of 2017, only thirty-one major motion pictures were shot on traditional film.[36] Conventional workflows use a DNLE system to produce a list of edits to be made to the film proper. Typically, this is produced in the form of an Edit Decision List (EDL) which is presented as a form of metadata.[37] For sound manipulation, the technology relies on the same organizational theory. Music is produced by organizing tones in time.

Figure 3.8: Timeline by Dan Faltesek

Once situated in a stream of time, cinema becomes possible. The dimension of the perception of time is the key to the understanding of cinematic experience. The cinematic itself is unstable, the presumption to this point has been that the frame order of the time axis would drive the experience. Interactive systems challenge this in profound ways.

3.5.3 Integrated Development Environment

The interactive element of media depends on the design of systems which have the capacity to respond to user intervention into the state of that system. Keyboards allow users to provide sophisticated text strings to the system. Pointing devices all users to experience the graphical user interface. These systems then are being manipulated by users and produce a variety of different states depending on the input. These editing systems can range from basic text editing programs (where code can be written) through sophisticated development platforms like xCode.[38]

What these systems are providing is the capacity for users to use a variety of provided abstract representations to write economical code that could produce the intended results. The complex structures produced with these systems could include autopoetic systems which organize and produce content for the users as an ongoing flow.

Figure 3.9: RStudio by Dan Faltesek

Each of these approaches to editing will continue, it is likely that interfaces for each will continue to have an appropriate use.

3.6 Output devices

In this section, we will consider the devices that allow us to access the information encoded.

3.6.1 Optical

3.6.1.1 Resolution

Optical media are presented for the most part on screens. Early screens relied on tubes with scanning beams or projectors that would modulate light from a powerful source. Technologies that operate on this property include movie screens and tube televisions. The refresh rate of the device depends on the principal of persistence of vision. For a film system, the refresh rate was typically 24 frames per second, for a television system the refresh rate of the system was linked to the AC frequency of the power grid. In the United States, this means approaching 60 hertz, in Europe 50. When the beam would scan, it would go every other line. The total resolution, or pixel density, was a function of the persistence of the two fields of scan in time. On those old televisions, there were 525 lines, 480 of which were visible.

For an analog television, the interlaced scan with the number of lines would be expressed as 525i. Students today are accustomed to resolutions of at least 720p. P in this case stands for progressive scan, every line is illuminated on every scan, which typically now is not a scan but a refresh of pixels which happens at far higher rates than 29.97 fps.

Contemporary systems use high densities of individual pixels. When the pixel density exceeded 300 pixels per inch Apple branded this the “retina” display. At extreme resolution, the eye is not capable of seeing the individual dots, much like how the eye could not see the individual dots of the offset printer. The pixels are OLED (organic light emitting diodes) which are a form of transistor integrated technology that produces light. These diodes are capable of producing very high contrast images with deep black colors. For larger devices, the future technology is the QLED, also known as quantum dots. Each crystal of this system can produce a pure color, which then can be blended through the additive scheme to produce an image.

As of this draft, 4k displays are increasingly common. It is highly likely that resolutions will continue to increase. The more interesting question: do these higher resolutions have a great impact on semantic and aesthetic dimensions of experience?

3.6.1.2 Stereo-optical

In the mid 2000s there was a surge of interest in stereo-optical films. These were the next big thing, and films were being reprocessed for display on a stereo optical system. The hype was short lived – many users do not enjoy stereo-optical films and the quality of such experiences was poor. Stylistically stereo films seem to insist upon simplistic jump scares and crude objects reaching out of the screen. On a very technical level, the stereo film misses were discussed in our vision section – most visual cues are in fact mono-optical. Depth can be interesting, but it needs to be aligned with the other codes of the filmic experience. Thus, a well-produced and designed Avatar could be a spectacular stereo epic, while a buddy comedy run through an algorithm would not be.

There are two primary stereo-optical cues: stereopsis and convergence. Stereopsis refers to each eye receiving a slightly different signal. Convergence is the slight crossing of the eyes to focus on an object at short or intermediate range. The addition of these factors in a scene can produce the perception of depth.

There are three major technologies for this display:

Lenticular overlays: lens layers attached to a flat image, the lens produces a level of stereopsis.

Anaglyph: a filter is positioned in front of each eye corresponding to a colored filter on one of two projection cameras. Typically, this is done with red/blue or magenta/green pairs which tend to correspond with the color response of the retina (discussed in section 2).

Polarizing: the same basic design as anaglyph but with an angled polarizing filter and lens combination for the glasses and projectors.

Figure 3.10: Methods for stereo-optical presentation, drawn by Dan Faltesek for this book

Exercises

Demonstration 1: EYE DOMINANCE
With both eyes open, point at a spot somewhere in the room, like a clock. Now close each eye, you will notice that one of your eyes likely steered your hand despite both eyes being open while you were pointing. This is your dominant eye – you now have proof that the integration of your visual field is not perfect.

Demonstration 2: POLARIZING FILTER
Polarizing sunglasses are common, find a friend with a pair (anything from Warby Parker will work). Find an older cell phone, an iPhone 5S will work beautifully, newer phones will work as well. Place the sunglass lens between you and the phone. Rotate the glasses. You will see the phone screen darken or even disappear. When the filter and the photon source are aligned, they come through, when they are not aligned, they light is blocked.

Holograms: Instead of recording the image itself, a hologram records the interference pattern produced when two beams of highly focused light intersect.[39] The film becomes an interference recorder. When hit with another laser the film produces a reproduction of the interference from the original scanning laser. The advantage to this method of scanning is that many possible points and angles are recorded on the film.

Holography is not a new technology. The necessary elements for holograms are decades old, and unlike many forms of contemporary stereo optical photography, are technically difficult. If you use a contemporary flagship phone, interesting images cued by the movement of the phone relative to a static image source are quite impressive.

3.6.1.3 Rendering

Bob Ross’ painting programs are enjoyable. When you watch him, you will notice that he has a very precise method for rendering objects. He starts with those furthest and layers over them. His assumptions about light and color follow a clear painter’s algorithm, where that which is closest to the foreground is rendered last and in the most detail, distances are rendered with a broader brush and less detail. This is much the same way that most computer graphics are rendered.

The alternative is called ray-tracing. In this method, objects are supposed to exist in the world and light (shadows and reflections) are not determined deductively but inductively. Cars was the original film to use this method for rendering and it offers superior results for complex scenes. Now, the methods in rendering are working through the objects once again.[40] With increasing power in graphics rendering the capacity for real time ray tracing of games is in the very near future. If this will increase the degree to which games are meaningful or impactful is another question.

Photorealism is the end goal of all current rendering technology, which makes sense given that the end perception point is the human eye. Realism on the other hand is a far more flexible concept. There is in some sense an assumption that the future of graphics technology hinges on the production of ever more photorealistic images, at the same time some particularly powerful experiences like That Dragon, Cancer have demonstrated that a lower level of detail in some parts of the envelope can be offset by enhancements in others. This game, which you may be assigned to play for class, removes critical macro details like faces, while emphasizing the small details like the textures of surfaces and real sounds.

3.6.2 Sonic

Speakers use magnets to convert an electrical signal into physical motion. Sound is a vibration. For the most part these sounds are produced by vibrating a cone, other technologies may use incisor diffusion to vibrate larger surfaces.

The key is that the underlying physical properties of the system do not change. Smaller objects cannot resonate at the low frequencies of a great bass performance. An ear bud will not rock your body. Yes, woofers do real work.

Innovation in this space will likely come with the use of larger fields of speakers, more precise tuning, and careful, meaningful sound design. Sound is a case where the resolution of the underlying technology (what a speaker is) allows further refinement of the experience of that thing (sometimes paradigm shifting innovation is less robust than simple improvement). Systems in the future may detect the locations of pared devices and the space around those devices to optimize how many speakers and in which bands those speakers operate.

The most profound dimension of sonic development is not in the entertainment space, but in the creation of adaptive devices. Mara Mills history of the miniaturization of the hearing aid is powerful: it was both the original use case for the integrated circuit and a form of technical miniaturization that transformed everyday life for many people.[41] It is the social role and image of technology, rather than the elements of the system which drive reality. At the same time the demand for invisibility of the device and continued stigma is a powerful factor:

Today, the imperative of invisibility largely persists as a design standard for hearing aids, with the demand for miniaturization often limiting device functionality. Recent examples of fashionable earpieces compete with new models of ‘‘completely-in-canal’’ invisible aids. As a long view of hearing aids makes plain, hearing loss has been stigmatized despite the increasing commonness of the diagnosis, and despite the fact that moderate hearing loss can be remedied bytechnical means. Just as inexplicable is the obduracy of the stigma that adheres to the technology itself—when hearing aids have otherwise represented the leading edge of personal electronics, and when they exist as one configuration of the same components found in so many other appliances.[42]

This is a powerful example as the benefits and expansion of communication enabled by the device, yet the coding of culture continues to dramatically shape how sensation is produced. Speaker technology is relatively stagnant, when we expand to consider what hearing and listening technologies could be the realm of possibilities and representations dramatically expands.

3.6.3 Haptics

The primarily strategies by which sensations are produced come in the form of small electric motors and electric charges. When these are mapped to other stimuli a full faceted haptic experience may be produced. At the same time, the dimensions of perception tied to the position of the body and perception of relative space may not be fully simulated by the system. More importantly, the actual kinematics of a human body are not effectively reproduced by an electric motor bar, point, or puck. Consider your perception of a brush against the skin of your forearm: there is both the friction of skin on skin, but also warmth, pressure, and variation across the stroke. There are at least five dimensions to plot. It would make sense that scandalous applications of the technology have been dominant so far – these are the low hanging fruit for the production of sensation with a simplistic criterion for success.

An important question raised by David Parisi, the major theorist of haptic communication: Are touch-screens haptic? His answer: to a degree.[43]What is important to understand about touchscreen systems is that they are not fully haptic – they are not the entire enfolding of sensation, but a very limited slice of that envelope. Force reactions on a Nintendo DS or a cell phone screen are intended but for one patch of skin and one set of interactions. The rhetoric of the touch screen is instructive here: the image always features a finger touching the screen, it does not move through the screen to form a contact point with the world beyond.[44]

What is especially striking from an interface studies perspective is the degree to which the ostensibly haptic interface of the touchscreen displaces what would have been richer interaction points like sliders.[45] Haptics demand the consideration of the relationship between the limited slice and the entire sensory enfolding.

3.6.4 Aroma and Flavor

Devices for the production of virtual smells and tastes were discussed in section two. For the most part the strategy here is to simply load a handful of relevant chemicals into a system that can then produce those chemicals on command. What is so difficult about these senses, is that they lack the deep similarity of basic inputs that the first three output systems share. There is no EM spectrum of spicy. There are technologies for taste which can use electrodes in the mouth to produce an electrical signal that tastes like something. An 800hz signal in the mouth tastes like lemon.[46]

It is possible that our best technologies in this area are in fact barred from use. Increasingly consumers are interested in natural foods, the taste of strawberry must come from the shattered cell walls of a morsel, not from a bottle. This is an interesting case where the purely refined sign is not really what people want, as if the sign of cherry is not so much the almond like taste but something else entirely.

Smell and Taste will not change, but the ways that we feel about particular aromas and flavors will. The new media of the future in these spaces does not look like the simulation of an entire enfolding, but the production of new experiences and technologies that would be consumed in the world as we know it: this is another important point to remind you that the virtual does not depend on goggles, you already inhabit that virtual world.

3.7 Abstraction

Abstraction is a powerful concept. In art, the move toward abstraction allows the artist to be free of the purely iconic or mimetic, to develop works that have qualities that might evoke a feeling without relying on so many established identities. Piet Mondrain and the artists associated with the de stijl movement attempted to reduce the work of art to the most basic elements, in the final form blocks of color set into grids.[47]

Figure 3.11: Composition no 8, Walid Kandinsky, 1923, https://de.m.wikipedia.org/wiki/Datei:Vassily_Kandinsky,_1923_-_Composition_8,_huile_sur_toile,_140_cm_x_201_cm,_Mus%C3%A9e_Guggenheim,_New_York.jpg

In software, abstraction allows the development of many powerful tools. Instead of taking a nearly pointillist orientation toward the encoding of software to actually run on the chip itself. At the lowest level, machine code drives the utilization of the gates that make the computer work. At higher levels, programing languages allow users to deploy abstractions. As the programming moves to higher and higher levels, the abstractions become increasingly understood by users. A page written in HTML and CSS may actually be quite readable by a human. Over time, language developers may produce new abstractions for functions that were once accomplished with much more labor at a lower level.

Figure 3.12: Blocky HTML by Dan Faltesek

The grid aesthetic is almost second nature for the web today. Users are not expecting a sophisticated layout that requires them to relearn how to use a computer. These tabular layouts were extremely common and clever websites could be made long before standards for our current layouts existed. Over time, users began to use more devices (such as phones and tablets) and the need for layouts that could adapt became clear. Web developers were then writing div descriptions that would operate in a variety of contingencies. Today, the HTML 5 standard includes abstract representations of positions on the page and adaptations. For APIs and software libraries, many of the functions they offer for data analysis of manipulation are not new. In computer science, this is called refactoring. When code is rewritten with base instructions as functions for repetition and ease of reading, that code is improved.

Abstraction as a semiotic process allows the formation of powerful symbolic signs that greatly increase the power of communication. Yet every abstraction conceals and excludes. An important dimension of our study and production of the future is the role of the abstract in granting access to systems for new developers and designers.

3.7.2 Graphical User Interface

Among the most powerful abstractions for new media is the graphical user interface. Instead of asking that users compose lines of code to access their information of perform operations, the GUI produces a continuous visual state where a pointing device can allow users to move blocks of information or select items in a metaphorical world of positions and objects. Users intentions are then mapped onto visual properties which become the interface with the complex system.

Figure3.13: Douglass Engelbart Institute, image from the Mother of All Demos, 1968. http://www.dougengelbart.org/content/view/209/448/

In 2005, Jeremy Reimer wrote a strong history of the GUI (to that point) for Ars Techinca. It is linked here and is suggested reading for everyone. A key detail in this history is the discussion of the Mother of All Demos – the occasion on December 9, 1968 where Douglas Engelbart demonstrated the first computing system with a graphical user interface (a mouse moving abstract blocks on a screen). You may notice that aside from a few changes, the basic technology from his round screen, the basic theory of the GUI is unchanged.

Graphical user interfaces hinge on the use of a pointing device. Typically, this has meant a mouse, trackball, touch-screen, or touch-pad. These devices allow the user to direct the system to recognize the importance of a functional point in virtual space. More recently, the role of the dedicated pointing device has been supplemented by touching directly on the screen. As multi-touch interfaces become increasingly common, users will begin interacting through touch in increasingly sophisticated ways.

Keyboards are relatively static. There are a number of ergonomic alternatives to the current layout. If the hands are positioned slightly differently repetitive stress injuries may be mitigated. Some new designs suppose that a projection of a keyboard, paired with sensors for where the fingers move over that light will replace the physical interface.

A great deal of change can be expected in this area, with-in some boundaries. It seems that the underlying trajectory toward less typing will continue. Typing has a high level of information entropy, people misspell things all the time. Further, it is reasonable to expect pointing devices to become more sophisticated but to so sophisticated that they will become cognitively taxing. Beyond the cognitive, a new set of limits may come into play: the structures of the human wrist. This also goes for keyboards. Is it really cognitively worth it for people who keyboard to learn a non-QWERTY format? For this author, the answer is decidedly no.  The future of the keyboard does not hinge on creativity, as much as it does on cognitive and physical limits. Any new innovation in the keyboard space will need to replace the functions of the existing keyboard for many users in a way that provides some very real benefit. Change in pointing devices is more likely than a change in keyboards.

In the market, currently there are roughly three major philosophies for GUI design. Apple’s human interaction guidelines push for a more photorealistic system of icons and animations. Google’s Material Design tends to push of a geometric look that uses a variety of layers to indicate priority and control. Microsoft’s Fluent Design emphasizes the underlying usability of a particular thing.

It is highly likely that all three of these frameworks will change. You will also notice that this list does not include the standards proposed by Facebook, Snapchat, or any number of other firms. Approaches to the graphical user interface will be various and important for understanding interaction on the whole.

3.7.3 Human Computer Interaction

The study of human computer interaction has become a robust field onto itself.

Usability studies, developed by Jakob Nielsen, contends that the primary focus for understanding interaction should be the user task.[48] The designer of a system has some particular thing that the user needs to be aware of and able to manipulate. Usability research deploys a number of different social science strategies for the analysis of user tasks, especially those that can work across the life cycle of the project. The drawback to this approach is that it is involved in problem-solving, not necessarily problem finding. Tasks are provided by the client, the goal is to make the thing work so that the task can be completed.

The challenge of these approaches to affect and HCI (human-computer interaction) is the question of ends. Consider these four major paradigms for the study of human-computer interaction in the introduction to a major research volume:  Emotional Design, Hedonomics, Kansei, and Affective Computing.[49] Emotional design is best described in this book as it is tied to usability studies. The aesthetics and configuration of a system are matched to the design of a system through a number of approaches to the evaluation of both the users and the system. Hedonomics supposes that a system must be designed to maximize the pleasure of the user, while Kansei engineering is an example of a framework that supposes a strong ontological typology of users. Consider for a moment the continuing popularity of personality types: what if we designed systems to match with the results of the Meyers-Briggs inventory? Finally, affective computing is a cybernetic model which would describe a situation where the computer might try to continuously adapt to, and modulate the affect of the user. The differences between these approaches are useful for our theory. Each one of these approaches must theorize who the user is, how many different types of users there are, if they can be designed for in advance, and if a universal design is possible.

Human-computer interaction is not a single concept that one simply learns as a set of best-practice recommendations, but an entire domain that fuses an ongoing trajectory of research and development. Every time Facebook, Snapchat or any of the other big social network companies change their interface you see this in action. They have particular ends that they must satisfy – keeping you engaged with great content while inducing clicks on advertisements. The balancing act between those goals is not solved directly by an equation. Furthermore, the aesthetic progression produced by these interfaces changes the conditions by which they are designed.

3.7.4 Brain Interfaces and Other Inputs

All media technologies are providing a poor substitute for telepathic communication. The goal for interface technology is the development of interfaces that would allow thoughts to be scanned directly from the brain and then delivered directly into the brain of another. This seems to be a long way off. The deepest problem here would seem to be neuro-plasticity: the brain of each person is decidedly different. Consciousness, as much as it is understood, is not a mechanical property of the brain that can be located in a single point but an emergent force of a number of different processes all blending together.[50] In 2008, Gary Small’s research was commonly discussed as providing evidence that Google searches prompted the brain to utilize more oxygen than reading – of course the problem, as he maintained, was that the interpretation of more oxygen utilization meant nothing.[51]

N. Katherine Hayles argues this in the context of the development of literacy: there is no single part of the brain that produces the ability to read, and we know that the development of reading and writing appeared as a paradigm shift in human behavior.[52] The key seems to be that multiple parts of the brain were developing reading like parameters at the same time, when these were linked the possibilities were dramatic.

Consider the model necessary for full telepathy: thoughts must be extracted and transcoded into a meaningful for reproduction through a decoding process at the receiver. Anything less than that, and we are once again dealing with a semiotic process where signs are presented to the senses. Thus, our concerns throughout this course with simulation and standardization. Singularity is unlikely.

This does not mean that interfaces involving the brain are not promising. Scanning methods have allowed scanners to read text from the brain. It is possible that new systems will allow those who are locked in to rejoin the world of symbolic production. This is wonderful. Cognitive pupillometry is a well-established concept, as the pupil dilation changes we can detect shifts in the level of cognitive work.[53] Eye focus scanning allows military helicopter gunners to track targets. Galvanic skin response and body position detection can offer rich interface possibilities. Motion scanners offer great fun for headset games, artistic work, and software interface.

In a world where people elect to carry computers (cell phones) there is much data to be collected about the physical location of phones, their relative speed, the accelerometer information (how is the phone moving in space), among other sensor inputs. People elect to use these devices to store biometric data as well, meaning that all of these ambient inputs also provide a world of information for interface with virtual worlds. What the challenge is with these other interfaces is that they are not so neatly intention driven.

There will be much change in the world of alternative interfaces. It is important to keep in mind that the change here is likely limited by the capacity for input into the human as well as the interposition of semiotic code relationships. This returns to the metaphor of the enfolding of the virtual: it is not simply finding a way to get thoughts into or out of a person, but that the ways that those thoughts refer to each other and others held by other people is an equal dimension of the experience of that which is virtual.

3.8 Games

Game theory has offered an important set of conceptual tools for the analysis of complex iterative systems. What does that mean? Games are important because they have multiple turns, and within those turns, the players consider the actions that others may take, in the context of multiple constraints, mechanisms, and story elements, crafting provisional strategies to reach victory or defeat.

Katherine Ibster provides two key points that distinguish games from non-game media: choice and flow.[54] The two points are interlinked – it is not merely the game continues to move but that the choices you make along the stream effect the flow. Flow exists in a tenuous equilibrium between challenge and player skill, great emotional design allows the user to stay in this seemingly ideal zone where they are learning and experiencing change.[55] It can be useful to consider the types of uncertainty generators that are present in games, as proposed by Costykian, including the player, a random generator, and other players.[56] Games vary greatly based on where they find the randomness necessary for fun.

Beyond the solitary game, playing together is important: it is the depth of interaction between people that makes these game systems truly deep.[57] Game theory relies on this assumption to provide critical insight into human behavior akin to the results of a system of equations: the assumption in a game involves active people attempting to arrive at some outcome.[58] Games are everywhere and are deadly serious.

Ian Bogost’s conception of procedural rhetoric is particularly useful for understanding the future of media, the ways that particular software affordances can be mapped to the experiences of the game player.[59] Rhetorically the video game must be understood both through its total semantic content and the coded means of delivery. The peak of games for Bogost is a complex system where a user is made to disidentify with their own position by manipulating a complex system: Sim City.[60] This perspective is ludological.

Kishonna Gray makes the counter point: rather than evacuating games of identity the identities in the game must be challenged.[61] This is narratological: the game is a story that is a way of understanding the world, it is not a psychological procedure but a way that people experience stories that make their personal worlds. Games are stories that deeply compelling. To see them as less than that or as flat objects that people are not invested in misses their power.

This framing device, a debate between positions, has some truth in it but also is largely artificial. I present it this way so that we might understand the two lines as they braid into the future: there will be new stories, but also new ways of interfacing with the system that will produce many new affordances. There is some value in debunking such a distinction between play structure and story, but also great utility in reading each.

At the same time, games are critical for understanding human behavior. As both a literal structure and a metaphor, games describe a recursive pattern of behavior. Thus, as a form for thinking about the future, games are important as they give us some sense of the real dynamics by which strategies form.

3.9 Narrative

Stories are everywhere. Stories increase emphatic awareness of others situations. Stories may even be the basic structure of being itself. Stories aren’t going anywhere – but the content of those stories will change. It is also important to understand that ideas from prior stories are referenced in future stories. This is called intertextuality. The play between the present consideration of a text and all of the meanings loaded into forms and tropes are important.

Joseph Campbell identified a commonly used structure known as the hero’s journey: this formalist template can be identified in many successful stories.[62] This does not mean that those stories are all the same or that there is no variation, but that rhetorical forms evolve over time. Changing stories have a low velocity. The mythic structure is particularly enduring regardless of the choice to include vampires or Mr. Darcy.

Leslie Baxter and Barbara Montgomery provided a different approach to understanding narrative in everyday communication – flowing from Bakthin, they see our personal stories as a product of forces that push people together and pull them apart.[63] This is an important idea as it can give us a way to read stories that are developing and those that are less than logically coherent. Reading the future is not a matter of binary logic, but of complex fuzzy interaction across space and time.

Kenneth Burke approached the use of narrative in public life through the idea of the pentad. The core of the pentad are the five basic units of a drama: scene, act, agent, purpose, agency.

Scene

The Setting

Act

The Action

Agent

The Character

Purpose

The Reason Why

Agency

The Method of Action

The pentad provides us the foundation of a theory called dramatism, where a formal structure can be applied to any number of communication phenomena. Professor Ragan Fox at CSU-Long Beach has a spectacular full chart that anyone interested in learning Rhetoric must review.[64] These appear as “ratios,” where two elements of a story combine to form the core logic of that story.

Where Campbell’s approach leads us to consider the sort of stock stories that populate our worlds, Baxter and Montgomery provide resources for seeing how those stories are translated into action in our everyday lives. Burke, on the other hand, provides a near guide to how the ratios might be deployed in everyday life. This returns to the point of the semiotic interposition of signs and facts.

3.10 Immersion

Marie Ryan argued in an essay over a quarter of a century ago that the real immersive power of the game is in the intertextual enfolding of the user.[65] Team chat and interaction are what is really interactive, merely pushing buttons is not. This is an important idea. What does it mean to have enough contact with an interface to say that you are in communication with a conscious system? The common answer to this question is the idea of the Touring test – the idea that a system that could simulate relatively banal interaction over a textual transmission would be AI. In this time period, we can see a remarkable number of systems which are capable of producing far more vivacious simulations of human interaction.

It is likely that what constitutes immersion will change. As social media systems have reached maturity, it has become clear that they are a vector for hateful and hurtful communication. The friendly banter that would have made a world compelling has fallen away.

Interactivity is merely the chain of indexical reactions to user input. Ryan proposed a useful framework for evaluating interactive controls: speed, range, and mapping. These controls allow us to have some vocabulary for evaluating different levels of interactivity with a system.

The most interesting question: do people actually want immersion? Are any of our new technologies that much more immersive than an excellent television show or a novel?

3.11 Language

It is likely that language, particularly the semantic dimensions of everyday code will change most. This is totally tubular. As much as ideas are constantly in flux in an attempt to symbolize what we mean, which is always to a very real degree barred from what other people might mean, language will be dynamically shifting in an attempt to keep up. This is not just a property of idioms, grammatical structures and spellings shift often as well. There are so many languages and codes. It should be clear why humanistic research is so important, we need to understand how these code systems work and how they are changing.

1. Claude Shannon, “A Mathematical Theory of Communication,” The Bell System Technical Journal 21 (October 1948): 379–423, 623–56. ↵

2. John R. Stuart, Language As Articulate Contact: Toward a Post-Semiotic Philosophy of Communication (State University of New York Press, 1995). ↵

3. Kathryn E Anthony, Timothy L Sellnow, and Alyssa G Millner, “Message Convergence as a Message-Centered Approach to Analyzing and Improving Risk Communication,” Journal of Applied Communication Research 41, no. 4 (October 2013): 346–64. ↵

4. This is a strong resource developed for a course at the University of Chicago in 2007. Hua-Ling Linda Chang, “Semiotics,” Keywords Glossary::semiotics, 2007, http://csmt.uchicago.edu/glossary2004/semiotics.htm. ↵

5. John E. Joseph, “Ferdinand de Saussure,” Oxford Research Encyclopedia of Linguistics, June 28, 2017, https://doi.org/10.1093/acrefore/9780199384655.013.385. ↵

6. Sean Hall, This Means This, This Means That: A User’s Guide to Semiotics (London: Laurence King Publishing, 2012). ↵

7. Daniel Chandler, “Semiotics for Beginners: Signs,” 2006, https://www.cs.princeton.edu/~chazelle/courses/BIB/semio2.htm. ↵

8. Roland Barthes, Mythologies, trans. Jonathan Cape (Paris: Noonday Press, 1991). ↵

9. McGee, Michael Calvin, “The ‘Ideograph’ a Link between Rhetoric and Ideology,” Quarterly Journal of Speech 66, no. 1 (1980): 1–16. ↵

10. Nancy Frasier, “Rethinking the Public Sphere: A Contribution to the Critique of Actually Existing Democracy,” Social Text 25/26 (1990): 56–80. ↵

11. Michael, Warner, “Publics and Counter-Publics,” Quarterly Journal of Speech 88 (2002): 413–25. ↵

12. Greene, Ron, “Rhetorical Pedagogy as Postal System: Circulating Subjects Through Michael Warner’s ‘Publics and Counterpublics,’” Quarterly Journal of Speech 88 (2002): 433–43. ↵

13. Thomas A. Hollihan and Kevin T. Baaske, Arguments and Arguing: The Products and Process of Human Decision Making, Third Edition (Waveland Press, 2015). ↵

14. Luc Boltanski and Laurent Thévenot, On Justification: Economies of Worth, trans. Catherine Porter (Princeton University Press, 2006). ↵

15. Donald A. Norman, Living with Complexity (Cambridge, Mass: The MIT Press, 2010); Donald A. Norman, The Psychology of Everyday Things (Basic Books, 1988). ↵

16. Arturo Escobar, Designs for the Pluriverse: Radical Interdependence, Autonomy, and the Making of Worlds (Duke University Press, 2018). ↵

17. Lee Vinsel, “Design Thinking Is a Boondoggle,” The Chronicle of Higher Education, May 21, 2018, https://www.chronicle.com/article/Design-Thinking-Is-a/243472. ↵

18. Chapter eight is the classic on affordance theory. Your instructor should likely provide you a copy of this chapter for additional discussion. James Jerome Gibson, The Ecological Approach to Visual Perception (Psychology Press, 1986). ↵

19. Norman, Living with Complexity. ↵

20. Nikil Saval, “Brutalism Is Back,” New York Times Style Magazine, October 6, 2016, https://www.nytimes.com/2016/10/06/t-magazine/design/brutalist-architecture-revival.html. ↵

21. Charlie Tyson, “Why Professors Distrust Beauty,” Chronicle of Higher Education, June 3, 2018, https://www.chronicle.com/article/Why-Professors-Distrust-Beauty/243548. ↵

22. Alex Williams, “Normcore: Fashion Movement or Massive In-Joke?,” New York Times, April 2, 2014, https://www.nytimes.com/2014/04/03/fashion/normcore-fashion-movement-or-massive-in-joke.html. ↵

23. J. Kenji López-Alt, The Food Lab: Better Home Cooking Through Science, n.d., accessed October 26, 2018. ↵

24. Good Eats was an important culinary television program that may be in production when you read this. ↵

25. Isaac West, “Performing Resistance in/from the Kitchen: The Practice of Maternal Pacifist Politics and La WISP’s Cookbooks,” Women’s Studies in Communication 30, no. 3 (2007). ↵

26. ? ↵

27. “Why the Fries Taste Good,” Lexicon of Food, November 2, 2015, https://www.lexiconoffood.com/post/why-fries-taste-good. ↵

28. Eric Schlosser, Fast Food Nation: The Dark Side of the All-American Meal (New York: Houghton Mifflin, 2012), 234. ↵

29. “Why the Fries Taste Good.” ↵

30. Lorna Roth, “Looking at Shirley, the Ultimate Norm: Colour Balance, Image Technologies, and Cognitive Equity,” Canadian Journal of Communication 34, no. 1 (March 28, 2009), https://doi.org/10.22230/cjc.2009v34n1a2196. ↵

31. Robert Perkins, “Ultra-Thin Camera Creates Images Without Lenses” (CalTech, June 21, 2017), http://www.caltech.edu/news/ultra-thin-camera-creates-images-without-lenses-78731. ↵

32. Steve Huff, “The Future of Cameras, Gear and Photography. The Mirror Is Dying.,” Steve Huff Photo, June 7, 2018, http://www.stevehuffphoto.com/2018/06/07/the-future-of-cameras-gear-and-photography-the-mirror-is-dying/. ↵

33. “Recording Studio Microphones: The Ultimate Beginner’s Guide,” E-Home Recording Studio, May 28, 2012, https://ehomerecordingstudio.com/types-of-microphones/. ↵

34. Benjamin, Walter, The Arcades Project, ed. Tiedemann, Rolf, trans. Eiland, Howard and McLaughin, Kevin (Cambridge: Belknap Press of Harvard University Press, 2002). ↵

35. Janko Roettgers and Janko Roettgers, “Netflix’s Secrets to Success: Six Cell Towers, Dubbing and More,” Variety (blog), March 8, 2018, https://variety.com/2018/digital/news/netflix-success-secrets-1202721847/. ↵

36. Vadim Rizov, “~31 Films Shot on 35mm Released in 2017,” Filmmaker Magazine, April 5, 2018, https://filmmakermagazine.com/105050-31-films-shot-on-35mm-released-in-2017/. ↵

37. Phil Green, “DI - The Conform,” The Digital Intermediate Guide, 2006, http://www.digital-intermediate.co.uk/DI/DIconform.htm. ↵

38. Xcode ↵

39. “What Is Holography? | Holocenter,” accessed October 31, 2018, http://holocenter.org/what-is-holography. ↵

40. “Stories | Racing for Realism,” accessed October 8, 2018, https://renderman.pixar.com/stories/cars-3. ↵

41. M. Mills, “Hearing Aids and the History of Electronics Miniaturization,” IEEE Annals of the History of Computing 33, no. 2 (February 2011): 24–45, https://doi.org/10.1109/MAHC.2011.43. ↵

42. Ibid. ↵

43. The discussion of the touchscreen genealogy as it intersects with the haptic is perhaps the strongest point. Parisi ↵

44. 280 ↵

45. 286 ↵

46. Honor Whiteman, “‘Digital Taste Simulator’ Developed That Tickles the Tastebuds,” Medical News Today, accessed October 31, 2018, https://www.medicalnewstoday.com/articles/269324.php. ↵

47. Mondrain ↵

48. Jakob Nielsen, Usability Engineering (Boston: Academic Press, 1993). ↵

49. Affective computing ↵

50. Antonio Damasio, Descartes’ Error: Emotion, Reason, and the Human Brain (Penguin, 2005). ↵

51. “Digital Nation,” Frontline (PBS, February 2, 2010), https://www.pbs.org/wgbh/frontline/film/digitalnation/. ↵

52. Daniel Kahneman, Thinking, Fast and Slow (New York: Farr, Strauss, and Grioux, 2013), 33. ↵

53. Kahneman, 33. ↵

54. Katherine Isbister, How Games Move Us: Emotion by Design (MIT Press, 2016). ↵

55. This is an important claim in a number of different works on games. The “flow state” is theorized as a form of positive interested attention, as opposed to obsession or addiction. Isbister, 6. ↵

56. Greg Costikyan, Uncertainty in Games (Cambridge, Massachusetts: The MIT Press, 2013). ↵

57. Marie-Laure Ryan, “Immersion vs. Interactivity: Virtual Reality and Literary Theory,” Post-Modern Culture 5, no. 1 (1994), http://www.humanities.uci.edu/mposter/syllabi/readings/ryan.html; Isbister, How Games Move Us. ↵

58. Ken Binmore, Game Theory, A Very Short Introduction (New York: Oxford University Press, 2007). ↵

59. Ian Bogost, “The Rhetoric of Video Games,” in The Ecology of Games, ed. Katie Salen (Cambridge: The MIT Press, 2008). ↵

60. Ian Bogost, “Video Games Are Better Without Characters,” The Atlantic, March 13, 2015, http://www.theatlantic.com/technology/archive/2015/03/video-games-are-better-without-characters/387556/. ↵

61. The Editors, “The Futures of Game Studies,” The Velvet Light Trap 81, no. 1 (February 23, 2018): 57–57. ↵

62. Christopher Vogler, The Writer’s Journey: Mythic Structure for Writers (Studio City, CA: Michael Wiese Productions, 2007). ↵

63. Barbara M. Montgomery and Leslie A. Baxter, Dialectical Approaches to Studying Personal Relationships (Psychology Press, 2013), 53. ↵

64. “Ragan Fox Breaks Down Kenneth Burke’s Pentadic Ratios,” Musings in Pop Culture & Pedagogy, October 21, 2013, https://ragan.blog/2013/10/21/ragan-fox-breaks-down-kenneth-burkes-pentadic-ratios/. ↵

65. Ryan, “Immersion vs. Interactivity: Virtual Reality and Literary Theory.” ↵

Section 4 – Methods

You should not be universally skeptical, but also not universally trusting. You need to understand how and why certain methods work and others do not. Aligning methods and questions may be the most important skill for the future. In this section I will layout a number of methods for doing research in media, with the hope that you will understand what good research looks like in each area, so that you may ignore and refute the bad.

Methodology is a loop. Without qualities, there are no meaningful qualities. Without some sense of quantity, qualities are unmoored from reality. There is no qualitative answer that can eliminate the need to count, just as there is no way to count to an assembled claim. Communication Studies is a lucky discipline as our questions are not welded to a single methodological chassis.

4.1 Qualitative

Qualitative research presents descriptions, interpretations, and criticism. The question of description: what was the text? What does it mean? How do the codes work? To what end were those codes selected?

Qualitative research is primarily concerned with the qualities of the cases in the study in question. Computational methods tend to impart indolence with regard to the question of the data themselves or worse an aggressive pruning of ambiguity which makes for clean data, but poor models of human communication. Great qualitative research is rigorous, deeply informed, and essential to decision making, ethics, and meaning.

4.1.1 Ethnography

As a practice of tracing the everyday performance of individuals, ethnographic research, particularly field work, sees the body and experience as the basis for analysis. Researchers in this tradition conduct interviews and spend time in the field. Rigor comes from demonstrating that one was really embedded in the system of meanings that are used by a group of people or in a particular context. To maintain a structured system of thinking, the researcher continuously produces field notes, which serve as an intermediate document of what things meant to the individual at a particular point in time. These structured notes then allow the researcher to remember key details to construct the finished account later. Finalized research then is the product of a review of these notes and a reconstruction of the symbolic system. This research is not published as a sort of abstract formula, but as a rich account of the meaning itself. Great ethnographic work may read more like a novel than a lab report.

Dwight Conquergood, a major ethnographer of communication, in his development of a critical ethnographic practice called for the ongoing revaluation of the sources of authority for those who would claim to know what meaning is in a culture. The key to this performance ethnography is constant self-reflexity: the researcher needs to understand how their mode of presenting themselves and the world produces their own academic authority. Descriptions of hardship often seem to provide ethnographers credibility. Why such an intense struggle? Meaning is owned by a group, it is not hard to find many troubling examples of cultural appropriation where information is scooped up and resold as a product with little remuneration of the original producers. More instrumentally, there could be multiple explanations of the same phenomena. Ethnographers have many interesting strategies for demonstrating rigor and building credibility. When these are well deployed, and when the account is ethical and engages the community, the results are profound. It makes sense why large corporations like Microsoft pay for academic ethnographic research. These are the insights that can make a game changing product.

It is the power of reflexive ethnography that is often mistaken as the seemingly magical dimension of design. If the designer is truly, reflexively integrated into the use context of a thing or idea their designs will really resonate in a powerful way. At the same time, it becomes comically excessive when people who are not embedded claim the same sort of authority or to make the same claims to meaning as those who have not engaged. But wait – notice that in the creation of authority for the ethnographer in the last sentence I have already deployed a rhetoric of effort and time, as if that work somehow gives a person the right to make authoritative claims about meaning simply on the basis of duration. Would it make the claim more powerful if I said they experienced hardship?

Ethnography is a powerful approach, and people who have the patience and the reflexive sense to do this well are rare. At the same time, this is not a special property of some people, but a skill that is refined over many years. It is entirely possible to do meaningful work in this area that does not rise to the level of publishable ethnographic research. Interview and observation methods have real value, and can contribute to your efforts. Good research in this vein will be deliberate and reflective, making modest claims with abundant evidence. Bad research will be quick and easy, a cover for pre-established conclusions and bad ideas.

4.1.2 Hermeneutics

The hermeneutic tradition comes from an ancient sort of grandmother discipline called philology which focused on textual interpretation. Many of the discussions of semiotics and particular media in this book offer approaches to understanding meaning.

The traditional seven question typology of hermeneutics as presented in the Stanford Encyclopedia of Philosophy:

1. Who (is the author) (quis/persona)?

2. What (is the subject matter of the text) (quid/materia)?

3. Why (was the text written) (cur/causa)?

4. How (was the text composed) (quomodo/modus)?

5. When (was the text written or published) (quando/tempus)?

6. Where (was the text written or published) (ubi/loco)?

7. By which means (was the text written or published) (quibus faculatibus/facultas)?[1]

I often recommend these as a starting point for any student looking to understand their readings in class. These questions can help you reconstruct the state of an academic discipline or in the original case, to reconstruct the text of the Bible. Many of the ideas in hermeneutics are similar to those deployed in the theorization of the conditions of possibility in communication studies.

Hermeneutic approaches follow a tradition called close reading, this employs an interpretive strategy to be used on a particular text. Good work in this area will employ a structured framework and demonstrate how different parts of the code work. Ultimately it is about the code. Or as Jacques Derrida put it: there is no outside of the text.[2] A deep account of who wrote what, when, where and why can change how you think.

For undergraduate audiences, hermeneutic critique is presented as the work of the  “masters of suspicion” (Nietzsche, Marx, Freud) as designated by Paul Ricoeur – this presentation flattens a number of unique interesting questions presented by each author, and tends to position the world in a troubling critical/non-critical binary.[3] In her critique of this binary, Rita Felski argues that the binary misses the range of possibilities and texture that come from any number of approaches to interpreting the text:

In a related essay, I scrutinize some of the qualities of a suspicious or critical reading practice: distance rather than closeness; guardedness rather than openness; aggression rather than submission; superiority rather than reverence; attentiveness rather than distraction; exposure rather than tact (215–34). Suspicion, in this sense, constitutes a muted affective state—a curiously non-emotional emotion of morally inflected mistrust—that overlaps with, and builds upon, the stance of detachment that characterizes the stance of the professional or expert. That this style of reading proves so alluring has much to do with the gratifications and satisfactions that it offers. Beyond the usual political or philosophical justifications of critique, it also promises the engrossing pleasure of a game-like sparring with the text in which critics deploy inventive skills and innovative strategies to test their wits, best their opponents, and become sharper, shrewder, and more sophisticated players.[4]

Skilled critics working with codes can do great work. We should take Felski’s warning seriously as we develop new interpretative frames for the future but also to understand when we should engage in the practice of translational critique. This is an important political moment in any academic project: when is the theory complete and the time for intervention?

4.1.3 Historicism

In 2015, a film crew went on a dingy quest in New Mexico in search of lost video games, in this case 881 ET Atari cartridges.[5] The question is why? The cartridges were dumped because of the lack of meaningful control of the production of games for the 2600 which resulted in a glut of terrible games. For undergraduate teaching this is a wonderful example – you can teach the political economy of the game industry, touch on the legal structures which enable production, and teach key points in history of the product: all at the same time.

Media history is a nexus that allows the introduction of compelling questions. Much of the excellent work of John Durham Peters has been concerned with the development of logistical media technologies (like calendars and towers).[6] The inspiration of Frierich Kittler on Peters should be apparent, with an important caveat: Peters tends to assume that the driving force for media development is longing, Kittler assumes armed conflict.[7] There are any number of ways of thinking and assembling evidence, for media historians the work is always translational, the assembly of ideas and archives are intended to work along an explanatory line. For more traditional academic historians this may be premature, as the inductive collection of facts in bulk is necessary before embarking on a structured pattern of noticing.

Good historical work actively reduces its own scope. Historical research expands geometrically – every time the project attempts to explain something else it grows to include the new idea and all the contact points between the idea and the existing project.

History, when done well, provides the most detailed possible accounting for the conditions of possibility for the present.

One of the most common forms of historical reason that you will see is not the academic history, but the business case study. A case is isolated in time and reduced to what Wes Rumelt describes as a kernel, where a single set of coherent policies could have produced actions that would have changed the direction of a business enterprise.[8] Business reasoning, as a form of historicism, assumes that the understanding of past forces can then be inferred as a continuous function that will shape the future. The greatest flaw to this sort of media history is that the accounts created will almost always feature “winners” discussing their brilliance. Narrow, self-serving, yet historical. Both undergraduates and MBA students will be writing cases for the foreseeable future. It should be clear that this mode of reasoning is in common use in everyday life and it can help us understand why it would be a good use of time and money to dig up a landfill.

4.1.4 Rhetoric

Rhetoric is an academic field that emphasizes the probability of effect for a particular utterance especially if that is a persuasive utterance. This field deeply informs this book and the impact of it can be seen across the text.

4.1.5 Policy

Figure 4.1: Mann’s Hockey Stick, this rendering by Klaus Bitterman, Wikimedia, 2013 https://en.wikipedia.org/wiki/File:T_comp_61-90.pdf

Policy analysis is concerned with the translation of other findings about the world into a meaningful course of action. This is normative, meaning that it concerns what SHOULD be done rather than what is, was, or could be. Normative work is elegant and enjoyable, but it often misses the messy scene that it would intervene into.[9] Human judgement is the final arbiter of political truth. The earth is getting warmer. It is likely that this will cause problems. These are facts. What is tricky is how these facts of techno-science are translated into a meaningful policy. Is the solution to outlaw cars or airplanes? Should all electrical generation be nuclear? Are hydroelectric dams (a major source of power at my current location) on balance better than coal?

Idealistic students are often vexed by the inability of the public to understand “science.” As Damien Pfister argues, the issue is not that the science of warming is in particular dispute, but that the translation of that into a viable policy is not a matter of techno-science but a question of policy, and the hockey stick graphic became a point where science, expertise, and participation interweaved.[10] Scientific authority is constructed around the discourse of description, the scientist’s instruments tell it how it is. The problem is that a measurement of atmospheric pressure is not particularly interesting. Among the most profound facts in our world are social facts – these include things like the unemployment rate and public opinion. These are deeply imperfect facts where the means of measurement were designed for the resolution of a problem, not the seeming objective question of science.

This is not to say that warming isn’t real, but that debates about warming are discussions of policy, not statements of fact.

4.1.6 Legal

Legal reasoning then is the application of an interpretation of what the law is to a particular set of facts. The framework for interpretation is largely determined by professional jurists, while the facts are determined by the rhetorical process of the jury trial. Juries serve as fact-finders. Once the jury determines what the facts are, courts up the chain of review operate on the basis of those facts when making their interpretations. Courts operating above the trial level, or appeals courts, are considering questions of interpretative theory.

The law itself is a combination of the laws passed by legislatures (like Congress), orders by executives (like the President), and regulations by agencies (like the FCC). These are then situated within the Constitutional framework that produced those laws and the tradition of common law. Common law principles provide a number of ideas in general operation, for example the common law would say that no one may be their own judge. In a country that recognizes the common law, the President would not be able to pardon himself as such a function of judgement cannot be done by alone.

Precedent is set by the Supreme Court of the United States (SCOTUS). They typically hear about ninety appeals each year. Cases are selected by the justices, this is called a writ of certiorari. The court only evaluates questions in the cases before it and the court is only the original jurisdiction for a handful of cases involving particularly thorny matters between state governments.[11] When we think of this in the context of the design of state institutions or speculative civics, the Courts in this sense are a trailing institution. They wait to take action.

Within the courts, precedent shapes the interpretation of what the law is. Precedents bind vertically, meaning that a court above makes a determination that should be followed below. This is called Stare Decisis.

Figure 4.2: Government Structure, American Bar Association, 2013

Generally, the Supreme Court is unlikely to overturn an existing precedent, opting instead to distinguish the current situation from that which formed the precedent in the first place. The law as such is the current interpretation of text and precedent. Black and Spriggs have found that depreciation occurs within a twenty-year window, it is not a question of all meaning over time, but how the courts have ruled recently.[12] If a case is not cited within the window, it is not likely that it will be included in the current understanding of the law. It is important to find a lawyer who is active in a domain of practice to understand what the law is actually understood to be in any particular area. A tax attorney may not be up to speed about current cable franchise law, and in areas where the court has been inactive in recent years, results may be far less predictable if half the justices on the court have never ruled in an area. Lawyers use many resources, including reference books like the restatement of law for a particular area to understand what the law actually is.

Why isn’t there a single answer or book with clear directions? There are many situations and different sets of facts. Much like the inability to directly translate the measurements of scientific devices into a policy finding, the analysis of the desirability of a result is very difficult to translate into an abstract legal rule.

Legal research is much like policy research, but tied more to a handful of hermeneutic frameworks. A central tenant of critical legal studies, a sub-discipline appreciated by communication researchers, holds that the law itself is indeterminate, that the text can be interpreted to have a wide variety of meanings. This is not to say that the legal text has no meaning, but that if there is an interpretive question it is more likely that the question will be resolved in favor of the side with more power.

4.1.6.1 Freedom of Expression

The First Amendment to the United States Constitution reads:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

This provides a right for Americans to distribute information, practice religion, and organize movements. If you have been doing your reading, you likely know many reasons why this is good. At the same time, you notice that this does not apply to non-state actors. If anything, private organizations have a first amendment right, in the context of the freedom of the press, to make editorial choices. It is not that we have a clear imperative to speak more, but that communication researchers are very interested in the careful balancing of public and private restraint and editorial judgement.

You may notice that this is a fairly limited right, it does not protect your right to say whatever you want without consequence, prohibit compelled speech, or state speech. There are powerful questions here:

- Should we protect lies?

- What do we do about hurtful speech?

- Should it be acceptable to say true things, even if they are hurtful?

- Does the truth actually win in public debate?

- How does speech intersect with intellectual property?

- What is journalism and should it have special protection?

- How should states respect the expression laws of other states?

- Should well-meaning restraints on communication be struck down if they both stop bad communication and chill good communication?

Freedom of expression is a major topic in communication and is a major theme for the future.

4.1.6.2 Intellectual Property

At this institution, Oregon State, IP law is covered extensively in Introduction to New Media. Intellectual property law includes:

Area

Topics

Protection

Copyright

The protection of texts and authorship

Life of the author + 70 years; statutory protection for all registered texts ($250,000 per violation)

Patent

Inventions that are useful, non-obvious, and correctly filed

Injunction, Damages

Trademark

Derived from the Common Law, protection for the uniqueness of a brand or mark. Non-exclusive.

Injunction, Damages

Trade Secret

Protection against theft of confidential information; criminal law

Criminal penalties and action to redress theft

These regimes of law have distinct sources. Copyright and patent law are derived from article one section eight of the US Constitution. Trademark is a common-law protection. Trade secret is primarily a matter of uniform state law with new Federal law, and is primarily criminal law related to theft.

To be actionable under copyright law one must have made a derivative work for more than a transitory period of time. This means that parts of another work could be included in your work, and despite the originality of your new work you could be sued. The defense in this case is called “fair use” which supposes that there are legitimate uses of copyrighted material such as parody. There is no fair use for patented subject matter. Known as the patent bargain, the idea is that the invention is disclosed for the continuation of the process of science and improvement. Thus, new inventions that include another must pay royalties during the twenty-year period. In some cases, the period may be extended to make invention in a critical area more rewarding.

Patents were common in our area until the decision in CLS v. Alice – where the Supreme Court roundly rejected patents for applications that attempted to computerize existing business processes. It is unlikely that you will see business method patents in our area again. At the same time, you are likely so see patents protecting many technologies that we love. Without artificial scarcity, prices in this sector would drop below viable levels.

4.1.6.3 Open-Source

Many software libraries are open-source, meaning that they are provided to freely use and distribute. These resources like development platforms: Lens Studio, Android Studio, Xcode, Rstudio allow us a great deal of flexibility in making things. Proprietary API systems and the law of the API are likely major factors in our future. The underlying legal structure, at the point of this writing the Computer Abuse and Fraud Act of 1986, is quite unclear. This will be a major area to watch in the future, access controls provide a powerful and dangerous form of private law.

4.1.6.4 International

Data sovereignty is a very hot area right now. Countries around the globe are enacting laws regulating how information is managed in their jurisdiction, with Australia going as far as to require extensive backdoors into systems. There are extensive discussions in Global Media that were seemingly resolved by the techno-utopian vision of the internet that are clearly still very active. A deep understanding of the major frameworks for international relations: realism, neo-realism, liberalism, democratic peace theory, and constructivism will be critical going forward.

4.2 Quantitative

Hadley Wickham and Garrett Grolemund open their handy book R for Data Science, with an important distinction between hypothesis generating and hypothesis confirming quantitative research.[13] For the most part you are likely used to hypothesis testing as the horizon of quantitative research. The elision between quantitative methods and positivism can obscure the potential of other modes of assembling ideas. Hypothesis generating social science can use mathematical techniques and visualization to promote the creation of new interesting questions. In this section I am interested in discussing the foundations of a number of quantitative methods and how they contribute to media studies.

4.2.1 Discrete vs Continuous

Discrete implies that something is a single category. JL Austin’s classic example from speech act theory: one is either married or not married. There is no “sort of” married. There are thus two output variables. Continuous measures have a range of possible outputs, like temperatures. It is possible to render continuous as discrete through binning. We might say temperatures >50 and <50. Now if 50 is a meaningful number, describing a change in state, temperature could become discrete. Boiling versus non-boiling for instance is a meaningful discrete category.

4.2.2 Mathematical Moments

The central processes by which we consider the moments are: central tendency, variance, skewness, and kurtosis.[14] First, we are concerned with the center of the distribution, this can be found with mean or median. Mode is the most common value.

Variance is described by standard deviation of any individual data point from the mean. This is most commonly presented as standard deviation. If we consider a plot of a distribution, the skewness shows how the plot leans, and the kurtosis how sharp the peaks of the distribution.

These ideas allow you to think carefully about where measurement of a thing or a probability. Most of our statistical methods work with manipulations of measurements of central tendency and variance.

4.2.3 Hypothesis Testing

Hypothesis testing is an intrinsically digital way of knowing – it relies on the creation of an if-then construction which can be tested against a reference value. Confusion often comes in the discussion of what is truly tested, meaning the null hypothesis rather than the hypothesis itself. Hypothesis testing research seeks to confirm that something happened, thus rejecting the null hypothesis. Do we necessarily know what happened? No.

The ideas of type one and two error come from a foundational 1928 paper by Jezey Neyman and Egon Pearson:[15]

Accept Null

Reject Null

Null Accurate

Correct – nothing for nothing

Type 1: False Positive

Non-Null Accurate

Type 2: False Negative

Correct – Something for Something

The recurring remark is that there must also be a type three error related to answering the wrong question with the right method. In designing a study to use traditional hypothesis methods we consider the validity and reliability of the tests intended for the hypothesis.

Reliability means: do we get the same result twice when testing the same sample? Would you trust a glucometer that returned very different blood sugar readings on the same vial of blood? The reliability and margin of error for a test is important in designing an experiment. Studies are only as strong as their least reliable method.

Validity refers to the idea that the test in question actually answers the research question. These are divided into internal and external validity. Internal validity is the coherence of the design of the original logical statement, if X then Y. As you remember from the discussion of logical operators and transistors, these processes go in order and only one way. External validity is the ability to see that the result makes sense in the world, that it can be generalized. If a study of 300 undergraduates at a large state university who claim to have never heard of the rapper Drake (one of the most popular figures in popular music at this writing) crosses your desk you know that either A. the data are not legitimate or B. the undergraduates in the study are a highly atypical population and any extrapolation from them is risky at best.

There is one more substantial hurdle: the study needs to be physically possible. Consider the study of food and nutrition. To be ethical, a study must be beneficent, meaning that no one is harmed. It would be unethical to do research that involved intentionally starving people to see what vitamin does. Many social research questions may require surveys that exceed the capacity of the researcher or their funds. Research involving social network data is limited to the data that you can actually extract from that network, Facebook is not cooperative. There are many questions that are not practical to answer.

Consider the following study: This study is intended for 15 year olds. Question one: how many minutes of drinking have you seen in the last year on television? Question two: how many drinks do you have per day?

What do you imagine the results would be? Would you be comfortable with this use of self-report data? If the correlation was positive and the null hypothesis was rejected, would it be prudent to ban representations of beer drinking on television? Should you believe the recollections of the fifteen year olds over the last year or their current use reports?

Correlation, even if you assume the study is sound, does not imply causation. Although only a very silly person would reject the correlation of being hit by a bus and grievous physical injury.

Correlations are generally reported from -1 to 1 (meaning the slope line and relative noisiness) with zero being no relationship. Different coefficients have different properties. There are many other tests including T-Tests that compare the differences between populations. For example: if two comparable populations were exposed to some treatment and then asked for attitude change, the analysis of the variance within and between groups would be a useful measurement. These methods are not a perfect truth machine, they offer important information about central tendency and effect size. The key is understanding the limits of the tests in question and how they align with your research.

4.2.4 Iteration

It is unlikely that a social scientist will dream up the exact single experiment that could make sense of reality. The universe is simply too weird for that. We can expect many studies to be developed that circle in on a possible causal relationship that makes sense of the world. Keep in mind, that this process never ends. Methods become more reliable, our analysis of validity more-fine grained.

As we circle in on the hypothesis confirmation step, it is likely that the visualizations and thought processes will generate even more new hypotheses. As you may have notified in the discussion of policy research there is no quantitative translational moment. Good science is autotelic – it produces more of itself.

What is required is a sense of meta-awareness of what the field has done before. This is why the literature review section of a paper is so important, researchers need to know which hypotheses have already been rejected. This is also why replication is necessary, there need to be important checks both on hypotheses that seem to be true and those that are rejected as false. Most of all, science isn’t magic.

4.2.5 Phacking

The underlying criterion of many of these studies involves the use of the threshold for significance, which in social science is .05 which refers to the likelihood that the null would be rejected in error. The default condition is to reject the null if the level exceeds .05. Phacking refers to the intentional manipulation of a study to arrive at the .05 threshold, which seems both possible and suspiciously likely as so many studies tend to cluster at the key level.[16] Christine Ashwanden argues the problem is not necessarily that there is a great deal of cheating, but that doing really good research is really, really hard:

People often joke about the herky-jerky nature of science and health headlines in the media — coffee is good for you one day, bad the next — but that back and forth embodies exactly what the scientific process is all about. It’s hard to measure the impact of diet on health, Nosek told me. “That variation [in results] occurs because science is hard.” Isolating how coffee affects health requires lots of studies and lots of evidence, and only over time and in the course of many, many studies does the evidence start to narrow to a conclusion that’s defensible. “The variation in findings should not be seen as a threat,” Nosek said. “It means that scientists are working on a hard problem.”[17]

The popular rhetoric of science does little to help in this case. Science is presented as magical and somehow value free offering simple answers to political and ethical problems. In reality, science, like publicity, is a process. Easy answers are not coming.

Hypothesizing after the fact is another version of this problem where many measures are deployed and after some significant result is found a study is reconstructed around it. We should be careful not to delegitimize inductive qualitative strategies. Someone searching for a foothold might test thirty hypotheses. They would then report them and iterate the positive findings to generate more results. Perfection only matters for those with the least at stake.  Exploratory data science is important for future hypothesis generation, at the same time exploratory work should not be passed off as something it isn’t.

When we consider the rhetoric of statistical design, there is a discourse that supposes that the double-blind controlled experiment is the only way to access the truth. If we reduce what we can know to only be that which is tested in this particular manner with a null rejection, there will be no knowledge left. Authority comes in the debunking of what would be meaningful results just as the skeptical game of hermeneutics becomes self-defeating when it hyper-signifies, statistics becomes decadent when the empirical is lost to the purely quantitative.

4.2.6 Bayesian, effect sizing, ANOVA, multiple hypothesis

For the purposes of this paragraph, you do not need to pretend that you care about football, but you need some awareness of what it is.  How do we determine the best college football team in the country? Do we simply count wins and losses? There will be many teams with many wins, are we sure that the farm lads of Iowa play the same level competition as the engineers of MIT? Perhaps we should simply ask some keen sportsball fans? There are no easy answers.

Rankings are hard, especially when there are many teams (around 120) and only ten or eleven games per season. What do you do as the season progresses, how quickly should rankings change from week to week?

In the context of chess competition, a potential solution was proposed by Arpad Elo, which used a slow-moving evaluation of the quality of a player by adding the relative quality of other players to their ranking and dividing.[18] This is an important idea – we can assume that a player who has a very high ranking has likely beaten other very good players and lower ranked players have likely not won such difficult matches. Translating this back into the football example, we can assume that a 9-0 team from the University of Alabama (an institution known for superior football performance) would very likely defeat a team from Concordia College (a team known for its remarkable corncob mascot). Our assessment of Alabama anterior to the test (the game) would be that they were a very good football team, after the win (Concordia has no chance) it would not appreciably increase (we gained little new information). The ranking after would be the posterior measurement.

Instead of seeing the game as a chance to reject the hypothesis that Concordia is better at football than Alabama, the Bayesian method allows us to actually think about the level of information encoded in the game in an intuitive way. Bayesian methods are preferable as they are more easily sized to datasets, allow researchers to think about the world as it is (they are more empirical), and they are concerned with the analysis of variance and effect size. Playoff games would produce far more meaningful information as they involve seeded interactions between teams which we know to be excellent.

In his open letter calling for Bayesian methods in the psychological sciences, John Krushke make the following excellent points:

Some people may have the mistaken impression that the advantages of Bayesian methods are negated by the need to specify a prior distribution. In fact, the use of a prior is both appropriate for rational inference and advantageous in practical applications.

* It is inappropriate not to use a prior. Consider the well-known example of random disease screening. A person is selected at random to be tested for a rare disease. The test result is positive. What is the probability that the person actually has the disease? It turns out, even if the test is highly accurate, the posterior probability of actually having the disease is surprisingly small. Why? Because the prior probability of the disease was so small. Thus, incorporating the prior is crucial for coming to the right conclusion.

* Priors are explicitly specified and must be agreeable to a skeptical scientific audience. Priors are not capricious and cannot be covertly manipulated to predetermine a conclusion. If skeptics disagree with the specification of the prior, then the robustness of the conclusion can be explicitly examined by considering other reasonable priors. In most applications, with moderately large data sets and reasonably informed priors, the conclusions are quite robust.

* Priors are useful for cumulative scientific knowledge and for leveraging inference from small-sample research. As an empirical domain matures, more and more data accumulate regarding particular procedures and outcomes. The accumulated results can inform the priors of subsequent research, yielding greater precision and firmer conclusions.

* When different groups of scientists have differing priors, stemming from differing theories and empirical emphases, then Bayesian methods provide rational means for comparing the conclusions from the different priors.[19]

The advantages of working from a set of priors are clear: when you can debate the nature of the priors the underlying validity of the study can be determined in great detail. Rouder, Haaf, and Aust noted that Bayesian models are already becoming common in communication research.[20] Through a comparison of both approaches to a study of a story about refugees, they show that the null hypothesis would have rejected findings that could move the understanding of political communication forward. The sticking point would be the lack of a clear moment where the BayesFactor would call for the reporting of significance – the authors rightly critique such an assumption. The context of the discussion, more than some arbitrary number should drive the evaluation of the significance level.

Beyond Bayesian developments, methods like ANOVA which can deal with variance between multiple groups, structural equation modeling, and multiple hypothesis approaches are becoming more common. Even the basic techniques of social science are advancing, scientifically. Sometimes better science means a less convenient test of significance and a greater discussion of the qualitative.

4.2.7 The Replication Crisis

If Malcolm Gladwell has taught us anything, it is that counterintuitive results sell books. Among the fields that can produce the most fascinating counterintuitive results is social psychology, where seemingly small things are resented as having systemic effects on beliefs over a long period of time. One of the trickiest representational problems for media research is that many of the effects that we discuss should have some relatively simple experimental evidence.

One of the most commonly cited examples of the lack of reproducibility is priming theory.[21] Priming supposes that exposure to a word or image would unconsciously effect the cognition of a person afterward. For example, one of the most ridiculous examples is the idea that seeing a single image of an American flag can durably increase Republican voting intentions for months to come.[22]

Replication problems are endemic. Artificial intelligence researchers rarely share code or facilitate reproduction of their work.[23] Oncology papers had a 90% failure rate on replication.[24] In basic biological science, error rates for cell line identification are substantial.[25] What does that mean? When scientists apply a chemical to a sample of cells, they may not know what kind of cells they actually are.

Does this mean that science is bad or entirely fraudulent? No. It means that science is hard and the performance of credibility may often imbue unearned ethos for ostensibly scientific results. Researchers need to publish positive results that are interesting to continue their work. Aligning career results with experiment results is a short-circuit that will burn down the house of knowledge.

Some methods are not designed to produce identical results – network methods based on random walks vary based on the point at which the walk started. Unless random seeds in the methods are intentionally fixed, the graphic will not render the same way twice. Topic modeling systems will not assign the same topic number on multiple runs.

4.3 Big Data

MC Elish and danah boyd argue that the discourse of big data depends on “epistemological duct tape.”[26] The underlying methods of big data are quite routine, they are simply bigger. To hold things together, they identify the role of the rhetoric of magic – it becomes something of a strength as the model exists in a special place where normal rules don’t apply. The example of duct tape that is especially pressing is the idea of a “face detector,” as it is not a true detector of faces but a system that detects things that it was told fall into the category, faces.[27] Although this may seem like a trivial distinction it is really quite important as big data presents a difference in degree, not kind. In less abstract terms, if you had the wrong model on the small scale, getting bigger won’t make it right.

4.3.1 Resolving Assumptions

Given that most major models in big data are relatively straight forward, the major challenges in big data are in the area of validity. How do you resolve the collection of datasets of varying ages and qualities? If big data sets continue to affirm what we already know, is it better to made a decision without the cost and time of big data?

As you work with data science methods in communication, it will become clear that the fastest part of the process is executing the code. The much longer part of the process comes in cleaning and structuring the data and the code.

4.3.2 Inductive models

For the most part we begin with constructions that work abductively, we have probabilistic ideas about the truth and we work forward from there. What we can do with big data sets is run correlations and variance analyses on massive datasets. From these explorations, we might begin to see real power in running thousands of trials. These methods are implemented through neural networks, instead of looking for properties and directly testing, a neural network might use a chain of inductive cases to find an outcome. This outcome variable is not itself inductive – you selected by other means.

A key distinction to keep in mind is between supervised and unsupervised modeling. Supervised modeling retains regular human interaction, quality checking and maintaining the dataset on a regular basis. Unsupervised modeling would turn a process loose on a dataset without regular human intervention. There is still a real degree to which this is supervised in as much as the human operator selects which inputs to for the unsupervised process are present. One of the most important inductive moments today is the use of neural networks in unsupervised learning.

The first place you will see this as a student is in the selection of stopwords for natural language processing. In order to find more semantically rich terms, many articles and prepositions will be excluded from the dataset. This supposes that the terms that give sentences structure are unimportant, it seems possible that they are important as well, especially in ngrams. At the same time, as you tokenize and filter the data, you are losing information. If you have already taken the discussion of Bayes to heart, the use of stopwords will not be a huge problem for you. At the same time, you should feel a bit of trepidation as the level of interaction necessary to produce the results would seem to shift the study back toward the qualitative, you would be using less of that magic computational duct tape.

4.3.3 Direct Detection

The promise of big data for media research is direct detection of phenomena.  Self-report data is notoriously unreliable. As one of my colleagues said to me on the street, “you know that anytime you ask an eighth grader if they do weird stuff, they are going to say yes – like lizard people.” This is an important problem. We can’t trust people to tell us real things about themselves. If you have time to work through the layers of fakeness and facework, ethnographic fieldwork can provide robust data with seemingly reversed data.

Social network scraping methods provide one sort of direct data. We can use APIs for the platforms to access Twitter and directly extract swarms of Tweets. Analysis of the Tweets themselves is possible. Unfortunately, this does not allow us to see how exactly the information diffused without response. In public sphere theory, the most common formulations of publics and counter-publics form by the attention of the audience rather than their participation.[28] Continuing with big data along these lines would require even more data about how systems work and some evidence that particular individuals saw something in the first place.

Other methods might involve securing sensor input data, such as information about the location of cell phones and their activity. Selfies could be reverse engineered to look for changes in the medical status of users. This is not the passive big data of magic, but a very active form of big data that calls for the active ingestion of massive datasets.

Self-reports and surveys are messy and inaccurate. Direct detection of the data offers a real transformation in social research. The challenge is getting access to directly detected data. Companies with such data are not often willing to share it, and publics increasingly wish to protect their data.

Topic modeling and sentiment analysis offer researchers the potential for working with directly detected data. Topic modeling produces a probabilistic model of the topics that should be assigned to particular documents within a corpus. This is limited by the relatively narrow confines of the interpretative frame detected. Sentiment analysis may be more limited, functioning primarily on the basis of join and count methods against existing dictionaries. At the same time, direct detection is promising as it offers new possibilities for the analysis of information.

Another form of direct detection comes in the form of bio-foundationalism. This could include the use of sensors, scanners, or other means to monitor the physical state of a person experiencing media. An EEG can allow detection of electrical activity in the brain through a skull cap, saliva samples can measure some chemical levels, blood is better in other cases. Deployed through less invasive methods an fMRI machine (which can detect the utilization of oxygen in the brain) or a PET scan (which detects the utilization of glucose) can give us indications about the use of faculties when exposed to certain kinds of media. These are direction detection methods and they offer a selection of answers to important questions but they are not themselves the entire answer to the problems posed by future communication.

4.4 Artificial Intelligence

Lt. Commander Data is a wonderful television character. As an emotionless, sentient android he offers a fresh take on the human condition. The promise of AI in this televisual sense provides a referent that is both profoundly human, creative, and more than human as a mechanical form. It is Data’s capacity for growth and adaptation that makes him a remarkable and is the magical promise of the system, much as the magic of big data is the prospect that we might ignore epistemology.

Much of what we call AI does not hinge on the development of a synthetic sentient form, but on raw processing power. Ian Bogost argues that the threshold to be called AI has dropped so far that simple string processing methods meet the term, meaning that your command or apple F in word is AI, as he put it:

By protecting the exalted status of its science-fictional orthodoxy, AI can remind creators and users of an essential truth: today’s computer systems are nothing special. They are apparatuses made by people, running software made by people, full of the feats and flaws of both.[29]

Computer vision technologies will allow more effective categorization of media products. Neural networks and other inductive methods will allow increasingly complex simulations of thought.

True artificial intelligence will be special and weird. It will not be quickly and easily added to a discussion board system.

4.5 Digital Humanities

The term digital humanities primarily refer to computationally enhanced literary studies. Consider a claim about the historical evolution of books, let’s say a book by Jimmy Jimmerson had a major impact on syntax as it moved across Italy. Instead of taking the word of scholars seriously that this book really was the hinge point, you can look at a comparison of the texts produced in the region during those times. If the style changes after the introduction of Jimmerson, you know that Jimmerson mattered. What if scholars made a claim that a certain film had an extremely balanced pattern of interaction among characters, would you take their word for it, or find a way to read the interactions in the next to produce a fingerprint of interaction? What would it mean for narrative scholarship if the ostensibly balanced story was, in fact, unbalanced?

What digital humanities typically means, is that contemporary quantitative tools are integrated into the quantification/descriptive side of a study. This does not mean that the qualitative interpretative or critical side of that study are eliminated. It is likely to be the reverse. Scholars now spend a great deal of time struggling with the layers between micro and macro, or invoking incomplete systems theories that skip levels. An economical computer model that can account for the selection of a key exemplar or provide a concise model of diffusion can make our research much more vivacious. This model of humanities scholarship has less hedging and qualification of claims, a better model for iteration of claims.

4.6 Visualization

A central concern for the future of research is the capacity to develop new graphics that can effectively communicate complex concepts. Narrative is underrated: good writing is an amazing vector for information transmission given the synthetic dimension of imagination and sensation. The technology of the novel is superior to the holodeck. Visualization can be misunderstood as the top of the continuum for digital humanities when there are really a number of important, next level, digital methods that intervene into debates and theories. To produce a visual, information must have been cleaned and organized. Visuals may include augmentations of reality, simulations, and abstract graphics. The simulations may be fruitful as they provide evidence of the structure of a system or they may provide this insight through the affective dimension of play.

Of the theorists of the rhetoric of the graphic, Edward Tufte is a clear leader. His theory of graphics rests on a seemingly linguistic break between “nouns” and “verbs” which is then framed by a procedure to reduce all graphics to their minimum necessary content.[30] The point of a great graphic is then to encode enough information for the viewer while maintaining a certain aesthetic sensibility. Some graphics intentionally over encode the information, becoming what Tufte refers to as “confections.”[31]

For the most part, efforts in contemporary information visualization do not pose new methods for graphical production, but increasingly smooth workflows. Data science coursework is really about the development of skills in R, Python, or another language that can facilitate these thoughts.

New immersive systems can provide complex models of the real physics of systems. These new models can provide real insight into the world in ways that were not possible without new displays and interfaces.

What is most important is that you understand that each of these new methods will have a grammar and a rhetoric. The best uses of these new tools will have a balance of aesthetic and conceptual information.

1. C. Mantzavinos, “Hermeneutics,” in The Stanford Encyclopedia of Philosophy, ed. Edward N. Zalta, Winter 2016 (Metaphysics Research Lab, Stanford University, 2016), https://plato.stanford.edu/archives/win2016/entries/hermeneutics/. ↵

2. Bryce, “There Is Nothing Outside of the Text,” Bryce E. Rich (blog), February 27, 2011, http://www.brycerich.com/2011/02/there-is-nothing-outside-of-the-text.html. ↵

3. G.D. Robinsion, “Paul Ricoeur and the Hermeneutics of Suspicion: A Brief Overview and Critique,” Premise 2, no. 8 (September 27, 1995), http://individual.utoronto.ca/bmclean/hermeneutics/ricoeur_suppl/Ricoeur_Herm_of_Suspicion.htm. ↵

4. Rita Felski, “Critique and the Hermeneutics of Suspicion,” M/C Journal 15, no. 1 (November 26, 2011), http://journal.media-culture.org.au/index.php/mcjournal/article/view/431. ↵

5. Megan Geuss, “881 E.T. Cartridges Buried in New Mexico Desert Sell for $107,930.15,” Ars Technica, August 31, 2015, https://arstechnica.com/gaming/2015/08/881-e-t-cartridges-buried-in-new-mexico-desert-sell-for-107930-15/. ↵

6. John Durham Peters, “Calendar, Clock, Tower” (Media in Transition 6, Massachusetts Institute of Technology, 2009), http://web.mit.edu/comm-forum/mit6/papers/peters.pdf. ↵

7. John Durham Peters, Speaking into the Air: A History of the Idea of Communication (University of Chicago Press, 2001); Kittler, Fredrich, Gramaphone, Film, Typewriter, trans. Winthrop-Young, Geoffrey and Wutz, Michael (Stanford: Stanford University Press, 1999). ↵

8. Richard Rumelt, Good Strategy, Bad Strategy (Currency, 2011), https://www.amazon.com/Good-Strategy-Bad-Difference-Matters/dp/0307886239. ↵

9. Pierre Schlag, “Normative and Nowhere to Go,” Stanford Law Review 43, no. 1 (1990): 167–91. ↵

10. Damien Smith Pfister, Networked Media, Networked Rhetorics: Attention and Deliberation in the Early Blogosphere (Penn State Press, 2014). ↵

11. “Jurisdiction: Original, Supreme Court | Federal Judicial Center,” accessed November 20, 2018, https://www.fjc.gov/history/courts/jurisdiction-original-supreme-court. ↵

12. Ryan C. Black and James F. II Spriggs, “The Citation and Depreciation of U.S. Supreme Court Precedent,” Journal of Empirical Legal Studies 10, no. 2 (2013): 325–58. ↵

13. “R for Data Science,” accessed November 20, 2018, https://r4ds.had.co.nz/. ↵

14. Eric W. Weisstein, “Moment,” Text, accessed November 20, 2018, http://mathworld.wolfram.com/Moment.html. ↵

15. J. Neyman and E. S. Pearson, “On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference: Part I,” Biometrika 20A, no. 1/2 (1928): 177, https://doi.org/10.2307/2331945. ↵

16. Christie Aschwanden, “Science Isn’t Broken,” FiveThirtyEight (blog), August 19, 2015, https://fivethirtyeight.com/features/science-isnt-broken/. ↵

17. Ibid ↵

18. His name is literally the name of the function. This is one of many implementations of the process. Nate Silver and Reuben Fischer-Baum, “How We Calculate NBA Elo Ratings,” FiveThirtyEight (blog), May 21, 2015, https://fivethirtyeight.com/features/how-we-calculate-nba-elo-ratings/. ↵

19. John Krushke, “An Open Letter to Editors of Journals, Chairs of Departments, Directors of Funding Programs, Directors of Graduate Training, Reviewers of Grants and Manuscripts, Researchers, Teachers, and Students,” 2010, http://www.indiana.edu/~kruschke/AnOpenLetter.htm. ↵

20. Jeffrey N. Rouder, Julia M. Haaf, and Frederik Aust, “From Theories to Models to Predictions: A Bayesian Model Comparison Approach,” Communication Monographs 85 (December 18, 2017): 41–56, https://doi.org/10.1080/03637751.2017.1394581. ↵

21. Christine R. Harris et al., “Two Failures to Replicate High-Performance-Goal Priming Effects,” PLOS ONE 8, no. 8 (August 16, 2013): e72467, https://doi.org/10.1371/journal.pone.0072467. ↵

22. Travis J. Carter, Melissa J. Ferguson, and Ran R. Hassin, “A Single Exposure to the American Flag Shifts Support toward Republicanism up to 8 Months Later,” Psychological Science 22, no. 8 (August 2011): 1011–18, https://doi.org/10.1177/0956797611414726. ↵

23. Matthew Hutson, “Artificial Intelligence Faces Reproducibility Crisis,” Science 359, no. 6377 (February 16, 2018): 725–26, https://doi.org/10.1126/science.359.6377.725. ↵

24. C. Glenn Begley, “Reproducibility: Six Red Flags for Suspect Work,” Nature 497 (May 22, 2013): 433–34, https://doi.org/10.1038/497433a.b ↵

25. JH Duhnam and P Guthmiller, “Doing Good Science: Authenticating Cell Line Identity,” Corporate Page, Doing Good Science: Authenticating Cell Line Identity, 2012, https://www.promega.com/resources/pubhub/cell-line-authentication-with-strs-2012-update/. ↵

26. M. C. Elish and danah boyd, “Situating Methods in the Magic of Big Data and AI,” Communication Monographs 85, no. 1 (January 2, 2018): 57–80, https://doi.org/10.1080/03637751.2017.1375130. ↵

27. Ibid, 71. ↵

28. Michael, Warner, “Publics and Counter-Publics,” Quarterly Journal of Speech 88 (2002): 413–25. ↵

29. Ian Bogost, “‘Artificial Intelligence’ Has Become Meaningless,” The Atlantic, March 4, 2017, https://www.theatlantic.com/technology/archive/2017/03/what-is-artificial-intelligence/518547/. ↵

30. E. R. Tufte, Envisioning Information. (Graphics Press, 1990), http://psycnet.apa.org/psycinfo/1990-97726-000. ↵

31. Edward R Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative (Cheshire, Conn.: Graphics Press, 2010). ↵

Section 5 – Provocations

5.1 The Truly Alien

We often project ourselves into our fantasies of others. An important assumption of this book has been to look just a few iterations forward, to consider things that are slightly like us or possibilities that are still very much tied to the conditions of the present. Wells was for the most part concerned with war, as we are now. Martinetti wanted to throw off the romantic vision of the country side that held back progress. The MIT futures conference wanted to extend techno-managerial science into the near future.

One of John Durham Peters best critical moves, one taken by many cited in this book, is the identification of artificial simplification. His critique of the Turing test and dialogic are both important. Dialog can’t solve every problem, there are people who you definitely shouldn’t talk to. The Touring test doesn’t speak to artificial intelligence because it only proves that in the highly contextual situation of one text game that a guess has been provided that is satisfying for a very weak set of priors. Any encounter with a contemporary text bot will leave you knowing the critique well, merely identifying the right next turn does not arrive at the quality of intelligence. The Turing test was never enough.

Artificial intelligence is thus sublime, it appears outside the confines of what you already know. Much of what goes by the name machine learning is merely the arrival of adequate computing power to use old methods. We have many methods across the social sciences and humanities to cope with the idea of such a sublime.

When you imagine the truly alien, it is likely that you can’t imagine a form of life without a symbol system. Aliens on television and in the movies, are too close to humanity to be interesting. Nearly all of your attempts to think of these will reduce the other intelligence to semiotic qualities. Among the most truly alien was the Borg of Star Trek. Their deep drive is the annihilation of subjectivity, resolving the problem of the human condition by liquidating the enfolding of the single mind. Even The Borg were lowered in the form of leaders like Locutus (Jean-Luc Pickard himself) or other symbolic avatars who provide some relief from the image of pure, depersonalized drive. The Borg are just that terrifying.

What if the truly alien is more monstrous than you can possibly imagine?

5.2 Torn Enfolding

Existence depends on sensation. Metaphysics is among the few fields not discussed extensively in this text, the closest appearances come in the discussion of simulation. What this book offers for communication instead is a three-fold model of the enfolding: sensation, textuality, and reproduction. It is not simply that we must consider all the sensations that we produce, but the degree to which those sensations are affected by their means of production, envelopment with other sensations, and memories.

The enfolding is an unstable foundation for academic work. There is no consistent ontology, sensation itself is inadequate. From an academic politics perspective, this is one of the greatest liabilities for communication and media studies. We have no territory to defend that is not quicksand. This is also one of our greatest points of appeal for students: there is no basic communication research, we are always concerned with the world and how research on that world is evolving. This is what it truly means to be virtual in this time period: meaningful new thoughts in this field will come from the intersection of possibilities, not from the isolation of existing meaning. To become virtual is to accelerate the process of abstraction.

What humanist critiques so often depend on is the idea of an outside – some sensation that is sublime or beyond the regime of the already known. Leverage gives the humanist something to move the world – this could be in the form of eschatological fantasy in the scene of global warming or in the idea of touch as outside of regular communication. One of the important dimensions of this book has been to situate the entire sensorial envelope. Although it seems difficult to produce satisfying haptic, somatic, and neuro media, it would be folly to exclude them from the regime of simulation. Taste and touch are just more of the enfolding – not the outside.

5.3 The Impossibility of Alterity

Decision is inevitable. The question is what decision will be made. Pure alterity, leaving space open, is an enchanting possibility, but only a meaningful one once a text has been made. It would be too easy to stop there, to see the potential of alterity as an answer to the violence of signification. But that misses the point: reaching some form of stasis in the name of not deciding is still a decision. Alterity as an alternative to signification is an intoxicant to be used in moderation. What if we push these forms to their limits by embracing forms that push toward alterity itself? Can the Dankest memes and the haze Vaporwave overcome the harsh rules of the symbolic? Perhaps the play of signs of poetry is the answer…

Within this is the question of the ethics of the future. Where and how do we make choices about proceeding? The answer is almost always a leap of faith, a passionate choice to engage the world. This is why the opposition of reason through Hegel and passion through Kierkegaard is worth considering with each new cohort of students. Reason is almost always elevated, passion denigrated. The promise of the future is that our new virtual worlds will be rational and collected. But how do you weigh the lost possibilities, all of those worlds not realized by the discourses that actually circulated? Is there a duty to the unlimited future? If so, how do we deal with the multiplicity of intervening actors and probabilities?

5.4 The End of Work

The critical idea in the presentation of liberalism, used here with a small “l” to describe the intersection of labor and capital. The promise of liberalism was that something like effort or merit would replace the traditional role of hierarchy of birth and station. If you were to use your hands, your labor, to modify natural material, you would reap the rewards. This is not a new idea: the concept of liberalism appears in the Book of Isaiah.[1] The capacity of a person to engage in this work and thus become a productive member of society is a central feature of American culture. Liberal culture was not devoted collection of money, but to a number of other values.[2] Capital accumulation was not viewed as an unlimited good, if anything financialization was nearly a form of idolatry.

Why so much build up? Because work is such a critical part of the philosophical fabric of this culture. The recent deformation of liberal work into neoliberalism (exchange economy with no value except prudence) has been difficult for regular people to process. Extreme concentrations of wealth simply have no tie to the hands of the ultra-wealthy worker, if they even continue to fall into that category. On a more threatening level, the arrival of self-driving vehicles and automated factories displace human workers from what would be high paying jobs. Technologists promise that lost long-haul trucking jobs will be replaced by something better. Promises that cannot be enforced carry little weight with workers.

As automation of intellectual tasks continues via topic modeling software and automated customer interfaces the human will be replaced as well. Why see a doctor when the next level WebMD will do? If the legal system were made to conform more to a civil law orientation, a program could replace the judge and jury. There is nowhere to hide from automation.

Increasing corporate profits and stagnant wages are the first glimpse of this future world where success is delinked from the economy of work entirely, more than in the development of a financial system that fully decouples rewards from labor. The threat or robots is not that they will attack us, but that they will cause such social strain that the reaction against those systems will tear us apart.

5.5 The Nightmare of Satisfaction

Replacements for meals and simulated experiences are important. Over time the idea of meal replacement, minimally sufficient food stuffs, or ubiquitous cafeterias have been common. I mean, who really wants to eat? Eating as an activity is pleasurable and social. Once these distractions are removed we will all be so much more efficient.

The reason why replacements and deactivated forms fail is that they are unable to be as satisfying as the originals. If this holds, it makes sense that drive (and desire) will continue. What happens if the simulations finally become completely satisfying and the drive is eliminated? The existential threat of simulation may not be that we are simply on a holodeck, but that we might apply adequate simulated stimulation to ourselves to short circuit the human condition.

But eating is never satisfying. You will be hungry again as your friendly mitochondria will combine glucose and oxygen via the Krebs cycle to power your body. As time winds along your senses will dull and foods that were once too strong will become palatable. Tastes, techniques, and technologies will change and the foods you want will be old and unfashionable. Eating is an easy topic to consider here: you need to do it. There are other desires that are also unquenchable, people chase them for their entire lives, or even build entire structures of desire around the chase itself. Psychoanalytic communication researchers discuss this as the transformation of desire into drive. You are driven to continue doing your behavior, this seems to be the human condition.

But what if you could satisfy your desire? What if a meal replacement could take on a symbolic role that would make it satisfying? One way of thinking about the future would see the real promise is that which eliminates desire and drive or a future where corrected, purified simulations replace dangerous real things. Do you want to live in a world without want?

5.6 Hacking the Mind

Let’s for a moment say that our neural interface technology improves, that we can get information efficiently in and out of the human brain. Advocates of the singularity focus on the idea that this would mean unlimited communication. The idea of singularity is positive as it offers immortality and likely omniscience. At the same time, this promise of immortality hinges on the technology working and not being wiped out by a solar flare. More troubling, this technology would need to encode human consciousness through some kind of relation of logic gates. Security is difficult if not impossible. The gates will be reprogrammed. You will be made to think.

Choices are difficult: this is a world of conflicting incentives, incomplete information, and flawed human minds. Libertarian paternalism supposed that we might “nudge” people toward the right options. Agency was never absolute: when allowed to hypertrophy it becomes ineffective just as much as if it were never there. We are always trying to change behavior, to make things slightly different and better. This fails when people know about it: how dare someone try to shape their choices? No one should manage their desire.

Is immortality without agency really worth it? Perhaps The Borg were right: individuality is the problem.

5.7 Without Plurality

Liberal pluralism has been one of the greatest breakthroughs in political form in the last few centuries. There are two terms: liberal and pluralism. Liberalism has been explored earlier in these provocations: this is the idea that one might work and keep the rewards of their labor. Pluralism implies an acceptance or even celebration of diversity. In a state which is not tied to a nation, pluralism is essential, there is no single group with clear practices, no inside or outside.

The problem with pluralism is that political formations that hinge on resentment can break apart the social whole at fragile seems. Cyber-utopianism was insipid. Theorists assumed that everyone was just like them – that if they had access to computer games and the internet they might become California libertarians. They did not.

Legitimation is at a premium. In the opening of this book, legitimacy was presented on a perpendicular pair of axes. Pluralism seems to depend on a combination meaningful symbolic action and physical supply: the problem is that even the most symbolically legitimate and satiated populations fall prey to the discouragement of the future anterior. We can speak of the future to come as if it is going to happen, the promises of the distant future are deployed to make failure in the present palatable. As these promises build up, discouragement accumulates. The old promises of the group and the nation become an easy fall back.

Movements promising unity offer an alternative to the state as we know it, but at the same time, are fundamentally limited as they lack symbolic legitimation of the group, even if they provide materially. Even when they appear to function, they often leave underlying antagonisms unresolved.[3] This returns to the problem with HG Wells prayer for the future: how do we actually convince all humans to join one world peace, one movement for the future? The Other is holding special enjoyment, you have been excluded, this demands violence, exclusion, and war. Magritte’s painting of war is powerful here: the war looks appealing, the ugly face hidden by the bouquet.

If the threats of apocalypse in the forms of nuclear war, pandemic, hunger, or climate change have not been enough, why should we hope that anything will come along that will change minds?

5.8 Infinite Text

Content comes from somewhere. During the golden age of the social network era, it came from the users. Beyond the phatic posts of lunches and television schedules, were actual expressions of emotion. Those halcyon days when the first political Facebook posts made everyone an activist. Of course, this could not last – the research on boundary coordination is clear that once the rules we use to coordinate our ownership are broken, we rarely feel the same way we did before. Facebook denied for years that it could be broken, it was the unstoppable giant of the social network universe. Well before the 2016 election, the cracks were beginning to show. What the election brought was something more profoundly negative, something deeper, and more hurtful: our source of positive emotional energy and support had been deeply corrupted. Why post when you feel hurt? Why post on a platform that clearly doesn’t share your values? Why post when there is no one there to comment because your content isn’t so interesting that it deserves to be placed between two great advertisements?

We need more content. Where will we get it? What fake news reveals is not the capacity of hackers or propagandists, but that the public is willing to accept even rickety texts as if they were excellent. There is no collective intelligence that will engage in large scale downstream editorial judgement. This is a zombie virtuality – an enfolding that is continuously moving and consuming but only at the most basic level. Every expression refactored into the most abstract and simple quality. Publicity, automated, will learn to produce whatever works. The editorial role in shaping the circulation of the public will be lost. Algorithmic content systems will shuffle about mumbling “clicks, clicks…”

Romantic genius is often presented as the answer to this problem: the greatest content will surely rise to the top, the technological will connect many more with the sublime. Gruesome fare that never would have seen the screen during the broadcast era is now common, this is passed off as a new golden age of television. More short writing and publishing than ever before, riddled with propaganda and anti-social meanings, treated as a profound new literature. Genius won’t save us. The truth is that the vast majority of publicity processes mill over dreck. Many of our thoughts are not worth thinking. Why do we care about agency when so many of our choices are between fetid and fusty? The problem: there is no floor in the preference level of the market, but definitely a lack of demand at the top. Without intervention, the marketplace of ideas becomes a collection of hawker booths for multi-level marketing scams. Information theory holds: as the noise level reaches cacophony, the signal becomes unrecognizable. It isn’t simply that noise fills the channel, but that the infinite text that floods through will be acidic, dissolving the future like an astringent wave of chaos, order, pain, and pleasure in unpasteurized forms.

5.9 Me and My Bots

Privacy is real and important. Becoming visible is risky – you can be seen by others but are subjected to power in new and different ways. The expanded surveillance regime does mark a real change from that which came before – are you comfortable with the idea that you cannot have a sensitive conversation in your living room anymore because your smart television is listening? The true living room, rather than the formal parlor, is your primary space for living. Open concept homes expand and converge the roles of core spaces to create a more expansive space for interiority. Privacy is a tricky concept, it is both essential that one have a place and space of their own, but can serve to entrench existing power relations by excluding the domestic from the political.

Little strips of tape offer user’s protection from their own bots, but this does nothing for the mall kiosk checking in on your location for a central database, or your selfies being read as an index of your liver function. Expanded use of cryptography in the form of public-key messaging and bitcoin offer some escape from the regime of bots. One of the major themes of the Federalist papers was the impossibility of full awareness or communication: this was a form of natural protection against tyranny.[4] Facilitated by heavy duty processing and bot driven detection, it becomes possible to produce a new kind of virtual awareness that flattens the distinction between people and machines. More troubling, we often treat the results of these assemblages of human and machine as machine alone: the results of process are seemingly objective. For how we still require temporally proximal or the operation of war machines, someone needs to confirm the operation. As our metamorphosis continues, it seems likely that we will assume that the machine-human hybrids are just as good as humans.

What is truly striking is not that people need space or that the seeming benefits of mechanization of observation would be seen as outweighing the cost, but that the interplay of people with machines does not seem to make the machine more human, it makes the people more machine like. The hacker ethos extends not only to the manufacture of things, but to the quantization of the self. Perhaps interiority won’t be needed anymore, the ambiguity of the unseen will have passed.

5.10 As Good As It Gets

The promise of the future has been continual improvement. Just beyond our horizon is exponential increase, the jetpack future. When the expectation of that future is revealed as hollow we must either push out the future beyond the horizon or take stock of our wounded attachments to a world that never was. Things have improved, utter poverty has decreased, treatments for many diseases are available, there is more than enough food, but the utopian future has not been realized. Extreme inequality tears at the foundation of plenty in advanced democracies, even basic treatments like Insulin are now scarce as a result of pure avarice, food continues to be allocated on political lines resulting in hunger, and resurgent nationalist movements seem to be arming the world for war.

What if this is as good as it gets?

A future is coming, it seems possible that this will include consumer goods, medicine, and conflict. The law of diminishing returns is real. Too much can be enough. The idea here is relatively straight forward: sometimes we reach the best version of a thing or an idea. The Wall Street Journal reported in a feature on the Future of Everything, that some products (wallets and luggage) had already reached perfection as they were functional, durable, and stylish.[5] The claim to trans-historical style aside, the premise is sound, if one wishes to carry paper money and coins a wallet is likely the best mechanism. The idea of style does offer a sort of depth: this is not a simplistic structure for utility maximization, more pleasure and less pain, if those terms really mean what they think we do. The best wallet has properties that are not simply those that increase or total units of pleasure. This is a good thing. Objects in the view of those interviewed for the story become better with age, they become part of the symbolic lives of the owners, they are really in a sense virtual.

In this chapter of provocations, I have provided an argument against satisfaction, it is also important to consider the profound importance of being satisfied. Psychologists move toward this idea in the context of mindfulness or the contemplative life. Philosophers have the pleasure machine and many other useful thought experiments. Business theorists describe disruption as the replacement of a superior good with one that is merely satisfying at a lower price.

From a perceptual standpoint, the reason why the coming era of more must be so incredible and impressive is that this future must outweigh the pain of the passage of time and mortality. We will not arrive in the utopian future. The day when you realize that your ticket is marked for a station before the end of the line is difficult.

How do you tell people that they should be satisfied, there isn’t more, or that they really shouldn’t have jetpacks? This is where the communication perspective on futurity is so important: our proper object is not the coolness of new gadgets, but the prospect of producing meaning and coordinating action against all the possibly that we have foreclosed, not simply the potential of a limitless future.

We must consider the real possibility that this is as good as it gets, that our success is complicated, in a complex world.

1. “Isaiah 65:22 No Longer Will They Build Houses for Others to Inhabit, nor Plant for Others to Eat. For as Is the Lifetime of a Tree, so Will Be the Days of My People, and My Chosen Ones Will Fully Enjoy the Work of Their Hands.,” BibleHub, accessed November 21, 2018, https://biblehub.com/isaiah/65-22.htm. ↵

2. McCloskey, Diedre, The Bourgeois Virtues(Chicago: University of Chicago Press, 2006). ↵

3. The work on the Hugarian context is especially pressing, as the socialist moment did not resolve underlying antagonisms along lines of racism, sexism, or nationalism which would also be further inflamed by neoliberalism. József Böröcz and Melinda Kovács, Emperor’s New Clothes: Unveiling EU Enlargement(Telford, Shropshire: Central Europe Review, 2001), http://www.rci.rutgers.edu/~eu/Empire.pdf. ↵

4. James Madison, “The Utility of Union as a Safeguard Against Domestic Faction and Insurrection,” Daily Advertiser, November 22, 1787. ↵

5. “The Unimprovable Awards: Indestructible Items to Buy and Hold,” Wall Street Journal, October 10, 2018, sec. Life, https://www.wsj.com/articles/the-unimprovable-awards-indestructible-items-to-buy-and-hold-1539183011. ↵

1

Glossary

This is a glossary of important terms from this book. It is possible that there are important terms that will be on your study guide that will not appear here. These are not intended to be exhaustive entries, you should develop them more as you work through your notes, and the content in the book.

Term

Definition

Abduction

A process of logical reason which proceeds from probabilistic principles, this allows for speculative arguments. Often misidentified as deductive reasoning.

Absorption

The degree to which a surface absorbs energy from a wave bouncing off it.

Abstraction

Resignification of a sign or system of signs to one that is less concrete, more inclusive.

Accelerationism

A loose academic cluster which challenges the conservative orientation of the dominant thread of critical/cultural studies as they rely on a vision of what has been lost. This approach to theorizing inverts value hierarchies and dismisses academic fatalism.

Aesthetics

Principals associated with the appreciation of beauty.

Affordance

The technical capacity of a thing or system, perceived or not.

Agency

The capacity for individuals to make decisions.

Alterity

The consideration of all the meanings that could have been in any given situation, all the things we do not say.

AM

Amplitude modulation. An encoding system for typically analog information on a single wave length.

Amazon

Amazon is a large corporation active in multiple domains of commerce including retail and information services.

Anaglyph

A stereoptical system which uses colored filters to selectively deliver a video signal to each eye of a viewer.

Ancestor Simulation

A conjecture that advanced societies would use their massive computing power to build full, vivacious simulations of ancestor civilizations.

Anti-fragility

A position within a system which may benefit from instability or the destruction of that system. This can also be known as a preference for the exit option in political theory. Generally, we assume that participants in a system are interested in the preservation of that system, inverting this assumption can be useful.

Anti-Vax

A person or public holding the view that vaccination is counterproductive. Typically used as an example of an error in reasoning.

Argument

A special discourse that concerns the attempt to resolve a disagreement. These can either be directed to a dialectical or a rhetorical resolution. In the dialectical mode, a specialized judge will adjudicate the claims at hand. In the rhetorical mode, the discourse is designed to persuade an audience who will determine the truth.

Artificial Intelligence

The idea that a computer system could produce an intelligence that is like a human.

Artificial Scarcity

Most media goods are non-rivalrous yet excludable. This means that there is no reason why an infinite number of copies of the text could not be produced. Often technical means (anti-piracy software) and legal means (copyright or patent law) are used to make a good artificially scarce, supporting the price.

Assortment

Retail stores follow a programmatic logic. The space of a retail store is limited requiring careful planning and decisions between mutually exclusive outcomes.

Autopoesis

Systems which automatically produce text. Autopoetic systems are not autotelic (they don’t call themselves into existence). We can judge the quality of the assumptions in autopoetic systems.

Axiology

The study of systems of value. Values are inevitable, there is no system of thinking that does not find some things to be better, more important, or more valuable than others. This is a critical area as many people fail to critique the basis of their value systems or assume that other people will come to share their perspectives.

Bandwidth

Within a rivalrous pool of a resource the conception of the allocation of a quantity of that whole.

Barbershop Quartet

A group of harmonious singers who produce a range of tones beyond those produced by the four singers.

Bayesian Methods

Bayesian approaches to research differ in that they do not focus on the rejection of the null hypothesis, but the evaluation of the relative probability of an event given the inclusion of new information. This method calls for the rigorous production of an anterior probability or prior, which is then changed, which is expressed as an effect size. The Bayesian moment represents a movement beyond binary conceptions of significance and a more complex discussion of the existing literature.

Beauty

Qualities that are aesthetically satisfying. This is a phenomenological loop.

Bell Labs

Facilities in New Jersey where many critical innovations were developed, primarily a result of the telephone monopoly.

Big Data

A popular term for the use of computers with very large datasets. This does not refer to a new selection of statistical techniques, but the capacity to do things at scale.

Black Swan

An event which is seemingly unpredictable.

Brain Interfaces

Interfaces that would at least read the thoughts of a user if not input thoughts into their brain by means other than the senses.

Brutalism

An approach to design which emphasizes the conditions of possibility for large structures, including untreated concreate and metallic elements.

Californian Ideology

The fiction that California is a special place away from the government where innovation happens. This fantasy disavows the role of the state in producing technology.

Camera

Any technology which records an image. This could include both traditional film, digital, and synthetic cameras.

CCD

Charge-Coupled Device. A digital sensor used when extremely high-quality images are needed, such as in super-cooled cameras.

Central Tendency

The middle of a selection of values, typically taught as mean, median, and mode.

Civil Law

A code driven legal system utilized in Europe. Emphasizes a clearly written law with less interpretive flexibility.

CLS v. Alice

9-0 decision of the United States Supreme Court, majority opinion by Clarence Thomas. The key holding for this case is that simply adding a computer to an existing business process does not make that business process patentable.

CMOS

Complimentary-Metal-Oxide Semiconductor. A digital sensor produced at a lower cost with superior processing design.

CMYK

A color space typically used to assign colors for printing. Cyan, magenta, yellow, and black. Other colors can be included as well, typically orange or green depending on the system. More simplistic color spaces could be produced that rely on a pantone.

Cochlea

The part of the inner ear where sound waves are translated into a neural signal by hair cells.

Codec

The method for encoding video information for reproduction as a display or further editing.

Codes

The condensation of many signs in relationship. Signs within codes continuously redefine each other.

Color

The resulting perception of sensations of the color of an object by a photoreceptor. Philosophy has many rich discussions of color, unfortunately these are less interesting for communication.

Coloring Book Hypothesis

The incorrect assumption that the brain primarily renders the world in black and white (like a coloring book) which then is filled in with color. Simultaneous processing is a better model, this also explains several optical illusions.

Common Carrier

Carriers who provide a basic transportation or communication service on a fixed fee schedule regardless of other properties of the thing to be conveyed.

Common Law

The hermeneutic tradition that poses a number of legal procedures that have been in operation for several centuries.

Complex

A system with many parts.

Complicated

A design or experience which either does not align with the expectations of the user or which tend to be associated with negative emotions.

Conditions of Possibility

Those facts which must be true to produce the present.

Cones

A set of photoreceptors which typically function in higher light conditions producing a wider gamut of colors.

Constitutive

A term which organizes the discourse around a particular thing.

Conspiracy

A discourse featuring an ostensibly repressed truth which has been dampened by a villain. Once the block of the villain is removed the new information will change society.

Continuity

The idea that the past continues. The opposite of Rupture.

Continuous

On the smallest level, a number is always immediately next to another number on a line, the difference between these numbers is only decided at the level of the problem. Similar to the Arrow paradox: if an arrow at any given moment is half as far from the target as it was at the prior moment, will it ever arrive?

Convergence

The transformative potential of the intentional connection of systems or ideas.

Copyright

A legal regime protecting the expression of authors. Authors in this case are broadly defined. Copyright protects and expression and possible derivative works, not the idea itself.

Counter Publics

Much like publics, these are imaginary alignments of people who suppose they are being spoken to, but that these are not directly tied to the primary alignment of the social structure described.

Creative Destruction

Described by Schumpeter: economic systems are zero-sum, often the destruction of one form opens space for a the new.

Critical Legal Studies

An approach to the study of law which emphasizes the discursive element of the law, particular the role of power, rhetoric, and identity in the formulation of the law rather than a normative hermeneutic approach which sees the law as the result of a seemingly scientific process.

CSS

Cascading Style Sheets. An approach to conveying style information for online document.

Cuisine

A cultural code for the process where food is made delicious.

Cutaneous Rabbit

A sensorial illusion that a touch is moving across rather than jumping from point to point.

Decibels

A logarithmic scale for the measurement if sound volume.

Deduction

A process of logical reason which proceeds from explicit principles.

Deep Fakes

Simulated video experiences produced by neural nets that overcome the uncanny valley.

Depth Perception

The degree to which the visual system can produce a rendering of the world which accurately accounts for the distance between objects.

Derrida

Jacques Derrida, literary theorist. Developed an approach to deconstruction which emphasized the resolution of false binaries. This method could put key terms under erasure, resignify them, or emphasize the degree to which the terms of a binary depend on each other. The additional meaning that is concealed in the construction of the binary is positioned as alterity. IT An important quote from Derrida “there is nothing outside the text,” is often misread as a form of solipsism, when the implication of the statement is that any calculation of reality itself depends on discursive assumptions which should also be subject to critique. There is no simply disavowal of discursive critique through the invocation of the “real world.” This approach to reading social science as literature has been quite influential.

Design

The understanding of a plan for the construction of a thing (conceived broadly).

Desire

A feeling of wanting. Foundational to the human condition.

Desire Lines

Lines where the grass has been worn away from a lawn by people walking where they want, rather than on a designated path.

Dialogic

The quality of communication which appreciates the rich potential of dialog.

Diffusion

The flow or spread of an idea or light.

Digital Humanities

The disciplinary term used to describe research in the humanities that utilizes contemporary computational tools.

Digital Sublime

From Vincent Mosco, the idea that a new technology, in this case digital communication, presents a radical new potential for change.

Digital Sublime

The idea that new technology, in this case quantizing processors, represent a rupture in historical continuity.

Discipline

An academic construction that isolates key epistemic and ontological features of a domain of study. Psychology, for example, would be concerned with the psyche of the individual. All academic disciplines utilize founding narratives and other mechanisms to maintain their coherence. Often these moments depend on constitutive exclusion, they define the code of what the discipline is by what it is not.

Discrete

Objects for analysis that are not smooth like a double, but a series of distinct values or statements.

Dramatism

An approach to understanding communication which supposes that human understanding is structured like a narrative. The proper tool for understanding communication are those of storytelling and drama, like events and characters. An example includes Burkes Pentad.

Echo

A reflection of a sound that includes the attack from that sound.

Electromagnetic Spectrum

The range of possible wave lengths for electromagnetic energy.

ELOChess

Ranking of players in games can be hard. ELOChess provides a continuous means for integrating new information about chess players.

Emergence

The transformative potential of novelty that emerges from the proximity of systems or ideas.

Emotional Design

A specific approach to understanding design which attempts to produce a specific emotional reaction.

Engineering

The systematic practice of designing systems or things.

Entropy

The tendency for systems toward disorder. In information theory, this is the tendency for noise to overwhelm the signal.

Envelope

The presentation of an entire sound from inception to end.

Episteme

The structures of thinking which must be present to make sense of what has appeared.

Epistemology

The study of systems of thinking. Epistemological thinking accounts for the ways that ideas are assembled.

Ethnography

The practice of writing about culture. As an approach to social science, ethnography refers both to the experience of the researcher in the field, the process for organizing recollections, and the ways in which those recollections are presented in an authoritative context.

Excludable

A good which can be effectively controlled, limiting consumption.

Existential Risk

Threats to the continued existence of human life.

Factual System of Signs

The propensity for signs to be taken as facts in the consideration of other signs.

False Consciousness

The idea that some message or idea has induced the reliance on wrong thoughts or information.

Fashion

A trend in style.

Federalism

An organizational scheme which uses layers with split responsibilities to facilitate creative decision making

Field notes

The intermediate documents produced in ethnographic fieldwork. Intermediate documents are critical in many methods, in ethnography a core consideration is the production of and organization of notes.

Film

A strip of material coated in an emulsion which registers light.

Flow State

A psychological state where an individual is engaged in strenuous cognitive effort but enjoys such effort to the degree that it seems easy.

FM

Frequency modulation. An encoding system for information for information on multiple wavelengths, not analog.

Foucault

Michel Foucault, social theorist. Developed an approach to the theory of power where the discourses of academic theories are understood to be immediately shaped by power. Foucault’s objects of critique include most social institutions. He was not a Marxist, as the theory of power and meaning in materialist work is inherently structuralist. Foucault’s post-structuralism enables a wide range of new social theory which understands the reflexive nature of power and discourse. Power and conflict are not monolithic, but a multiplicity of practices both positive and negative.

Foxes

A character from Isiah Berlin describing an intellectual character who is widely interested and willing to consider multiple disciplines

Freedom of Expression

The idea that discourse should, by rule, be permitted. This argument was developed by Louis Brandeis in his concurrence in Whitney v. California with the key points being: judgment of speech is good, but that judgement should happen through social means, not through police power. The underlying theory of the public sphere inherent in Freedom of Expression supposes that some rhetorical or dialectical means will lead to progress if discursive processes are allowed to unfold.

Games

Communication process that is game like. If this seems circular, you are right. Most definitions of game refer to the amusement or pleasure. What matters for us in gaming, is the understanding of the reflexive, goal oriented, rule driven structure of things that would-be game like.

Genius

The romantic idea that an individual has a gift enabling novelty. The assignment of the quality of genius has substantial implications as it tends to wrap features of structures into the individual personality.

Geography

The physical placement of things space.

Glutamates

Chemical compounds in food known to have a particular savory flavor.

Graphical User Interface

Most computer users perceive of the computer environment as a world of images and objects, rather than text. Graphical interfaces are compelling abstractions that we rely on to understand the abstraction of information worlds.

Greenhouse gasses

Carbon Dioxide and a number of other gasses are likely causing the earth’s temperature to increase. Although the temperature has fluctuated before, human society is not well adapted to deal with these changes at this speed. Many researchers see this as an important issue for the idea of the future as it would be an intractable driver of change. It is also interesting because of the cognitive and communication issues associated with long change processes.

Grocery Stores

Grocery stores are a particular kind of retail facility that stocks a large number of food items. Grocery is an especially interesting industry as these stores have unique logistical challenges that tend to preclude centralization.

Gruen

Victor Gruen was the key figure in the development of the modern introverted shopping mall. The mall as a civic technology was intended to reproduce the qualities of Vienna on the wind swept, frigid plains of Minnesota.

Haptics

The study of the perception of touch as media.

Hearing

The capacity of a system, especially a human, to perceive sounds.

Heat

Energy, especially excess energy.

Hedgehogs

A character from Isiah Berlin describing an intellectual character who is primarily devoted to the exploration of a single intellectual burrow.

Hedonomics

An approach to design which emphasizes user satisfaction, specifically as user pleasure.

Hermeneutics

The study of the production of a text and the particular questions which help a critic understand a particular text.

Hero’s Journey

A formalist approach to understanding narrative which sees the progress of a character through a number of important transformations

Hologram

An image that exists within a medium which refracts laser light to produce a seemingly three-dimensional image.

HTML

Hypertext Mark-up Language. A simplistic language for the presentation of online document content.

Human Computer Interaction

The study of the features of user interaction with the abstract system of the computer.

Humanities

The study of the human condition, especially through the texts produced to explore that meaning.

Hypothesis Testing

A model for the progression of science which supposes that the default condition of a hypothesis is null (that nothing happened), research progresses when the null hypothesis is rejected by a procedure where a test is conducted with less than a .05 chance of randomness indicates that something did happen. Rejection of the null does not confirm the hypothesis, it merely indicates the rejection of the null.

Icon

A sign that looks like the thing. The signifier literally looks like the signified.

Ideograph

A special kind of sign which has taken on special qualities that organize publics with minimal positive content. The most important of these is the idea of “the people.”

Ideology

A discourse which handles the cognitive dissonance of holding multiple seemingly contradictory ideas at one time. All people are ideological.

IFF

International Flavors and Fragrances, an organization which produces standardized accounts of the tastes and smells.

Immersion

The perception of a person that they are “in” a story.

Index

A sign that likely is the causal result of another thing. Smoke is a sign of fire, it is an index of the action of fire.

Induction

A process of logical reason which proceeds to derive principles from a large number of individual cases.

Information

Information refers to data that is meaningful. Information theory is derived from the work of Shannon and Weaver, which hinges on the ratio of signal to noise in any given channel.

Infrastructures

Infrastructural research focuses on the ways in which communication systems work in a concrete sense. These can include platforms for distribution, logistics, and many other seemingly mundane details.

Interactivity

The property of a system to respond to user input.

Internet

A term describing networks of computer systems which provide ubiquitous connectivity and storage

Interpretant

The idea of the sign in the mind of the receiver, not the sender. For the purposes of semiotic analysis, meaning in the world outside the sender is much more important than he sender’s intent.

Intertextuality

Texts refer to other texts. To some degree all texts refer to each other, this is a key idea in the idea of the Death of the Author.

Iteration

The idea of repeating a process with feedback to improve that process. In research design, this can refer to a study that includes multiple smaller studies which sequentially improve.

JavaScript

A programming language commonly used for developing interactive websites.

Jet Pack

The idea of the jet pack is used as a stand-in for any comical future technology like a hoverboard or a mars colony.

Justification

A model of argument which assumes that the evaluation of an argument depends on the values of that situation. The specific fallacies or rules in that argumentative situation follow from the alignment of the type of argument presented in the world in which that argument operates.

Kairos

The experience of time as a point. Examples include: religious celebrations (Lent), States of Emergency (war), cultural forms (March Madness), affective modulations (summer).

Keyboards

A device which allows the user to quickly and efficiently input text into a computer system.

Lacan

Jacques Lacan, psychoanalyst. Developed an approach to psychoanalysis which complemented Freud. The core structure for Lacan was the Real – the experience of presymbolic wholeness before language, many stand-ins are presented for the real, either through sublimation or transference. Theories involving stand-ins are important for contemporary ideology theory. From clinical practice, Lacan appreciated that the ongoing ability to produce new signs was an indicator of progress for a patient, who was thus neurotic rather than psychotic. Lacan was also critical of the analyst’s desire, the tendency for therapy to focus on the production of what would seem to be prosocial outcomes at the expense of the individuality of the patient. Lacan’s reinterpretation of the death drive is useful for a number of theories as they account for seemingly bizarre outcomes.

Langue

A term from semiotics referring to the official version of language as used in a society.

Legitimation

All social systems must be legitimate, according to DeLanda this process has symbolic and practical elements. Systems must maintain code systems that are coherent, while continuing to provide for the physical welfare of the people. Failing on either regard leads to crisis.

Lenticular Overlay

A filter that is placed of an image to produce an illusion of depth or motion.

Light

A special subset of electromagnetic radiation which can be seen with the eye.

Logic Gates

Collections of transistors which function as basic logical operators.

Ludology

The study of game systems which emphasizes the difference between games and other forms of media, typically uses a psychologizing rhetoric of experience.

Masters of Suspicion

Ricoeur’s term referring to Freud, Marx, and Nietzsche; typically used in lower level courses to introduce the idea of language not necessarily being a transparent record of reality. It is important to understand that these theorists have very different approaches.

Melodrama

Theorized extensively by Joni Anker. A story with exaggerated characters, often dismissed as an ostensibly feminine form, critical for understanding of public culture.

Methodology

A Method includes: ontology, epistemology, axiology, and rhetoric. Once established, a method will produce a particular sort of knowledge.

Microphone

A device which converts sound waves into a usable electrical signal.

Moments

Measurements of the shape of a function, particularly mean and variance.

Mother of all Demos

An event in 1968 where Douglas Engelbart presented all major elements of contemporary computing in a single system.

Mouse

A pointing device which allows a user to interact with the computer environment.

Move Fast and Break Things

An early slogan at Facebook which signaled their agile culture. This was abandoned when Facebook reached maturity with stability as a key infrastructural value driver.

NAND/NOR

A logic gate which reads as true if both input are not true (NAND), or a gate which only reads as true if both are active (NOR).

Narrative

A story which typically includes characters, details, events, and some sense of progression.

Narratology

The study of game systems which emphasizes the similarity of those experiences to books or films, typically uses a humanistic/hermeneutic rhetoric of experience.

Net Neutrality

A policy for the management of the scarce resources of the internet which sees traffic as a common pool.

Neural Networks

A computer process designed to simulate neural processing structures.

ngrams

The idea that a collection of words, (two are a bigram) can be the basis for computational processing rather than individual words.

Normcore

A possibly fictional fashion movement that emphasized what would be normal or less desirable. These movements recur with some regularity.

Ontology

The study of the state of being. Philosophically inflected researchers tend to use this term to refer to the study of the metaphysical level of reality, engineers and others tend to see this as the description of the states of being with in a particular episteme.

Orthogonal Graphics Editors

Visual perception hinges on the perception of edges, textures, and occlusions. The primary means by which images are produced are orthogonal editing platforms which present the image as a result of a composite z-axis, typically presented as “layers.”

P-Hacking

A description for practice where researchers manipulate their methods to arrive at a study where the P value is less than .05, thus calling for the rejection of the null hypothesis and generally publishable results.

Pantone

A color produced by the Pantone corporation. Color research related to the development of Pantones can be quite valuable. Individual Pantones can be reliably reproduced.

Parole

A term from semiotics referring to the actual spoken language of a society or common use.

Patent

A legal regime protecting an invention. The full details of that system are disclosed in exchange for licensing rights. This is known as the patent bargain: in exchange for legal protection, science is fully disclosed.

Photorealism

The quality of a synthetic image that is like a photograph.

PHP

Hypertext Preprocessor. A language providing a slate of specific tools to facilitate server side interactivity on a website.

Pick-up Pattern

The specific area around a microphone where that device tends to optimally “hear” sound. Knowing where your microphone records sound is a key skill for a sound recordist.

Pierce

Charles Sanders Pierce, American, key theorist in semiotics, especially the triadic sign.

Polarizing

In optics, a filter which organizes light.

Polarizing Stereo

A stereoptical system which uses clear filters (polarizing) to selectively deliver a video signal to each eye of a viewer.

Policy Claims

A claim that a policy is normatively desirable.

Pragmatic

The level of communication where action is coordinated.

Precedent

The base hermeneutic concept of the common-law tradition, courts interpret the law in ways similar to other courts in the same system. This does not mean that all precedents are considered equally, but that those precedents that are well regarded in that system are considered.

Price

Is a rationalized value assigned to an item? Before the advent of department stores, prices were far more fluid.

Probability

A measurement of the potential for an event to take place.

Problem Finding

The practice of examining a system or a presented problem to find what would be a more vivacious problem.

Problem Solving

The practice of producing a matrix of possible alternatives to solve a problem and the ongoing cycle of redefinitions of the problem until it is solved.

Proprioception

The experience of the combination of multiple sensations, especially touch, hearing, and vision to understand the position of the body in space.

Public Policy

An academic discipline related to the formulation of normative rules or laws that would improve the general welfare

Publics

These are imaginary alignments of people who would suppose that they are being spoken to.

Quantum Computing

Computing systems which apply methods for the resolution of logical operators other than transistors.

Radio

A special subset of electromagnetic radiation which is often modulated to carry information.

Radio Transmitters

Devices, typically vacuum tubes, which amplify signals for distant reception.

Rare Earths

A selection of relatively common metals which are useful for advanced magnetic and electronic applications.

Reduction to the Absurd

A process of logical reason which proceeds to reject a potential explanation by arriving at a point where an obviously false premise must be true to proceed.

Refactoring

The process where code is rewritten to make it more functional.

Reflection

An energy wave bouncing off a surface.

Regulation

A subset of policies produced by executive agencies, rather than by deliberative bodies (legislatures) or courts.

Relational Dialectics

An approach to understanding relationships is negotiated between positions and identifies. This offer an alternative to psychologizing definitions which suppose that communication behaviors are keyed on psychological qualities or mechanistic stimulus responses.

Render

A process where a signal is reproduced in a visible form.

Replication

The idea that study in social science should be possible to replicate given the described methods in an article. In humanistic research, this means that the researchers description of the field would be perceived with some continuity by others in the same situation.

Resolution

The pixel density of a display.

Retina

A sense organ in the rear of the eye where photoreceptors transduce light for transmission to the brain via the optic nerve.

Reverb

A reflection of a sound that does not include the attack from that sound.

RGB

A color space typically used with light, the mixture of these colors of light appears white.

Rhetoric

Refers to the study of a system of codes which are used to co-produce meaning, coordinate action, and imagine possible alternatives.

Rivalrous

A good which can only be consumed by one person at a time.

Rods

A set of photoreceptors which typically function in low light.

Romanticism

A discourse which emphasizes the genius individual and their perception.

Rupture

The idea that the past has broken. The opposite of Continuity.

Satellites

Devices in earth orbit which receive and transmit signals.

Saussure

Ferdinand de Saussure, French, key theorist in semiotics, especially the reflexive circulation of the signified and the signifier.

Scalable

The idea that an idea or system could be expanded to apply in many cases.

Scarcity

The property of a limit or a limited supply.

Sears

Sears-Roebuck was a large retail chain notable for developing commercial infrastructure. The Sears Catalog was the original everything store, access to products outside the of the local supply chain was transformative. Off-shoots of Sears revolutionized many industries.

Semantic

The level of communication where contours of meaning are developed.

Shirley Cards

Standardized cards used to calibrate cameras.

Signifier

The label (expansive idea) for a thing or sign. When used in design theory, this refers to the practice of explicitly labeling a part.

Signified

The thing that has been labeled, this could also include other signs.

Signs

The key unit of semiotics, a fusion combination of elements.

Signs are Evolutionary

Signs are continuously changing. The individual producing the sign must consider all the potential signifiers and the way that they are tied to the interpretant.

Simple

A design which aligns with the psychological expectations of the user.

Simulacrum

A simulation that has taken on qualities more realistic than the original.

Simulation

The production of stand-ins for things.

Singularity

It is possible that given adequate processing power, storage, and neural interfaces that all humans could become part of a single artificial intelligence.

Smell

A sensory modality primarily tied to aroma, perceived through the inhalation of gasses, particles, or other vapors.

Social Sciences

A process for understanding the human condition through the processes of natural science.

Speakers

Devices which produce sound waves from an electrical signal.

Speculative

An approach to thinking that compares multiple future narratives.

Speed of Light

The speed at which light moves, 186,000 miles per second.

Stereo-Optical

An image which is designed to provide the illusion of depth through means of isolated representation to each eye.

Stopwords

A list of words to be removed from a corpus for analysis.

Supervised vs Unsupervised Machine Learning

The degree to which a user controls the progression of a machine process.

Syllogism

A simplified statement which can be used for basic logic.

Symbol

A sign that is entirely synthetic. Almost all forms of writing in the west are symbolic.

Synecdoche

A part describing a whole or a whole describing a part. This is one of a number of important rhetorical forms.

Taste

A sensory modality primarily tied to the mouth and the sensation of flavors.

Taste Buds

Sensory organs primarily found on the tongue, capable of detecting a range of chemicals. Other touch receptors in the mouth also process relevant information for the sensation known as taste.

Taubman

Alfred Taubman was a designer of shopping malls in the United States, known to be careful with sensations.

Technological Determinism

A critique of research that determines that a change in technology was likely the controlling force in a historical shift.

Temporality

The subjective experience of time.

Time

Describes both chronos (the literally description of when something is) and Kairos (the description of now as a moment or point). Both are important for understanding how the future is shaped.

Time Zones

A form of political cartography which would seem to stabilize local times apart from solar time to facilitate control.

Time-Axis Manipulation

4-D media require the progression of time. The manipulation of a time sequence offers a considerable range of possibilities not possible in the traditional theater, book, or painting.

Transcendental

The idea that a feeling or thought can exceed context.

Transistors

Semiconducting assemblages which replace many functions of vacuum tubes and allow the creation of solid state logic systems. Bipolar transistors allow the amplification of a signal introduced into a circuit. MOFSET transistors allows the field effect to control a second circuit as a switch.

Uncertainty Mechanism

All games depend on some mechanism where uncertainty is produced, this can include the player.

Undersea Cables

Necessary infrastructure for the connection of various Internet systems around the world.

Usability

A specific approach to understanding design which highlights task analysis and user satisfaction.

Video

The method for recording a stream of images on a magnetic tape

Virtual

A condition typically referring to a simulation, either by a system or by the mind.

Vision

The perception of light.

Visualization

The production of visual models for complex systems. This is one of the primary digital methods along with: search, navigation, augmentation. simulation, and play. At times visualization is used as a synecdoche for all digital methods.

Walter Cronkite

An avatar for the Trusted news person of the mid-twentieth century.

WikiLeaks

An organization lead by Julian Assange which spreads ostensibly secret information.

World Systems

An approach to thinking about the combination of logistics and states.

2

Creative Commons License

This work is licensed by Daniel Faltesek (©2019) under a
Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC)

You are free to:
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if
changes were made. You may do so in any reasonable manner, but not in any way that suggests
the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

3

Recommended Citations

APA outline:

Source from website:

- (Full last name, first initial of first name).  (Date of publication). Title of source. Retrieved from https://www.someaddress.com/full/url/

Source from print:

- (Full last name, first initial of first name).  (Date of publication). Title of source. Title of container (larger whole that the source is in, i.e. a chapter in a book), volume number, page numbers.

Examples

If retrieving from a webpage:

- Berndt, T. J. (2002). Friendship quality and social development. Retrieved from insert link.

If retrieving from a book:

- Berndt, T. J. (2002). Friendship quality and social development. Current Directions in Psychological Science, 11, 7-10.

MLA outline:

Author (last, first name). Title of source. Title of container (larger whole that the source is in, i.e. a chapter in a book), Other contributors, Version, Number, Publisher, Publication Date, Location (page numbers).

Examples

- Bagchi, Alaknanda. “Conflicting Nationalisms: The Voice of the Subaltern in Mahasweta Devi’s Bashai Tudu.” Tulsa Studies in Women’s Literature, vol. 15, no. 1, 1996, pp. 41-50.

- Said, Edward W. Culture and Imperialism. Knopf, 1994.

Chicago outline:

Source from website:

- Lastname, Firstname. “Title of Web Page.” Name of Website. Publishing organization, publication or revision date if available. Access date if no other date is available. URL .

Source from print:

- Last name, First name. Title of Book. Place of publication: Publisher, Year of publication.

Examples

- Davidson, Donald, Essays on Actions and Events. Oxford: Clarendon, 2001. https://bibliotecamathom.files.wordpress.com/2012/10/essays-on-actions-and-events.pdf.

- Kerouac, Jack. The Dharma Bums. New York: Viking Press, 1958.

4

Versioning

This page provides a record of changes made to this guide. Each set of edits is acknowledged with a 0.01 increase in the version number. The exported files for this toolkit reflect the most recent version.

If you find an error in this text, please fill out the form at bit.ly/33cz3Q1

Version

Date

Change Made

Location in text

0.1

MM/DD/YYYY