Some enchanted evening…You may see a stranger…
Across a crowded Zoom
Someone may be laughing…You may hear her laughing…
Across a crowded Zoom
Well, not so much. And therein lies our lack of enchantment with current teleconferencing and telepresence tools.
Many will recall those lines from the hit musical and movie South Pacific, though it was a room that was crowded. (It is well worth a listen, whatever your taste. Feel free to play it as you read on.)
The song goes on to say,
Who can explain it? / Who can tell you why?
It is instructive to consider how such serendipitous seeing, hearing, and feeling is missing from Zoom — and how we might add at least some of that back in to such systems. Many feel “Zoom fatigue.” In this year of isolation we are gaining insight into how limited these systems are, and many are working to remedy that, including with virtual and augmented reality.
I think these lyrics encapsulate the fundamental challenge: How can digital group communications become “high-fidelity” enough to convey the ever-so-subtle signals that are cut off in our current virtual realities? Those are the driving signals of mirror neuron bonding in social interaction – not only enchantment, but the lifeblood of human culture.
Because our mutual responsiveness to social signals has evolved over millions of years of increasingly sophisticated interaction, we will not easily learn to be expert social navigators in virtual space. Many efforts to improve virtual interaction attempt to create new analogs to what we do in real settings, but are too dissimilar to flow naturally in our unconscious awareness and behavior. We need to find ways to restore more of that nuanced energy flow of real social interaction.
The song suggests how deftly we can find our way in groups. Think of a cocktail party or large business meeting. We may weave through an emergent network of ever-changing groupings, just gaze fleetingly at a speaker, or around a meeting room. Whatever group we may be in at a given moment, we can still see a stranger across a crowded room or ”hear her laughing”” — even far away. Who can explain what draws our attention? But clearly it involves high-fidelity social signals.
Seeing is very selective. Lack of eye contact is perhaps the most powerful impediment to making virtual human connections. Whether one to one, or in groups of any size, eye contact is how we most clearly signal interest and lock in on each other. The other may meet our gaze and move on, or linger momentarily (or more) to signal interest. Whether with the mosaic of videos in Zoom — or the avatars and virtual spaces of more advanced systems — eye contact would be the easiest and most natural form of signaling.
Perhaps future group communications systems can use gaze tracking to see where you are looking, match that to others in the virtual space, and modify the image seen by the other to simulate eye contact from you – and vice versa. Maybe modifying the eye image to seem to be looking right at you (and not at the screen), or with some not quite so natural a signal, but still easy to adapt to and perhaps more noticeable, like brightening the iris. (Apple’s FaceTime has attempted this with a recent feature that makes it appear that you have eye contact with the person you’re talking to even when you’re looking elsewhere, but there have been complaints that its constancy is disconcerting.) Other improvements in visual fidelity are also needed to convey the microexpressions that drive our mirror neurons to dance. Maybe we can have cameras positioned behind the center of our screens, somehow, to avoid that awkward oblique gaze so characteristic of Zoom and its ilk.
As the song suggests, hearing is also very selective. Without effort, we hear many things across a crowded room: laughter, other sounds of emotion, names, snippets of conversation. But across a crowded Zoom, we are in little boxes that are soundproof most of the time. What if we could hear and distinguish someone laughing across a crowded Zoom, isolating their distance and direction in virtual space, well enough to lock into whatever sounds resonate for us?
Perhaps high-fidelity, always-on, surround sound might help. Perhaps systems will learn how to do some of the selective filtering for us, to help us hear most clearly the peripheral sounds that seem most likely to be important to us. And of course it would be better and maybe easier in virtual reality, but then you’ve got the video verisimilitude challenge to deal with. (Zoom’s relative success in this year of video group chats can, in part, be attributed to its prioritizing audio so that is the last thing to fail when a participant has a weak connection.)
What is lost now is the energy among people. The seeing and the hearing. The glances and microexpressions that convey subtle signals that tunnel directly from subliminal to mirroring and awareness.
We need to regain the fidelity of these essential signals so we can restore the serendipity of group interactions. Not only having one or a few direct conversational partners, but awareness of being within a larger grouping of others who might also be important to us. Discovering there is someone we want to engage with in a side conversation or just gesture to, or a group we want to go over to and join. The dance of locking in and veering away.
Regaining all or even most of that will be very hard. But just critical portions of it, such as eye contact and less choppy sound might make a transformative difference in the power and ease of virtual person-to-person interaction. We will all keep suffering Zoom fatigue until we can do better. We should all do what we can to be aware of and compensate for the limitations of our communications tools. For the researchers, developers, and entrepreneurs out there, fixing them is where the money is.
——-
Richard Reisman’s book, FairPay: Adaptively Win-Win Customer Relationships, and blog, FairPayZone.com, introduce a new customer-value-first revenue strategy for digital commerce that was described in Harvard Business Review. He is President and founder of Teleshuttle Corporation, and has managed and consulted for businesses of all sizes, developed pioneering online services, and holds over 50 patents licensed by over 200 companies.
