The AI of Interpreting Emotion With Duane Varan

How do you really feel? This might seem to be an unusually personal and direct way to start this post but bear with me. You see, my recent guest, Dr Duane Varan, specializes in answering this seemingly innocuous question using cutting-edge technology.

When I first heard about how AI and bio-feedback is being used to identify how online users are feeling, this development did not fill me with delight. Like many people, I’ve been concerned with the ethics of invasive tech manipulating human emotions for commercial gain. However, looking deeper at the potential of this technology, my concerns are now tempered with optimism.

Dr. Varan is CEO of both Media Science, a world-leading audience research firm, and Hark Connect, an advanced platform for qualitative research. He’s also an award-winning former professor and author of over 80 peer-reviewed papers on the new and growing field of neuro-marketing.

Understanding Ourselves through Technology

Dr. Varan explains how there’s a flaw in the ever-expanding therapy industry. When we put our feelings into words, this is our natural tendency to describe what we think we ought to be feeling, rather than offering up the literal truth. Absolute self-knowledge is difficult to obtain, and therapists often deal with approximations and their clients’ natural reticence.

My wife provides talking therapy, and she often describes how patients reach for the truth about their feelings through a layer of intellectual interpretation. When asked how I feel at any given moment, I often find it hard to reach the absolute truth myself.

Every decision we make online is motivated by emotion, whether curiosity, desire, worry, boredom, or other states of mind. The motivation to explore AI-driven analysis is clear — identify what browsers are really feeling and you can deliver exactly what they want, even if they don’t know themselves. Does this sound sinister yet? Don’t worry — there’s a silver lining.

From Disney to Media Science

Dr. Varan’s initial research was sponsored by industry players, in exchange for early access to the results. ESPN and Disney were amongst the first sponsors Dr. Varan worked with.

In 2005, he ran a study on how effective video ads would prove on cellphone users. This was in the days before cellphones had the capability to show video content, so the iPod Classic was used as a proxy device. The study proved the viability of cellphone marketing. ESPN, and others, began to develop mobile content ahead of the rest of the marketplace.

Disney loved the potential of such research that they created their own independent media and advertising lab. This evolved into Media Science over time.

One early Media Science project involved using neurometrics to identify successful advertisements for Mars products so that these elements could be replicated. These new measures achieved 78% effectiveness, as opposed to surveys which proved little better than a coin toss. The neurometric measures used included facial expression analysis. Whereas a test subject might be honest in saying that an ad made them smile, a neurometric measure captures an unguarded, immediate response.

Other sensors used in neurometrics measure heart rate, galvanic skin response, eye movements, EEG and more. The vital linking factor between all these measures is that they are involuntary, and therefore truthful.

The Quirks of Machine Learning

Because machine learning systems create their own algorithms for solving a problem, even the programmers of such systems don’t know exactly how they work. Although we can achieve stupendously accurate results, we don’t know exactly how the AI has achieved it.

For instance, changing the chairs in the cubicles where subjects were being analyzed by an AI caused an algorithm for blink detection to drop from 92% to 84%. Oddly, nobody knew why.

However, despite the peculiarities of machine learning, neurometrics and AI, these techniques are a huge improvement on surveys and focus groups.

The Online Revolution

Speaking of focus groups, the pandemic forced Dr. Varan to redesign a product intended for in-person sessions. Once modified, it could include a host of data tools, including sentiment analysis and video content analytics. The results were significantly more useful.

A real challenge is how to take a bio-feedback approach out into the real world, and away from a laboratory setting. Filtering out the noise inherent in a chaotic at-home situation is crucial.

Another issue Dr. Varan faces is the problem with users misinterpreting physical data. Galvanic response, for instance, only tells you about the intensity of a response, not whether the user is having a positive or negative experience. For more specificity, the metric must be cross-referenced with facial expression analysis, for instance.

The Ethics of AI and Marketing

A lot has been said about the social media bubbles we create around ourselves, aided by algorithms which select for intensity and attention-grabbing content above all else. Dr. Varan believes this constitutes a breach of marketing etiquette. Advertisers should be bending the product to fit the user, not manipulating the viewer to desire the product. Kids are especially vulnerable to such manipulation and need to be protected for the technology’s misuse.

Dr Varan explains that there is a positive aspect to microtargeting too — the lack of waste in terms of ad content being pushed in front of users with no interest in it.

Media Science believes in putting the profits it makes back into the company, to improve its research capabilities and make employees feel more involved. In addition, they do frequently turn down clients if they have ethical concerns.

Looking Forward

Dr. Varan admits there’s a long way to go in terms of improving neurometric algorithms and exploring interactivity in content production. New ethical concerns and opportunities will inevitably arise as our devices get to know us better, and perhaps even help us to know ourselves.