How will real-time monitoring of emotions impact the internet of things?
Cutting-edge tech being harnessed by the advertising industry could take the “internet of things” in fascinating new directions. Ad agencies are beginning to use emotion detection solutions, which employ webcams to track the movement of muscle groups on a person’s face, to understand their reactions to videos and other content.
At the Cannes Lions International Festival of Creativity last week, Matt Celuszak, CEO of Crowd Emotion, described how his company’s emotional detection technology combines algorithms and a webcam to understand facial expressions. The solution “is tracking about 32 muscle groups on the face about 15 times a second,” he told the advertising executives gathered at the event. Crowd Emotion and its clients, such as the BBC, are using this technology to help refine trailers for new TV series or video commercials. Celuszak described how the technology can be used to detect specific emotions, such as puzzlement, fear and happiness, as well as measuring the overall level of engagement.
Empathetic entertainment
Such systems could transform the entertainment industry as well as the advertising sector. In time, sports stadiums and events organizers are likely to use connected cameras and algorithms to monitor how live audiences react to what they are watching. The results could be used to shape future productions and team selections and even make real-time changes during an event. At a music concert, the band could use data showing the emotion of the audience to decide what to play next. Is it time to up the tempo? Given the intensifying competition for consumers’ time and money, and dwindling attention spans, all genres of entertainment are under pressure to be more responsive to the demands of their audiences.
Noting 400 hours of content are uploaded onto the internet every minute, Celuszak described how the BBC recently used the system to help develop a one minute trailer for a new video-on-demand series called the “Life Story,” featuring the acclaimed naturalist David Attenborough.
“We asked 150 people to watch 20 minutes of content,” Celuszak told delegates at Cannes Lions. “How do we take 20 minutes down to a minute? We look at where the low points of inflection move to the high points of inflection. Fear was the thing that drove people in and happiness sustained.”
Understanding customers and employees
Enterprises could make use of similar technologies to better relate to their customers and employees in a wide range of contexts. Better empathy can yield valuable insights. For example, rather than asking a consumer to rank their shopping experience, a retailer could use connected in-store cameras to automatically capture the emotion of customers as they browse particular displays or wait in line to make a purchase. Are they bored, frustrated or happy? The same technology could also be used to monitor employee engagement in a meeting or job satisfaction more broadly.
James Larman, head of strategy at advertising agency Drum, said emotional detection technology and consumer surveys yield very different results: the facial expression monitoring shows that people are not always engaged by content they later say they like. Moreover, the technology potentially gives much more detailed insights than a survey ever could. It can be used, for example, to detect how a person’s mood changes second by second.
Better citizen engagement
There are also many potential smart city applications. Municipalities could use emotional detection systems to see what citizens really care about. They could, for example, be used to gauge peoples’ reactions to billboards proposing a new road, a new transit station or a new recycling facility. They could also be used to detect what kind of messages resonate with residents and help improve citizen engagement. In time, connected webcams could even be used to monitor the moods of protestors and help determine how large a policing presence is required.
Of course, this kind of technology is open to abuse. Given the Orwellian overtones and the potential for serious invasions of privacy, emotional detection systems should only be used with the subject’s consent or where there is a very clear public interest. But, with the right controls in place and supported by appropriate metrics, equipping the internet of things with this kind of smart technology could produce a step change in the level of understanding between service providers and their customers, and between employers and employees. As competition for good people and good customers intensifies, enterprises will place a premium on empathy.
