US technology giant Microsoft has patented a tech system that scores video chat participants’ body language and facial expressions during calls.
Microsoft’s ‘Meeting Insight Computing System’, as described in patent filings, is a mix of physical hardware and software that could be integrated into future Microsoft products, such as tablets.
The system would use a combination of cameras, sensors, and software and even thermostats around the walls for both virtual and in-person meetings.
It would monitor factors such as body language and facial expressions, picking up on signs indicating a drop in productivity like tiredness and boredom, as well as number of people in a meeting and even conditions like light and temperature.
All this data would be used to create an overall ‘quality score’ that would help inform the conditions of future virtual meetings, with the aim of maximising a business’s productivity.
The patent filing, which was first spotted by GeekWire, has led to concerns from privacy experts, especially as millions of employees are now working from home.
Microsoft has already riled privacy concerns with its recently announced Productivity Score tool – which ranks employees based on activity such as how many emails they send.
Microsoft’s ‘Meeting Insight Computing System’, as described in patent filings, would use cameras, sensors, and software to monitor people and the conditions in a meeting – both in-person and virtual
‘This is a surprising move coming from Microsoft and could dangerously lead to greater levels of employee surveillance in the workplace and more worryingly, their homes,’ Jake Moore, an independent security specialist, told MailOnline.
‘When it comes to invasive productivity monitoring, you tend to see it in a small handful of businesses who may already have a particular fear of staff playing truant or not behaving as they should.
‘However, with Microsoft now entering the space of workplace surveillance, we are likely to see far more businesses feel this intrusion is more acceptable and possibly more important than employee privacy.’
Data researcher Wolfie Christl said: ‘This normalises extensive workplace surveillance in a way not seen before.
‘This is so problematic at many levels… not least, Microsoft gets the power to define highly arbitrary metrics that will potentially affect the daily lives of millions of employees and even shape how organisations function.’
While Silkie Carlo, director of the campaign group Big Brother Watch, said the system was ‘invasive’ and a ‘major step back for workers’ rights’.
Meeting Insight Computing System would combine several different components that would be positioned around a physical meeting room’s walls, as well as inside future Microsoft hardware, the patent suggests.
Meeting Insight Computing System could therefore be an umbrella term for a whole family of internet-connected devices that contribute to meeting quality scores.
The new system that can tell when employees are bored has been condemned as an alarming form of ‘workplace surveillance’ (stock image)
An enhanced camera inside a future Microsoft tablet, for instance, would record which of the invited meeting participants attended the meeting, their body language and/or facial expressions and even the amount of time each participant spends contributing during the meeting, according to the patent.
Employees who don’t contribute as much to a meeting could be called-up on that fact at the start of the next meeting and asked to kick proceedings off.
Meanwhile, a microphone may detect speech patterns consistent with boredom, fatigue and record how much time each participant spends speaking.
And a thermostat recording the current temperature in the meeting room could measure air composition – including oxygen and carbon dioxide levels – which ‘can have a negative effect on meeting participant health and comfort’, Microsoft says in the patent.
‘Similarly, a user may schedule a meeting for mid-afternoon in a room that tends to be intolerably hot during the afternoon hours.’
Microsoft has already recently riled privacy concerns with its recently announced Productivity Score tool – which ranks employees based on activity such as how many emails they send
This could help inform a quality score that would advise the meeting organiser to change the time of a meeting for when it’s much cooler.
The system would then flag potential challenges when an organiser is setting the next meeting up that were discovered from the last meeting and recommend alternative venues, times or even people to include in the meeting.
Useful data could make employee meet ups – which are often criticised for their ability to waste workers’ time – more productive in the future, Microsoft hopes.
‘Many organisations are plagued by overly long, poorly attended, and recurring meetings that could be modified and/or avoided if more information regarding meeting quality was available,’ the firm says in the patent filing.
‘Because conventional computerised scheduling systems lack real-world context, users may not be aware that they are attempting to schedule non-optimal meetings, which may result in meetings that are unproductive at best.
‘Accordingly, the present disclosure is directed to a meeting insight computing system that reports useful and meaningful information regarding meeting quality.’
MailOnline has contacted Microsoft regarding the patent for comment, although Jared Spataro, corporate vice president for Microsoft 365, has already stressed that the firm is ‘committed to privacy’ and the system was ‘not a work-monitoring tool’.
Also this week, Microsoft was forced modify its Productivity Score tool, which was announced in October, after criticism that it allowed snooping on behalf of business managers.
Spataro said in a blog post dated Tuesday that the firm is removing employee names that are linked to employee data, as a result of the criticism.
‘Going forward, the communications, meetings, content collaboration, teamwork, and mobility measures in Productivity Score will only aggregate data at the organisation level-providing a clear measure of organisation-level adoption of key features,’ he said.
‘No one in the organisation will be able to use Productivity Score to access data about how an individual user is using apps and services in Microsoft 365.
‘At Microsoft, we believe that data-driven insights are crucial to empowering people and organisations to achieve more.
‘We also believe that privacy is a human right, and we’re deeply committed to the privacy of every person who uses our products.’
