A smart display is a smart device consisting of a speaker and a screen. It has built-in far-field microphones
The screen shows video, text and graphicinformation, you can use it to make video calls. Some people think of a smart display as an electronic tablet, but they are conceptually different devices. The interface of the smart display is deliberately simplified, since its main purpose is to solve everyday tasks with the help of a voice assistant: control a smart home, planning and managing a calendar, searching for information, setting a timer or an alarm clock.
Smart displays do the same thing as smart speakers, but they add extra functionality to that. For example, here's what the Google Nest Hub Max is capable of:
- Show videos and movies, but due to the nature of DRM, Netflix appeared on Hub and Hub Max relatively recently, only in July 2020.
- Display photos.
- Search information and show search resultson the screen. Interestingly, recipes occupy a large share in the statistics of search queries from smart displays - this gives one of the most popular locations of the device in the house.
- Make video calls with Duo and Meet, and since August 2020, also Zoom.
- Translate voice commands into foreign languages in real time, making the translation easier to comprehend when reading from the screen.
If we talk about controlling a smart home, then by selecting the desired device from the list, you can, for example, adjust the brightness of the lamp with one touch. Very comfortably.
An interesting option is also available in the implementationvideo calls, - auto-framing (also known as auto-zoom or auto-pan). With autoframing, the camera focuses on a person and "follows" him while moving (within the camera's field of view, of course). This is achieved using computer vision and image cropping. For the first time on smart displays, autoframing was implemented on the Facebook Portal, and then on the Nest Hub Max. This feature is actively used by Google and Facebook, and users are responding positively to it - this can be seen from the Net Promoter Score.
How it all began
Smart display prototypes without voice assistantsappeared in 2000. Digital electronics maker 3Com (co-founder Robert Metcalfe invented Ethernet) pioneered the development of screen IoT devices. The 3Com Audrey was conceived as a kitchen device and had Internet access, writing and sending emails, and playing audio and video.
In 2006, the first touchscreen device fromChumby for the consumption of media content, which marked the beginning of the line of devices of the same name. Sony Dash, released in 2010, used a platform from Chumby with alarm, radio, Internet access, audio and video playback and a wide variety of applications. Despite the stable operation of the device, in 2017 Sony suspended the work of the servers serving Sony Dash without releasing another analogue.
The main thing that distinguished these devices from current smart displays is the lack of a virtual assistant, gesture control, and voice control.
Sony Dash, 2010
From early adopters to the consumer display boom
Over the past three years, only in the USA and Europe came outmore than 30 models of smart displays from more than 12 brands. In June 2017, Amazon launched sales of its first smart display, the Amazon Echo Show, and in September of the same year, sales of the Echo Spot, a smart alarm clock with a screen and a voice assistant.
Until the summer of 2018, the online giant wasmonopolist in the smart screen market until Lenovo and JBL released their devices with Google Assistant. Google itself announced the Google Home Hub (later renamed Nest Hub) in October 2018, positioning the device as a hub for managing a smart home, while its partners, who used the Google Assistant, relied on sound quality.
A month later, Facebook Portal came to the market withvoice assistant Alexa from Amazon, and its main user scenario was called video calls. Russia also did not lag behind the trend - at the end of 2020, SberDevices announced SberPortal - the first Russian smart display with a system of virtual assistants on board. By the way, the design of the Russian portal was carried out by the notAnotherOne team, and already in April 2021 the device won the most prestigious design awards - Red Dot Design and iF design awards - and with them recognition from the global design community. Finally, the recently announced update to the iMac line, although it cannot be called a smart display, a number of functions are derived from smart displays - a microphone matrix of three microphones and six built-in speakers. These innovations allow you to both use the device for conferencing and control functions using a voice assistant / voice input (starting with the macOS Catalina revision, users have access to advanced voice control functions, in addition to Siri functionality).
Today we can see the explosive growth of the smart display segment all over the world.
A bit of analytics
Smart display market
First place is occupied by Amazon with 55% market share,followed by Google - 33%, and Facebook - only 2% with its three generations of devices: Facebook Portal, Facebook Portal Plus and Facebook Portal Mini. Smart displays from Lenovo, JBL and LG with built-in Google Assistant account for 6% of the market (no brand breakdown).
The growth in popularity of displays is clearly illustrated by the numbers: in the first quarter of 2018, the share of smart displays was less than 2% of the market, in the first quarter of 2019 - 10%, and by the end of 2020 - already 26%.
* Global Smart Speaker Market Report Q3 2020
As of Q4 2020, smart display sales accounted for just over 25% of total smart speaker sales. In Q1 2018, the share of sales of smart displays was less than 2%, in Q1 2019 - about 10%
Smart display locations
The places where it is customary to place smart displays and speakers is highly predictable: 44% of devices are in living rooms, 37% of devices are in the bedroom, 32% are in the kitchen.
If in the living rooms there are often speakers, since the TV takes over the display function, in the kitchen there are smart displays, then in the bedroom smart alarm clocks and smart watches dominate.
In the image above are several representatives of thesedevices. They are united by a small basic screen, the resolution of which is enough to display basic information, as well as a microphone matrix, speaker and voice assistant. The price tag for such a device can start from $50. Amazon initially occupied the premium segment by adding a camera and high-quality speakers.
We should also mention the fresh release of smartdisplay from Google - Nest Home Hub 2. Thanks to the built-in mmWave-radar, the device can monitor the phases and duration of the owner's sleep, so it is suitable for placing on the bedside table.
Using an array of microphones, the device is also able to measure breathing rate, duration of snoring and the number of coughs per night.
It is not yet clear whether the device recognizes the owner by snoring, but we will definitely test this point.
Displays for the kitchen
In 2018 General Electricannounced a smart cooker hood with a 27-inch touchscreen, two cameras and an on-board Google assistant, the Kitchen Hub. The cooker hood can make video calls, view recipes and play videos on the display. Voice control of the device is carried out through the remote control.
Having already launched Kitchen Hub sales, GE announcedKitchen Hub 2, microwave oven with screen. The developers have placed a third AI camera inside the device, which makes it possible to recognize the ingredients and the readiness of the dishes. Unfortunately, there is no information about the start of sales yet.
Another prototype presented at CES 2019KitchenAid Smart Display - 10-inch KitchenAid Smart Display with Google Assistant from Whirlpool. Its main difference is moisture resistance and an oleophobic coating of the display, thanks to which there are no fingerprints on the surface - a common thing in the kitchen. Alas, the device never went on sale.
Displays for other rooms
In 2018, Kohler announced a smartmirror. Unfortunately, its functionality is limited to voice assistant only and it is not capable of displaying information. In 2019, several Chinese companies began developing a smart mirror with an integrated display. Despite the large number of DIY products, mass and commercially successful smart mirrors for the home have not yet happened.
We would like to look into the future and show how we see the concept of a smart mirror that fits perfectly into any bathroom.
Several possible features of this device:
- Focus on privacy - no cameras + physical button to turn on the microphone or gesture to turn on and off.
- Electrochromic glass - a variable degree of specularity and transparency depending on the conditions and content that is displayed there.
- Waterproof, this will help the use of MEMS speakers and piezo microphones.
- Built-in mmWave radar for recognitionusers, gesture control, recognition of vital parameters (temperature, pulse, skin color, pupil condition). The radar can, without the use of cameras, while maintaining the user's privacy, recognize the user's skeleton and movements, and read the heart rate and breathing rate.
What could happen to the smart display market in the near future?
- Over time, the display market will become even moresegmented, the specialization and usage scenarios of the device will increase: from kitchen displays to advanced bedroom displays. For example, a smart alarm clock with soft lighting that will gently wake up the user.
- Emergence of a new segment:There is a shortage of premium devices in the market today. The average cost of a device for a home is about $400. Perhaps there is a need on the part of manufacturers of luxury equipment, including premium audio brands.
- It is possible to use enabler technologies:displays (microLED, AMOLED, 3D displays, projectors), speakers and microphones (arrays of CMOS speakers, parametric speakers, further increasing the number of microphones in matrices, the use of piezo microphones), radars and sensors (mmWave radars, environmental sensors), connectivity (WiFi 6, 5G).
- Further growth of the segment with a small amountnew players: only a limited number of companies are ready to partner with Google / Amazon with a large investment volume and low margins. But all the cream today is skimmed by a small number of players with a complete infrastructure of services.
Displays for B2B
Displays in the corporate segment
In many companies, especially Fortune 500,the use of devices with microphones and smart speakers is prohibited, as there is a huge mistrust of privacy and data protection. One of the few companies that managed to enter the large corporate segment was Cisco.
Cisco updates its line of smart displays in 2020for conference calls. There are now two models. The youngest model is 23 ″, the older one is 27 ″. The camera is equipped with a physical shutter in both models. The older model supports a stylus, connecting a laptop with parallel charging, AI / dynamic backgrounds, a patented noise reduction system. The cost of the younger one starts from $ 1,490. The cost of the older one starts from $ 5,990.
Separately, it is worth mentioning the co-branding initiative from Zoom -Zoom for Home,a line of professional devices for working fromHouses. The recommended retail price for the first device in the line is $599. At the same time, the device has a wide range of interfaces, starting with a 27″ touch display with pen support, ending with three cameras, an array of eight microphones and stereo speakers. With all this, this is a highly specialized device for video conferencing and presentations; other functionality is limited at the OS level. Lacking electronics expertise, Zoom Inc. attracted conference solutions manufacturer DTEN as a partner, which bore fruit - the device was awarded the title of Best of The Best Red Dot design award in 2019.
Hospitality Smart Displays
A big trend is the use of smart speakers and displays in hotels.
Although Amazon launched Alexa for Hospitality back in June 2018, support for this program on the Echo Show is not yet available.
Nest for hospitality - program and initiative fromGoogle aimed at using smart devices in hotel chains. A custom version of Hub Max without a camera is used (in the custom version, the camera is present, in the version for the hotel it is removed) and a special version of the firmware / UI (functionality is limited, administered remotely; the emphasis is on privacy, after the user has checked out, all information about it is erased). The main purpose of the display in this case is to provide the user with information about the services of the hotel and its surroundings, to obtain financial information (invoice for accommodation and additional services).
Another interesting case for hotels wasintroduced by CIRQ + last year. This modular system allows you to control air conditioning, multimedia and lighting systems. The device has a radar to control systems. For example, if the radar is in another room, it adjusts the intensity of the air conditioning in such a way as to save energy costs.
But, as in many other industries, China is ahead of Western competitors - Chinese startup Xiezhu has introduced five smart devices aimed at the hotel segment:
- Self-check-in display designed to replace the front desk staff.
- Smart video doorbell for hotels with the ability to let visitors in without getting out of bed.
- Smart display for rooms, integrated with lighting, climate control and multimedia systems.
- Smart TV remote control with voice control.
Smart displays in telemedicine
The pandemic has led to increased interest in telemedicine, both in services and devices for telemedicine examinations.
A number of players (Amwell, H4D, etc.) announced telemedicine kiosks, the central element of which is a smart display. Regarding devices, basic recommendations/instructions are given by a voice assistant, and the process itself occurs under the supervision of a doctor.
Let us fantasize a little on the topictelemedicine kiosk: a portable system whose brain is a smart display. Most often it is required to direct the screen closer to the user; automatic adjustment is assumed using servos. An array of cameras and sensors to track vital parameters/indicators.
- A portable telemedicine system centered on a smart display.
- Automatic adjustment of the position of the screen on the servos.
- An array of cameras for analyzing vital signs, facial expressions.
- Highly sensitive far-field microphones (analysis of breathing, coughing, wheezing).
- Platform power sufficient for processing short circuit tasks.
Development and popularity of smart display technologiesdriven by the unconditional demand for their functionality: from smart home control to sleep monitoring and demonstration of recipes in the kitchen. Total social isolation has further fueled interest in such devices, and the development of the Internet of Things and wireless communication technologies will make it possible to adapt the use of smart displays for more applied cases, including in difficult application conditions.
See also:
Smart clothes: what it is and how it works
Scientists have presented a microscope that allows you to see the smallest cellular structures
The animal came to life after 24 thousand years of hibernation in the Siberian permafrost