Motivation

IoT industry is booming. There are variety of products on the market. Many of them are aimed to end users, especially in smart home applications. We can imagine that every electronic devices in the home will connect to the internet in the near future. They can be controlled over the internet, and can even be trained to work automatically. However, installation and configuration of electronic devices are usually bothersome, which stops most non-technical background families from adapting smart home solutions. Further more, improper configurations usually make those products not work as original designed, e.g. waste more energy after installed Nest.

The hot selling of Amazon Echo in North American proves people’s acceptance of voice interface. We believe that voice interaction will account for large part of human-environment-interaction modes in smart home. There are already lots of third party skills for Amazon Echo to control smart home appliances. Since people are comfortable in control appliances with voice interface, we think it is reasonable to make configuration via voice interface as well.

Design Concept

In our design, there will be a centralized monitor system in the smart home. The same design is also adopted by several commercial products like SmartThings and Philips Hue. All smart appliances have network connection. When an appliance is installed, it will upload it’s profile to the system database. The profile is provided by manufacturer. It contains the uniform resource identifier of the appliance, the type of the appliance, the services provided by the appliance, etc. With this information, the system are able to request the services from the appliance and to control it. For example, a dehumidifying dryer provides services to dehumidify, to query the humidity, to query the current state, etc. In addition, all appliance events will stream to the system database. When users want to refer to a smart appliance in the interaction with the voice interface, they can change the device state with traditional interface, e.g. buttons, switches. Since the system monitors every state changing event, if there is only one event at the moment, the target appliance is clear. If there is ambiguity, users will be asked to reproduce the event.

Implementaion

We builded a simple prototype with two smart lamps to demonstrate our idea. A smart lamp was made with a lamp, a LinkIt Smart 7688 Duo, a button, a relay, and an electricity sensor. The LinkIt Smart 7688 Duo is connected to the internet, and has it’s shadow device on AWS IoT platform. The lamp can be controlled either by the physical button or the command from the internet. The state changing event and current reading are streaming to the database on AWS RDBS. The voice interface and the logic behind it were builded with Alexa Skill and AWS lambda. We name the interface after the intelligent agent Jarvis in the famous science fiction Iron Man.

With this prototype, we demonstrated how to configure the lamps with custom names. Once the name of the lamp is set, a user can turn on/off the lamp via the voice interface. In this demonstration, I turned on/off a lamp with physical button in order to let Jarvis know to which lamp I referred. I demoed the situations that Jarvis sensed the correct device, Jarvis did not sense any device, and Jarvis sensed multiple devices.

We can further extent this prototype, such as teach Jarvis how to interpret sensor readings, learn the location of a device, or even some more advance command like “close the living room widow curtains and turn on the porch light after sun light.”