For several days after I returned from my vacation, the lights in my house would automatically turn on around 6 pm and then again at 7 am. This would have been far more frustrating in the morning if it weren’t already light and we weren’t mostly up anyway. But there were a few times when my husband rolled over and seriously questioned my commitment to smart home technology — an emerging category of futuristic household conveniences — and, more importantly, his commitment to me.
Meanwhile, I desperately tried to figure out what turned my lights on at the appointed times. It was not something I had expressly programmed. Rather, something “smart” in my house had, on its own, come up with its own schedule.
Finally, I figured out the problem. All of my Philips Hue lights, turned on and off randomly. The issue started after I had linked those lights with the Nest Thermostat, a connected “learning” thermostat, so that my lights would automatically turn off when my thermostat was set to “away.” If it detected that my thermostat was set to “away” for more than a day, the Nest would tell my Hue lights to turn on and off randomly based on the schedule my Nest had figured out. Ironically this was the feature that led me to write an article calling Nest the killer user interface for the smart home.
It may be, but it needs some work. First off, as you can see, the original settings are not restored after you come back from an extended period of being away, leading to awkward moments of lights turning on without you telling them to. Worse is that there’s no place to see what the heck your house is doing when the lights start coming on and off seemingly at random. This problem was worse for me, because I’m an outlier in that my lights are also tied into a series of other smart home gadgets. They include a Wink home automation hub that controls some of my devices, an Amazon Echo that controls other devices, several If This Then That Rules that also control devices, the Nest system and several other apps by dint of my job as a technology editor.
However, even a normal user might still wake up to glowing bedroom lights and want to know why the heck they decided to turn on. In another snafu, the Philips Hue software had decided I lived in some crazy time zone and scheduled my wake up call four hours earlier than I had expected. That also took a little bit of digging to figure out the reason. But this isn’t a complaint session about my Philips Hue lights or my Nest thermostat, which are actually some of the best connected gadgets I own.
These stories illustrate some of the problems that smart home technology has yet to solve, even three years after some of them were first mentioned. They also show why having a truly smart and intuitive home is still the stuff of science fiction (or very expensive programmers).
The wrinkles came to mind because of my recent experiences, but also because I just saw that yet another company, Fin, is trying to create some kind of operating system (OS) or artificial intelligence to help make the smart home a little more intelligent. In the words of the company’s founders, they “are trying to create something like the OS from Her today.”
For those who didn’t see the movie Her, the OS was a sort of artificial intelligence with a very human personality (actress Scarlet Johansson provided the voice) that lived on a desktop computer and a mobile phone. In addition to controlling the gadgets, emails and calendar of the main character (played by a Joaquin Phoenix), the operating system helped to manage his life. Aside from becoming a disembodied love interest, the technology was in many ways exactly what people say they hope is the future personal assistants and home automation.
But when it comes to the smart home at least there are several hurdles standing in the way of any Her like interactions. Some are technical like a lack of enough sensors in the home. Some are social and related to demographics like people being more comfortable telling a light to turn off rather than flipping a switch, and will change over time. But because I eat, sleep and breathe in a smart home, and I happen to interview people in the smart home industry all day long, I figured it’s worth explaining why exactly we’re still really far off from Her.
One hub/one dashboard/one anything
: To achieve the Her-like experience or even a great artificial intelligence, you need a central place for all of your smart home information. Maybe it’s a hub, maybe it’s a router, maybe it’s software on your phone or in the cloud, but everything in your home has to talk to this central hub. It should also should have some kind of mechanism to tell the consumer why their bedroom lights turn on at 7 am. Right now, most companies are fighting to provide this software, which is causing no end of drama for the consumer and other players trying to build a business in the smart home. For example, Apple just started certifying gadgets under its HomeKit program, but not every gadget will work with the program, and Android users won’t be able to use it at all. Meanwhile, many physical hubs only work with certain devices, leading consumers to be confused about what works with what.
In an ideal world every new device a consumer brings into the home should be able to simply work with the central hub device or software and start sharing data with it. We will need one or two software protocols such as the AllSeen Alliance’s AllJoyn protocol or perhaps something designed by another group that developers can easily build into their products and software, but the only capabilities it really needs are a way to say what a device is and what it can and can’t do on the network. This will allow developers from many different companies to take those capabilities and write software that can take advantage of them.
We don’t have the inputs we need
: A good artificial intelligence will need context to build great automation. But homes don’t offer computers that context yet. There is both a lack of understanding of where people are in the home (presence detection), who that person is (computer vision or just grabbing a unique ID off their phone if it’s on them), what they are doing (computer vision), and the power dynamics in the home. For example, if a six-year-old demands that the stereo switch from Brahms to the latest Taylor Swift song for yet another rendition of Bad Blood, is the home smart enough to know that the adult’s previous command is the one to stick with? At least if the speaker is in the kitchen and not in the six-year-old’s room?
We don’t have training data yet
: And that previous example gets us to another technical hurdle for artificial intelligence. Any artificial intelligence has to be trained on very particular data sets. When you train a computer to learn, you don’t train it to learn about everything. You train it to learn about one very specific thing such as identifying an appropriate cartoon caption for the New Yorker, or identifying a person’s face based on what’s around them. We already have some great artificial intelligence for the smart home around computer vision, figuring out what temperature people like and when they are home. But we are still missing crucial things such as when people like certain things to happen and when they might now want certain things to happen. Morning music is a great example of this. Most mornings you might love to wake up with your stereo playing, but not if your kid was up all night puking. Then you might want the house silent in hopes that she stays asleep.
And because homes are places where things change as children age and circumstances suddenly shift, it can take time to train artificial intelligence to quickly pick up on each individual families’ routines. And while it learns, the mistakes can be glaring. An example might be having to expressly tell a computer that your child’s bedtime is later now, or that because your wife is having hot flashes, it’s time to dial back the thermostat at night.
Smart homes are shared
: The main character in Her lived alone, possibly because he wore those nerdy high-waisted pants, but also because it’s much easier to highlight artificial intelligence that manages the life of a single person in a home (his loneliness was also a key point in the plot). However, for most people, homes are shared environments while PCs and mobile phones are intensely personal. This changes how you have to build everything from the controls (can your tween change the air conditioning settings your home?) to how you update an app (how many individual devices do you want to update that lighting app on?). Other considerations involve managing for people who don’t have smartphones yet which may not show up as being inside a connected home and offering some way for guests to control some things in the home that make sense. Getting artificial intelligence to figure all that out and then to offer that requires a lot of training, security and access controls and scenario-building that most app builders are barely even thinking about yet.
There is no single way to control the smart home
: Do you talk to Siri or Google or type your texts? What about when you want to open the camera on your phone? Do you use a gesture to quickly flick the camera on or do you hit the buttons on the screen? Because of all the sensors in phones, people can interact with them differently based on what is the most convenient for them at that moment. The home will be no different, but there are also more distractions. In my house for example, you can turn on the living room lights at the wall switch, via an app, using a physical button that sits on a table, or your voice. Anyone who unfamiliar with the smart home, uses the switch. When the house is noisy, we use the movable physical button, and pretty much at all other times we use our voice. Sometimes, when the automation gets it right, we don’t use anything because the lights just happen to be one when we want them on. That’s exactly what we want, but, in fact, it’s pretty rare.
Most people say their hope is to get to that last point, where when the lights just turn on when they walk into the living room. But you can probably appreciate why that hasn’t really happened yet. And even when it does work, there will still be some glitches to be ironed out. Even as a dedicated fan of the smart home, sometimes I have to admit that the basic wall switch looks pretty good.
