My first ever smart home device were smart lights from Philips Hue a after graduating college. They were perfect when I just had a bedroom in a shared apartment. And more importantly they were fun! I could change colors, dim the lights, and pair them to movies. Over time the last 10 years I've slowly accumulated a bunch of smart devices from different companies to do different things.

At this point we're using connected devices to:

• Control our outdoor lights
• Control our indoor lights in rooms without light switches
• Play music throughout the house
• Ask questions to Google
• Turn the TV on/off and launch streaming apps
• Set timers in the kitchen
• Look at recipes
• Video call our family
• Open / close the garage door
• Lock our front door
• Look for packages and ring our doorbell
• Alarm and monitor the house while we're away
• Control our small sprinkler system
• And probably more

And I love it! It's awesome to be able to turn off all the lights in the house when I go to bed or to have the garage let me know I accidentally left it open. My absolute favorite feature to this day is being able to easily play music throughout the house just by asking Google. It still blows my mind. But sometimes things go wrong and something disconnects or dies and the whole system starts to fall apart.

Last week the lights in my living room stopped responding. Yesterday I found my wife on our front porch, locked out of the house by our dead Yale smart lock. Every day my office lights seem to flip a coin on whether or not they want to stay connected to WiFi. This isn't usually a big deal for me. I don't mind dealing with the troubleshooting most of the time. Unplug it, plug it back in and see if it's working. Reset the bulb and reconnect the WiFi. But as you add devices to the system the troubleshooting becomes more complex.

Lost connections

Recently IFTTT, a tool I've been using for years to connect various software and smart home services, decided to change their billing plan. Free customers would now be limited to three applets - the if this then that rules you use to connect two services - and the rest would be archived. At the time I'd been using IFTTT to:

• Sync my Fitbit scale data with Google Drive
• Connect Logitech's Harmony remote system with Google Assistant
• Send me text messages for my basement water sensor around my water heater

The Logitech connection alone used multiple applets to let me control my TV, speakers, and Xbox from Google Assistant. So when IFTTT changed their plan the voice prompts I'd been using for years - "Hey Google turn the TV on" - suddenly stopped working. And sure IFTTT is well within their rights to adjust their pricing strategy, but the issue here is that when the connections between these devices are so fragile, even a small disruption from one service can ripple through our connected home.

Abandoned by the manufacturer

An even more concerning issue is when the device manufacturer's change which connections they allow or stop supporting their connected devices completely. In 2014 the startup Revolv launched a smart home hub to help customers make sense of all the different smart home standards. Customers could connect their different devices to the hub and then use Revolv's app to create custom actions - similar to IFTTT. People used this hub to create custom actions like turning on their sprinklers when a motion sensor was activated. In October of that year Nest acquired Revolv to help build out their Works With Nest platform that opened up Nest devices to third party platforms. Less than two years later, however, Nest decided to shut down the cloud servers that powered Revolv, breaking the custom connections Revolv customers had set up. And then three years later in 2019 Google ended all new connections for the Works With Nest program! New Nest customers would have to use Google accounts that no longer supported the same third party integrations. What a journey. And this is only one company! There are plenty of other examples of smart devices suddenly becoming dumb devices or completely useless paperweights.

Troubleshooting

Who can you call when your smart home breaks?

When these connections break, it's not always easy (or even possible) to repair them with other services. My 1930s house has plenty of old systems that are still in use. We have steam heat radiators, cloth-wrapped electrical wires, and a 1990s home security system. While each of these requires maintenance, each is independent and none rely on cloud servers managed by their manufacturer. They can even be repaired by non-manufacturer experts! If one of our outlets stops working it can be repaired by any electrician. Familiarity with these systems fades in time as technologies like steam are phased out, but a 90+ year lifespan is a far cry from the 3 years of Revolv's hub.

Our Yale smart lock automatically unlocks for me most of the time, but never unlocks for my wife. We're both signed in with the app, have given it the appropriate location permissions, tried reinstalling the apps, and even resetting the lock per Yale's customer support. None of it worked and we're left with a lock that works but not as expected and there doesn't seem to be anything we can do about it. Yale shipped us a new lock but the problem persists. Even when companies are aware of issues they don't always fix them. Just this week Wink customers have been left without a working device for over a week with no updates from the company. Sometimes those fixes never come and customers are left with broken devices.

Overlapping services

Another frequent challenge is the number of overlapping services you can end up managing. If you have the time and money you might be able to plan out your smart home to strategically use compatible devices, but for someone like me who's built up my system over time it's easy for the services to accumulate. I have smart lights from both Hue and Wyze which each have their own apps. I typically manage these through Google Home, but could also use Apple's HomeKit. When our bedroom lights dimmed unexpectedly I had to check both my Google Home and Hue apps to find an old Hue automation that dimmed the lights. A true smart home would help me figure this out.

For someone like me who's interested in technology and is frequently an early adopter this is tolerable. But what about people who don't have the time, inclination, or expertise to put up with all this? They're left buying into devices and systems that may or may not work as expected. Designers of these systems need to understand the complexities of the environments they will be operating in and the wide range of customers who will be interacting with them.

Designing for everyone

Even when the system itself seems to be working as intended, our failures to design for all users can lead to unfortunate challenges. At our house the Google Assistant seems to have a harder time understanding my wife. She often has to repeat her query or say "Hey Google" more than once before it will start listening. Unfortunately it's not just our house that has these issues. Voice recognition in general has been shown to have gender and racial biases as the training data used to create machine learning models is often an incomplete sample based largely on data available from white males. As reported by the NY Times, a 2020 study from Stanford showed that speech recognition systems "misidentified words about 19 percent of the time with white people. With black people, mistakes jumped to 35 percent." Understanding the reasons behind these failures, as well as the implications for customers, is crucial to help smart devices work for everyone.

So where do we go from here?

Smart devices are a fun way to bring technology and automation into our homes and while all the challenges I complained about above are real, it's not all bad news. There are a few things smart home consumers and smart home manufacturers can do to make all of our lives easier.

For consumers

Plan for graceful degradation

While I'd love to assume that each company behind my smart home devices will both continue to exist and support their devices for their entire lifetime I know that is unrealistic. For less expensive and novel items (like a color changing lightbulb) this might not be a big deal. For mroe expensive and/or longer-lasting devices (like a lock, smoke detectors, or security system) can look for devices that can continue working without a cloud-based service. You may lose some of the best features, but at a minimum it should be just as functional as it's dumb home relatives. A lock should still unlock without WiFi, a smoke detector should still yell at you even if the power is out, and your security system should still beep even if it can't send you a text.

Invest in companies with a good track record

Many companies already have an extended track record of long-term support. Investing in products from companies like this as well as strong companies that have been around for a while is a good way to hedge against the eventual shutdown of a smart device.

For manufacturers

Diversify

Broadening our sample sizes and including customers and coworkers from traditionally underrepresented demographics can help create systems and services that work for more people. Those of us who work in technology should be thinking about how our products can work for people all over the world of different ages, ethnicities, income levels, education, and more. Everyone should be able to benefit from the efficiency and convenience smart, connected devices can bring. And as we continue to make these services work for more people, we need to continue ensuring that they work together and don't ask too much from the user.

Collaborate

To help with troubleshooting and setup, the systems we make should be able to easily talk to one another and there should be an easy way to see where a command came from. Who or what asked my lamp to turn on at 5am? We need to assume that customers aren't using these devices in a vacuum and are likely to have many other integrations, devices, and services running. It's up to us - not them - to ensure our products work well in that environment and that the customer can easily and successfully configure their system and troubleshoot when needed.

Open up

In an ideal world manufacturers would open source services they're shutting down so consumer can continue to use the product. This might be unrealistic but I think a realistic goal is to support additional integrations. If a company doesn't want to support a product then they should at least let the open source community continue to support it. Android's open source community is a great example of how old devices can be given an extended lifetime thanks to the developer community. I don't think we'll see something quite as flexible from smart devices, but if product teams start with more robust and powerful integrations it will be a fantastic start.