Alexa, tell Kabir…
Reusing an Alexa and an e-ink screen to improve communication within my house
In my house, my mom spends her time on the ground floor and I on the first floor, where my work setup is. Throughout the day, my mom shouts things up the stairs (I think it’s an Indian mom thing). “Dinner is ready.” “Check if the clothes are dry.” “I’m going out.”
It’s not a conversation, just small pieces of information or requests for something. But I would have to get up and come halfway downstairs to hear her well, only to realise it wasn’t something urgent. I got tired of the constant shuttling. Also, her voice gets affected trying to be so loud.
To find a way to solve this small friction, I used an old Alexa Echo Dot that wasn’t being used, and created an application that lets my mom inform me of small things through the day (non-urgent requests).
Making it work in one sentence
Version zero had to be used like this:
“Alexa, open Kabir.”
“What should I tell Kabir?”
“Dinner is ready.”
But that’s not how it should be, so a little modification was able to make it such that she only needed to say:
“Alexa, tell Kabir dinner is ready.”
Alexa reads the text back so she knows immediately if it misheard. The Echo gives the feedback; the screen is silent.
“Sent: Dinner is ready.”
Tech is super simple
The two devices can’t talk to each other directly. Alexa skills run on Amazon’s servers; the ESP32 is on home Wi-Fi. They need a meeting point on the internet, and it has to be free and need no server of my own.
01 · Ground floor
Echo
Mom speaks “Alexa, tell Kabir dinner is ready”
- voice to text
02 · Amazon’s servers
Alexa skill
Node.js; tidies the text, reads it back to her
- publish, TLS
03 · Free MQTT broker
HiveMQ Cloud
Keeps the last note (retained message)
- push, within a second
04 · First floor
E-ink
ESP32 on home Wi-Fi, subscribed to the topic; draws the note, stays
Learning from real usage
The first version worked technically, but free-form speech was harder than I expected. To make the first version work I had trained the application on around forty household phrases.
But when I asked her to use it, it immediately failed because she said things like:
“Alexa, tell Kabir that he has to pick up the clothes.”
“Alexa, tell Kabir to bring my shoes downstairs when he comes.”
So I added phrase detection that would pick these nuances up and pass everything after “that” or “to” freely to be shown on the e-ink screen.
After a couple of days we ran into another problem. Sometimes she would say “ask Kabir” or “remind Kabir”:
“Alexa, ask Kabir to water the plants when he is free.”
“Alexa, remind Kabir that we have dinner plans today.”
“Remind” could not be added as a phrase, as it is reserved by Alexa. Currently she has just been instructed not to use it. (I know, I know, it’s the wrong way around…)
The e-ink screen is super simple
My first version had a sender, timestamp, divider and dynamically sized text. All that information seemed irrelevant, so I reduced it to keep just the message and the timestamp.
I sometimes miss her messages
The prototype works, but at times I miss her messages because I am not looking up and there’s no audio feedback (yet). I could have just created a way for her messages to arrive on WhatsApp or as notifications on my phone by an application, but where’s the fun in that?
It also leaves room for things I could add next, like simple replies or Hindi support. But I want to resist making it do too much.
She still shouts, but at times remembers she doesn’t always have to.