Snips, Shelly and Streamsheets
In a previous blog, I already introduced the Wi-Fi switch Shelly 1 Basically this IoT-enabled switch allows to open or close an electrical circuit by sending the value “on” or “off” to a predefined MQTT topic. The electrical circuit can then be used to control an electric light or electrically operated machine.
Another very nice IoT component is Snips. Snips basically is an Alexa-like speech recognition system that runs locally on a Raspberry PI without sending your voice data to the cloud. Snips use MQTT and it simply translates your speech commands into MQTT Messages which makes it very easy to integrate Snips into any kind of automation solution.
You can build your own speech assistants using their easy-to-use web interface. Building speech assistants happens in the cloud, but as said before all speech processing is done locally on the Raspi.
I used Snips web interface to build an assistant that allows to switch on or off the light in different rooms of the house. In the following screenshot, you can see that the process is pretty straightforward and easy. Basically, you define intents (here “OnOff”) and then you define slots that allow parameterizing your speech command (here “room” and “on/off”).
If you have a look at the training examples in the middle of the screen, you can see what kind of phrases the intent is trained to understand (e.g. “Switch off the light in the bathroom”).
Once the assistant is deployed on the Raspi you activate Snips by saying “Hey Snips” and then maybe “Switch on the light in the bathroom”. Snips process the input and then delivers an MQTT message to a given topic (here “hermes/intent/kristianraue:OnOff”).
In the JSON payload of the messages, you find the name of the intent (here: kristianraue:OnOff”) and the values for the slots of the intent (Slot 0: “on” or “off”, Slot 1: “bathroom”, “bedroom”, “kitchen”, etc).
You can then build a Streamsheet that subscribes to the given topic and reads the JSON payload of the incoming messages. If you link the JSON fields slots.0.rawValue and slots.1.rawValue to cells in the Streamsheets via Drag-and-Drop, you can build sensible spreadsheet logic based on these values. This is shown in the following screenshot (which is in formula view).
Every time a message arrives, the Streamsheet will publish data to two topics simultaneously. In cell B19 the Streamsheet publishes the payload in cell A12:B13 as JSON to the topic “dialogueManager/endSession”. This very handy MQTT command ends the Snips session and sends the given voice output to the speakers of the Raspi, telling the user what action has been executed.
The second MQTT message is published to the topic where Shelly is listening. As explained in the beginning, Shelly is subscribed to an MQTT topic (here: “shellies/shelly1-BA01F8/relay/0/command”) and waits for text payloads of “on” or “off” to switch on and off the light.
Note: In this demo were are not making use of the slot 1 value, instead we always switch on and off the same Shelly device. But of course, you could easily extend the Streamsheet to use a VLOOKUP() function to look up the corresponding Shelly device name for each room and then change the topic for Shelly respectively.
About the author
Kristian Raue is a co-founder of Cedalo GmbH. Previously, he was the founder and CEO of Jedox AG in Freiburg, a leading provider of Business Intelligence software with locations in the USA, Australia, and Singapore.
From 1991 to 2000, Kristian Raue was the CEO of IntelliCube Software AG, which has merged with the INFOR Group. Kristian Raue studied industrial engineering with a focus on mechanical engineering at TU Darmstadt.
Subscribe for monthly updates
