In this video post you see how you can build a Streamsheet that receives data from Shopify via Apache Kafka The resulting Streamsheet generates aggregations and charts to serve as a real-time dashboard for a Shopify online shop.
Shopify Dashboard (Part 2):
How to aggregate and chart Kafka data in Streamsheets
This is the second part of the Shopify/Kafka/Streamsheet use case. In the first part we built a cloud based gateway to connect a Shopify store to Apache Kafka (hosted on the Confluent Cloud). In this second part we build a Streamsheet on a second Streamsheet Server that subscribes to the Kafka cluster and displays the real time data from Shopify in a dashboard.
This second Streamsheet Server runs on-premise behind the local firewall. Since the Kafka connection is initiated from the local Streamsheet Server, it can easily receive the event driven data from the Kafka cluster in the cloud. This is one of the beauties of the broker concept behind Kafka.
The following slide shows the setup of this use case.
In the video you will see how to aggregate and chart the data from the Kafka cluster that we used in the previous post. In the video I demonstrate 2 different ways on how to store the aggregated data. First I simply use Streamsheet cells to store the aggregated data. This will work if you only need the data persisted for a limited time (for example the last 300 sales). At the end of the video I show an alternative way using MongoDB. This allows to store the aggregated data persistently over long periods of time. And it also allows to retrieve the data from other applications. This could be another Streamsheet or even a 3rd party application build with another programming framework.
I recommended to watch the following video in full screen mode. The video is about 12 minutes long.
First Video: Build a Shopify-to-Kafka Gateway with Streamsheets
Did you like this blog post? Share it now
More Articles Like This
In this video post you learn how you can build a cloud-based Streamsheet that serves as a gateway between Shopify webhooks (executed by a Shopify Online Store) and Apache Kafka (hosted on Confluent Cloud).
This blog post introduces the basic operation of Streamsheets. Using the example of a smart WiFi switch with an integrated power consumption meter (Shelly 2.5), we set up a complete Streamsheet use case within 10 minutes.