Author: Benjamin Saladin
We all have probably been in discussions about automation of repetitive tasks and very likely the outcome was: “This seems too expensive to automate.” But this decision making process is most likely based on experience and gut feeling. Don’t get me wrong, this can be a valid approach if there is no underlying data or models that would help give you a better outcome on your ROI (return on investment).
On the other side, automation is often a valuable approach. Using the dashboard below, you can check out the impact on ROI and the break-even date from which your investment starts to pay off. The tool helps you to evaluate how much time and money you could save by automating repetitive tasks. Give it a try - automating will be worth it, believe me.
Managing the pipeline from raw data ingestion, storage on a data platform to data transformation plays a crucial role in each data-driven company. This pipeline, also called Extract, Load & Transform (ELT) can be heavily outsourced by using a modern data stack. This is where Fivetran, Snowflake, and dbt appear on the stage.
Two unicorns and a rising star will make our daily lives in analytics exciting, fast and more reliable. Snowflake is a fantastic offer for a data warehouse service either hosted on AWS or Azure. This service charges for computing and storage only. No usage, no bills. As with Fivetran, this offering is based on credits that depend on the tier and the location of your cloud offering your Snowflake instance is running on. We are currently working on the maintenance of our Snowflake data warehouse automated by Terraform - exciting times I would say.
To feed the Snowflake Data Warehouse we use Fivetran, of whom we are the first official partner in Switzerland. Amazing people in Munich to say the least. We are looking forward to visit them as soon as these not fun times are over. Fivetran essentially is a data pipeline service that takes over the heavy lifting for us - experts say the ingestion and maintenance can take up about 80% of an analytics platform, before any benefits in the form of actual analytics can be presented to the customer. If you are interested in their offering, let us know and we can support you on that journey.
dbt (data build tool) takes over the last part in ELT - Extract, Load & Transform. This application is a gem, a rising star in the analytics community and getting a lot of attention from Fivetran. The best is, this is open source and available to use for anyone. The community is highly active and very helpful in your modelling journey. On our own journey, we aim to build a product, based on these tools and other, to provide a comprehensive analytics suite to our customers.
We are currently working in a proof of concept mode working together with innovative companies that implement future-proof technology while applying fundamentally robust pattern to build upon. Very exciting times ahead, of course also for our Swiss market since Azure has since 2020 data centres in Switzerland which takes off concerns holding certain industries back from moving towards cloud solutions. Have you got experience with dbt or any kind of dbt offering in Switzerland?