Q: How does it interact with my data warehouse / database?
At the scheduled frequency our Data Observability platform will run a SQL query to generate the metric’s value. An email notification is generated if this is outside of the defined threshold.
Q: Is there any impact on performance of that data store?
The impact should be very minimal but it will depend on the complexity & optimization of the SQL query used to generate the metric.
Q: Is there any additional cost (e.g. compute on Snowflake)
The cost should be very minimal but it will depend on the complexity & optimization of the SQL query used to generate the metric.
Q: What access level do I need to give? read only?
On data warehouses, read only access is all that is needed. Databases we need the same permissions needed for CDC replication.
Q: Does our data get loaded into the Data Observability platform?
No, the SQL query is run on the customer’s data warehouse/database, only the metric’s result, a number, is sent to our platform.
Q: What is the easiest way to run Data Observability
Create a data warehouse account with read only views to the data that needs to be monitored. The Data Observability checks can be run on the data warehouse staging tables (for upstream systems) or the production tables.