article thumbnail

Audit_helper in dbt: Bringing data auditing to a higher level

dbt Developer Hub

However, ensuring that the values in the original table and in the refactored one match used to be a hard task that involved a lot of manual coding and some generalistic tests (such as counting the amount of rows or summing all values in a column).

article thumbnail

97 things every data engineer should know

Grouparoo

This provided a nice overview of the breadth of topics that are relevant to data engineering including data warehouses/lakes, pipelines, metadata, security, compliance, quality, and working with other teams. 69 The End of ETL as We Know It Use events from the product to notify data systems of changes.