article thumbnail

97 things every data engineer should know

Grouparoo

This provided a nice overview of the breadth of topics that are relevant to data engineering including data warehouses/lakes, pipelines, metadata, security, compliance, quality, and working with other teams. For example, grouping the ones about metadata, discoverability, and column naming might have made a lot of sense.

article thumbnail

What is a Data Engineer?

Dataquest

Most companies store their data in variety of formats across databases and text files. You’ll have a few different data stores: The database that backs your main app. Ride database. Customer service database. You’ll then need to store the parsed logs in a database, so they can easily be queried by the API.