Christophe Blefari

MARCH 1, 2023

docs — in dbt you can add metadata on everything, some of the metadata is already expected by the framework and thank to it you can generate a small web page with your light catalog inside: you only need to do dbt docs generate and dbt docs serve. The dbt snapshot page is the best illustration I know of the SCD.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Propel Data

MARCH 8, 2023

We'll share how to integrated the Apollo Studio API explorer into Propel’s documentation, options we considered, and some of the challenges.

Data Analytics 64

Data Analytics Data 64

Christophe Blefari

MAY 29, 2023

Writing design docs for data pipelines. Honestly this looks like a disguised Databricks. States of data season — Airbyte's state of data , Databricks's , lakeFS's. Engineering Levels: a simple framework for startups. Databend and the rise of Data warehouse as a code.

BI 130

BI Data Warehouse Data Data Pipeline 130

Christophe Blefari

FEBRUARY 2, 2024

DuckDB adoption numbers are demonstrating a real trend behind the "hype" DuckDB docs website gets 500k unique visitors per month and DuckDB has a new shiny website. DuckDB announcements The Duck creators announced that v0.10.1 is coming soon and before end of July we might get the v1.0.0.

Datasets 130

Datasets Data Lake SQL Algorithm 130

dbt Developer Hub

FEBRUARY 12, 2024

In the past several years, dbt Docs helped centralize the documentation workflow and dramatically improved the documentation process. While useful, dbt Docs only ever provides a single point in time snapshot, and lacks any sense of your platform’s deployment and execution information. Look at that lineage! passing model! passing tests!

Metadata 59

Metadata Raw Data BI Project 59

Start Data Engineering

SEPTEMBER 29, 2021

dbt docs 3.7. Configurations and connections 3.2.1. profiles.yml 3.2.2. dbt_project.yml 3.3 Data flow 3.3.1. Source 3.3.2. Snapshots 3.3.3. Staging 3.3.4. Marts 3.3.4.1. Core 3.3.4.2. Marketing 3.4. dbt run 3.5. dbt test 3.6. Scheduling 4. Conclusion 5. Further reading 6. References 1.

Building 130

Building Data Project Engineering 130

dbt Developer Hub

NOVEMBER 13, 2023

Make sure dbt Explorer always has the freshest information available ​ The old way : Your dbt docs site was based on a single job's run. However it doesn't make sense for dbt Explorer to show docs based on a PR that hasn't been merged yet.

Cloud 52

Cloud Database-centric Metadata Data Warehouse 52

Azure Data Engineering

JULY 16, 2022

For a detailed list of settings and sample JSON code, please visit the Microsoft Docs reference link below: Reference: [link] Service Principal: Service Principal of the data factory. User Assigned Managed Identity: User managed identity of the data factory.

Datasets 130

Datasets Certification Data Coding 130

Christophe Blefari

OCTOBER 19, 2023

See the doc. Private means the model is accessible only within the same group—a model can be only in one group. Protected means only a reference within the project and public from everywhere. That's all for the core project.

Project 264

Project Finance SQL Government 264

Tweag

MAY 16, 2023

A REPL nickel repl , a markdown documentation generator nickel doc and a nickel query command to retrieve metadata, types and contracts from code. i s _ m a t c h "^%{prefix}" ) in { c o n f i g | not_exported = { a p p _ n a m e | String | S t a r t s W i t h "mysql" | doc m%" The name of the mysql application. c o n t r a c t.

MySQL 131

MySQL Metadata Data Validation Coding 131

Snowflake

MARCH 15, 2023

If you’re already an avid docs user, don’t worry—your bookmarks will continue to work. The docs site helps curate the latest features with homepage highlights and easy access to the Releases section. We have preserved all the existing URLs, ensuring a seamless transition for our loyal users.

Accessible 52

Accessible Accessibility Project Building 52

Christophe Blefari

SEPTEMBER 25, 2023

Astronomer released Ask Astro — A LLM application that is able to understand Astro docs to answer most of the Apache Airflow questions. The source code is on Github. The implications of scaling Airflow — Sarah, who's working at Prefect, wrote a post about Airflow downsides at scale and how Prefect mitigates them.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

Christophe Blefari

SEPTEMBER 25, 2023

Astronomer released Ask Astro — A LLM application that is able to understand Astro docs to answer most of the Apache Airflow questions. The source code is on Github. The implications of scaling Airflow — Sarah, who's working at Prefect, wrote a post about Airflow downsides at scale and how Prefect mitigates them.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

Tweag

NOVEMBER 1, 2023

For example: { i n p u t s | not_exported = { f o o | String | doc "Doc of foo" , b a r | Number | doc "Doc of bar" , u n u s e d | Bool | optional , } , l o c a l | not_exported = { computed = s t d. Combined with field metadata, records can come close to the notion of modules in other languages. s t r i n g.

Metadata 105

Metadata Programming Programming Language Coding 105

know.bi

MAY 10, 2023

In this post, we'll start from the existing how-to guide in the Apache Hop docs, but add a bit more context and goes into a bit more detail on how to get everything going. As we started doing early this year, this post was contributed to the Apache Hop docs as an extended Apache Airflow how-to guide.

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

know.bi

SEPTEMBER 20, 2023

is available: Apache Beam upgrade, Google Dataflow docs and new transforms for Google Analytics 4 and Google Sheets Input and Output. Apache Hop 2.6.0

Google Cloud 59

Google Cloud Data Integration Data Engineering Data Engineer 59

KDnuggets

MARCH 29, 2023

Automate the Boring Stuff with GPT-4 and Python • Introduction to Python Libraries for Data Cleaning • Google Answer to ChatGPT by Adding Generative AI into Docs and Gmail • Top 15 YouTube Channels to Level Up Your Machine Learning Skills • 3 Mistakes That Could Be Affecting the Accuracy of Your Data Analytics

Python 69

Python Machine Learning Data Analytics Data 69

Towards Data Science

DECEMBER 1, 2023

Store snapshots in a separate schema Take a while to generate dbt documentation using the “dbt docs generate” command. run “dbt docs serve” to open it in browser Serving —  Snowflake Dashboard Finally, visualize your transformed data using Snowflake Dashboards. Good documentation provides better data discoverability and governance.

Data Engineering 74

Data Engineering Data Engineer Project Engineering 74

Netflix Tech

MARCH 10, 2021

Access the AWS console ( docs , talk , demo ) ConsoleMe allows users to access the AWS console through the use of temporary IAM role credentials. Retrieve and serve short-lived AWS credentials through Weep ( docs , talk ) Weep is ConsoleMe’s CLI utility. Users have a number of ways they can log in to the AWS console.

AWS 100

AWS Accessible Accessibility Cloud 100

Christophe Blefari

FEBRUARY 3, 2024

Even if you give your LLM access to the database, the codebase and the docs there is something the LLM does not have: the implicit (vocal) business rules that are written nowhere. But there is something that limits the LLM: his business understanding. conference.

MongoDB 130

MongoDB SQL Data Data Warehouse 130

Edureka

MAY 17, 2023

Google Docs : Google Docs is a word processing application that can be used to create and edit documents. These features include the ability to export and run code, the ability to generate images from text prompts, and the ability to integrate with other Google tools like Docs and Sheets.

Cloud Storage 52

Cloud Storage Education Government Accessible 52

Christophe Blefari

SEPTEMBER 15, 2023

First you need a great onboarding doc and then you need to successfully pass the "bootcamp" phase, which matches the 2 first weeks. ❤️ The key to building a high-performing data team is structured onboarding — The title say it all. Still in the article it mentions 2 key piece.

Data Warehouse 130

Data Warehouse Data SQL Python 130

Jesse Anderson

SEPTEMBER 14, 2023

As soon as people start using LLMs on a daily basis in Gmail and Google Docs, they’re going to expect it. Users won’t have to worry about starting GPT or another program to interact with an LLM. The metric I use for technology adoption is, what would people say if it were to disappear tomorrow? Think of autocompletes. We expect them.

Data Engineering 147

Data Engineering Data Engineer Engineering NoSQL 147

Christophe Blefari

JULY 3, 2023

LakehouseIQ is a way to use your Enterprise signals (org charts, lineage, docs, queries, catalog, etc.) The CEO of Databricks was on stage and use words that I like, he says data should be democratise to every employee AI should be democratise in every product Databricks vision about LLMs (in Wed. to contextualise LLMs used in UI assistants.

SQL 130

SQL Data Kafka AWS 130

dbt Developer Hub

FEBRUARY 15, 2023

C: To prepare for the exam I reviewed the official dbt Certification Study Guide and the official dbt docs , and attended group study and learning sessions that were hosted by Montreal Analytics for all employees interested in taking the exam. Additionally, I reviewed the Certification Study Guide and attended group learning sessions.

Certification 119

Certification Consulting BI SQL 119

Knowledge Hut

MARCH 27, 2024

Google Docs Google Search Google Maps Gmail Google Play Store I recommend you obtain a Web Design and Development course as a software engineer. Questions such as how you would design Google Docs, Google’s database for web indexing, Google Home, or Google Search play an integral part in the interview process.

Software Engineer 52

Software Engineer Software Engineering Engineering Programming Language 52

ThoughtSpot

AUGUST 22, 2023

However, we do make self-service resources available through our thorough Docs , Community , and eLearning resources for those who prefer to work solo. The entire mission behind providing this support is taking the burden off of you—the everyday user.

Data 80

Data Designing Accessible Accessibility 80

Cloudera

OCTOBER 28, 2020

Because of the way Solr generates its logs, a schema similar to the following is adopted. {. "name": "docs", "namespace": "doc", "type": "record", "fields": [ {. "name" : "field", "type" : {. "type" : "array", "items" : {. "type" : "record", "name" : "record_tag", "namespace" : "name", "fields" : [.

Kafka 63

Kafka SQL Professional Services Consulting 63

Christophe Blefari

MARCH 17, 2023

On the other side Google announced the same for Google Docs and Gmail. Google and Microsoft will compete to include AI copilots in their offices suites — Microsoft announced 365 Copilot that will work in Word, Excel, Powerpoint and Outlook. Can we develop a GenAI that generates protests slogans?

Data 130

Data SQL Deep Learning Kafka 130

Grouparoo

APRIL 5, 2021

forEach ( function ( doc ) { Object. keys ( doc ). const docs = await db. for ( const doc of docs ) { for ( const [ key , value ] of Object. entries ( doc ) ) { if ( value ! const docs = await db. for ( const doc of docs ) { for ( const [ key , value ] of Object. my_collection_keys.

MongoDB 52

MongoDB Non-relational Database NoSQL Relational Database 52

Data Engineering Weekly

MARCH 19, 2023

link] Hiflylabs: dbt Docs as a Static Website I often joke, “This data catalog tool could be the static website out of dbt docs.” ” The blog narrates how to build a data catalog without spending money on dbt docs!!! link] All rights reserved ProtoGrowth Inc, India.

Data Engineering 57

Data Engineering Data Engineer Engineering Media 57

Grouparoo

FEBRUARY 17, 2021

get ( url + ` /docs/config ` ) ; expect ( await getSessionItem ( "prevPath" ) ). toBe ( "/docs/config" ) ; await browser. toBe ( "/docs/config" ) ; expect ( await getSessionItem ( "currentPath" ) ). toBe ( "null" ) ; expect ( await getSessionItem ( "currentPath" ) ).

IT 52

IT Data 52

Towards Data Science

JUNE 19, 2023

Let’s see how to make these host connectors available in a Meerschaum project. In the compose file, all of the connectors we need for our project are defined under config:meerschaum:connectors.

PostgreSQL 82

PostgreSQL SQL Python Project 82

Data Engineering Podcast

MARCH 25, 2023

To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Grainite Blog about the challenges of streaming architectures Getting Started Docs BigTable Spanner Firestore OpenCensus Citrix NetScaler J2EE RocksDB Pulsar SQL Server MySQL RAFT Protocol The intro and outro music is from The Hug by (..)

MySQL 130

MySQL Python Architecture Machine Learning 130

Cloudera

MAY 5, 2021

Please refer to this doc to learn how to define TaskGroups. If you want to understand more about the design details, you can find the design doc here. The TaskGroups type is self-explanatory, each taskGroup represents a “gang” for the application, which is a group of homogenous pod requests.

Metadata 135

Metadata Algorithm Big Data Machine Learning 135

Datakin

OCTOBER 14, 2021

These are most conveniently found in Docs page of your Datakin instance. Once there, you will see two lines of code that look similar to these: export OPENLINEAGE_URL=[link] export OPENLINEAGE_API_KEY={{YOUR_API_KEY}} Run these two export commands, making sure to replace the {{ TOKENS }} if you didn’t copy and paste them from the docs.

Google Cloud 52

Google Cloud Datasets Bytes Metadata 52

Grouparoo

MARCH 3, 2021

Case Insensitive Sting Comparisons Postgres supports both the like and iLike operators for comparing strings, with the i indicating case-insensitive matching ( Postgres Docs ). Instead, if you really want your like function to be made case-sensitive, you would use the case_sensitive_like PRAGMA ( SQLite Docs ).

SQL 52

SQL Database Relational Database Coding 52