Christophe Blefari

FEBRUARY 2, 2024

DuckDB adoption numbers are demonstrating a real trend behind the "hype" DuckDB docs website gets 500k unique visitors per month and DuckDB has a new shiny website. DuckDB announcements The Duck creators announced that v0.10.1 is coming soon and before end of July we might get the v1.0.0.

Datasets 130

Datasets Data Lake SQL Algorithm 130

Start Data Engineering

SEPTEMBER 29, 2021

dbt docs 3.7. Configurations and connections 3.2.1. profiles.yml 3.2.2. dbt_project.yml 3.3 Data flow 3.3.1. Source 3.3.2. Snapshots 3.3.3. Staging 3.3.4. Marts 3.3.4.1. Core 3.3.4.2. Marketing 3.4. dbt run 3.5. dbt test 3.6. Scheduling 4. Conclusion 5. Further reading 6. References 1.

Building 130

Building Data Project Engineering 130

Azure Data Engineering

JULY 16, 2022

For a detailed list of settings and sample JSON code, please visit the Microsoft Docs reference link below: Reference: [link] Service Principal: Service Principal of the data factory. User Assigned Managed Identity: User managed identity of the data factory.

Datasets 130

Datasets Certification Data Coding 130

Snowflake

MARCH 15, 2023

If you’re already an avid docs user, don’t worry—your bookmarks will continue to work. The docs site helps curate the latest features with homepage highlights and easy access to the Releases section. We have preserved all the existing URLs, ensuring a seamless transition for our loyal users.

Accessible 52

Accessible Accessibility Project Building 52

Christophe Blefari

SEPTEMBER 25, 2023

Astronomer released Ask Astro — A LLM application that is able to understand Astro docs to answer most of the Apache Airflow questions. The source code is on Github. The implications of scaling Airflow — Sarah, who's working at Prefect, wrote a post about Airflow downsides at scale and how Prefect mitigates them.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

Christophe Blefari

SEPTEMBER 25, 2023

Astronomer released Ask Astro — A LLM application that is able to understand Astro docs to answer most of the Apache Airflow questions. The source code is on Github. The implications of scaling Airflow — Sarah, who's working at Prefect, wrote a post about Airflow downsides at scale and how Prefect mitigates them.

Data 130

Data Data Warehouse Data Storage Data Pipeline 130

know.bi

MAY 10, 2023

In this post, we'll start from the existing how-to guide in the Apache Hop docs, but add a bit more context and goes into a bit more detail on how to get everything going. As we started doing early this year, this post was contributed to the Apache Hop docs as an extended Apache Airflow how-to guide.

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

KDnuggets

MARCH 29, 2023

Automate the Boring Stuff with GPT-4 and Python • Introduction to Python Libraries for Data Cleaning • Google Answer to ChatGPT by Adding Generative AI into Docs and Gmail • Top 15 YouTube Channels to Level Up Your Machine Learning Skills • 3 Mistakes That Could Be Affecting the Accuracy of Your Data Analytics

Python 73

Python Machine Learning Data Analytics Data 73

know.bi

SEPTEMBER 20, 2023

is available: Apache Beam upgrade, Google Dataflow docs and new transforms for Google Analytics 4 and Google Sheets Input and Output. Apache Hop 2.6.0

Google Cloud 59

Google Cloud Data Integration Data Engineering Data Engineer 59

Towards Data Science

DECEMBER 1, 2023

Store snapshots in a separate schema Take a while to generate dbt documentation using the “dbt docs generate” command. run “dbt docs serve” to open it in browser Serving —  Snowflake Dashboard Finally, visualize your transformed data using Snowflake Dashboards. Good documentation provides better data discoverability and governance.

Data Engineering 75

Data Engineering Data Engineer Project Engineering 75

Netflix Tech

MARCH 10, 2021

Access the AWS console ( docs , talk , demo ) ConsoleMe allows users to access the AWS console through the use of temporary IAM role credentials. Retrieve and serve short-lived AWS credentials through Weep ( docs , talk ) Weep is ConsoleMe’s CLI utility. Users have a number of ways they can log in to the AWS console.

AWS 100

AWS Accessible Accessibility Cloud 100

Christophe Blefari

FEBRUARY 3, 2024

Even if you give your LLM access to the database, the codebase and the docs there is something the LLM does not have: the implicit (vocal) business rules that are written nowhere. But there is something that limits the LLM: his business understanding. conference.

MongoDB 130

MongoDB SQL Data Data Warehouse 130

Edureka

MAY 17, 2023

Google Docs : Google Docs is a word processing application that can be used to create and edit documents. These features include the ability to export and run code, the ability to generate images from text prompts, and the ability to integrate with other Google tools like Docs and Sheets.

Cloud Storage 52

Cloud Storage Education Government Accessible 52

Christophe Blefari

SEPTEMBER 15, 2023

First you need a great onboarding doc and then you need to successfully pass the "bootcamp" phase, which matches the 2 first weeks. ❤️ The key to building a high-performing data team is structured onboarding — The title say it all. Still in the article it mentions 2 key piece.

Data Warehouse 130

Data Warehouse Data SQL Python 130

Jesse Anderson

SEPTEMBER 14, 2023

As soon as people start using LLMs on a daily basis in Gmail and Google Docs, they’re going to expect it. Users won’t have to worry about starting GPT or another program to interact with an LLM. The metric I use for technology adoption is, what would people say if it were to disappear tomorrow? Think of autocompletes. We expect them.

Data Engineering 147

Data Engineering Data Engineer Engineering NoSQL 147

Christophe Blefari

JULY 3, 2023

LakehouseIQ is a way to use your Enterprise signals (org charts, lineage, docs, queries, catalog, etc.) The CEO of Databricks was on stage and use words that I like, he says data should be democratise to every employee AI should be democratise in every product Databricks vision about LLMs (in Wed. to contextualise LLMs used in UI assistants.

SQL 130

SQL Data Kafka AWS 130

Knowledge Hut

MARCH 27, 2024

Google Docs Google Search Google Maps Gmail Google Play Store I recommend you obtain a Web Design and Development course as a software engineer. Questions such as how you would design Google Docs, Google’s database for web indexing, Google Home, or Google Search play an integral part in the interview process.

Software Engineer 52

Software Engineer Software Engineering Engineering Programming Language 52

ThoughtSpot

AUGUST 22, 2023

However, we do make self-service resources available through our thorough Docs , Community , and eLearning resources for those who prefer to work solo. The entire mission behind providing this support is taking the burden off of you—the everyday user.

Data 80

Data Designing Accessible Accessibility 80

Cloudera

OCTOBER 28, 2020

Because of the way Solr generates its logs, a schema similar to the following is adopted. {. "name": "docs", "namespace": "doc", "type": "record", "fields": [ {. "name" : "field", "type" : {. "type" : "array", "items" : {. "type" : "record", "name" : "record_tag", "namespace" : "name", "fields" : [.

Kafka 63

Kafka SQL Professional Services Consulting 63

Christophe Blefari

MARCH 17, 2023

On the other side Google announced the same for Google Docs and Gmail. Google and Microsoft will compete to include AI copilots in their offices suites — Microsoft announced 365 Copilot that will work in Word, Excel, Powerpoint and Outlook. Can we develop a GenAI that generates protests slogans?

Data 130

Data SQL Deep Learning Kafka 130

Data Engineering Weekly

MARCH 19, 2023

link] Hiflylabs: dbt Docs as a Static Website I often joke, “This data catalog tool could be the static website out of dbt docs.” ” The blog narrates how to build a data catalog without spending money on dbt docs!!! link] All rights reserved ProtoGrowth Inc, India.

Data Engineering 57

Data Engineering Data Engineer Engineering Media 57

Towards Data Science

JUNE 19, 2023

Let’s see how to make these host connectors available in a Meerschaum project. In the compose file, all of the connectors we need for our project are defined under config:meerschaum:connectors.

PostgreSQL 83

PostgreSQL SQL Python Project 83

Grouparoo

FEBRUARY 17, 2021

get ( url + ` /docs/config ` ) ; expect ( await getSessionItem ( "prevPath" ) ). toBe ( "/docs/config" ) ; await browser. toBe ( "/docs/config" ) ; expect ( await getSessionItem ( "currentPath" ) ). toBe ( "null" ) ; expect ( await getSessionItem ( "currentPath" ) ).

IT 52

IT Data 52

Cloudera

MAY 5, 2021

Please refer to this doc to learn how to define TaskGroups. If you want to understand more about the design details, you can find the design doc here. The TaskGroups type is self-explanatory, each taskGroup represents a “gang” for the application, which is a group of homogenous pod requests.

Metadata 135

Metadata Algorithm Big Data Machine Learning 135

Knowledge Hut

SEPTEMBER 21, 2023

Entry Level Resume Template (Doc) Senior Azure Cloud Practitioner resume template (Doc) Top 10 Skills for an Azure Cloud Practitioner Resume      It's crucial to highlight the abilities that are most pertinent to the position while creating your Azure Cloud Practitioner resume.

Cloud 52

Cloud Amazon Web Services Cloud Computing AWS 52

Scott Logic

DECEMBER 15, 2023

Contributing and code of conduct docs were added to ensure that contributing to the project is easy and clear, and that the repo can remain a safe space, free from harassment. Even adding your thoughts to the discussion threads on open issues is very much appreciated.

Manufacturing 59

Manufacturing Coding Project Accessible 59

Data Engineering Podcast

AUGUST 3, 2021

To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Links Hudi Docs Hudi Design & Architecture Incremental Processing CDC == Change Data Capture Podcast Episodes Oracle GoldenGate Voldemort Kafka Hadoop Spark (..)

Data Lake 130

Data Lake Data Warehouse Hadoop Architecture 130

dbt Developer Hub

AUGUST 2, 2022

As described before, we need to run a dbt docs generate in order to create updated JSON artifacts used in the pre-commit hooks. For that reason, we will need our CI step to execute this command, which will require setting up a profiles.yml file providing dbt the information to connect to the data warehouse.

Python 52

Python SQL Data Warehouse Project 52

Rockset

SEPTEMBER 25, 2020

What you’ll notice is there is a clause that states while cursor.alive: This while statement allows your code to keep checking to see if your cursor is still alive and doc references the different documents that captured the change in the oplog. first = oplog.find().sort('$natural', sort('$natural', pymongo.DESCENDING).limit(-1).next()

MongoDB 52

MongoDB Kafka Database NoSQL 52

Netflix Tech

MARCH 5, 2020

status, title, description, priority, etc,) and Google Doc and Google Drive for managing data itself. with FastAPI (including helper packages) VueJS UI Postgres We’re shipping Dispatch with built-in plugins that allow you to create and manage resources with GSuite (Docs, Drive, Sheets, Calendar, Groups), Jira, PagerDuty, and Slack.

Metadata 81

Metadata AWS Management Architecture 81

dbt Developer Hub

NOVEMBER 14, 2021

I've surrendered to just searching for the docs. So I made this handy 2 x 2 matrix to help sort the differences out: I am sorry - that’s just a blank 2x2 matrix. Standardizing your DATEADD SQL syntax with a dbt macro ​ But couldn’t we be doing something better with those keystrokes, like typing out and then deleting a tweet?

Data Warehouse 52

Data Warehouse SQL Software Engineer Software Engineering 52

dbt Developer Hub

FEBRUARY 7, 2022

Check out the dbt docs for the project for an explanation of the fields. Here’s the output of final jafflegaggle_facts table : Referring to the DAG from the dbt docs, you can see how we are already benefiting from merging at the user level for analytics information related to jafflegaggle_contacts.

Data Warehouse 52

Data Warehouse Datasets Data SQL 52

Ascend.io

JUNE 22, 2023

Get started with a free developer-tier Ascend Cloud environment and begin loading your data into MotherDuck today ( docs )! Ascend is thrilled to announce the availability of our newest feature: the ability to deliver data directly to the MotherDuck analytics platform!

NoSQL 52

NoSQL MySQL Google Cloud Data Pipeline 52

Monte Carlo

FEBRUARY 13, 2023

To dive into other new releases, check out our docs. This streamlined monitor creation workflow also makes it easy to specify the field, monitoring schedule, and relevant monitor documentation. Interested in learning more about how Monte Carlo’s Table Health Dashboard helps leading data teams improve data reliability at scale?

BI 52

BI Metadata Data Engineering Data Engineer 52

Knowledge Hut

MAY 2, 2024

Documentation links for python Python Doc Conclusion This article will help you with stepwise instructions on the installation of python on mac. You can create this file by a simple touch command, and this file does not need to have any data inside it, All it has to do is to exist inside the directory, for that to work as a package.

Python 52

Python Entertainment Medical Programming 52

Data Engineering Weekly

APRIL 28, 2023

Some exciting reads include patterns like an internal Wix Docs approach & integration of the documentation publishing as part of the CI/ CD pipelines.

Consulting 52

Consulting Kafka Lambda Architecture Engineering 52

Cloudera

FEBRUARY 21, 2023

We could just add the following fragment describing the new field and its time into the schema under the “Schema Definition” tab: { "name" : "data_timestamp", "type": "long", "doc": "Injected from a custom data transformation" } The last step is to change the Kafka row time to use the new row that we just created.

SQL 107

SQL Kafka Raw Data Data 107