dbt Developer Hub

AUGUST 2, 2023

For Postgres/Redshift/Databricks {{ config ( materialized = 'materialized_view' , ) }} For Snowflake: {{ config ( materialized = 'dynamic_table' , ) }} note We are only supporting dynamic tables on Snowflake, not Snowflake’s materialized views (for a comparison between Snowflake Dynamic Tables and Materialized Views, refer docs.

Datasets 52

Datasets Kafka SQL Cloud 52

dbt Developer Hub

JANUARY 23, 2023

This led us to have two main classes of tests — Data Integrity Tests (called Generic Tests in dbt docs) and Context Driven Tests (called Singular Tests in dbt docs), with varying levels of severity across both test classes. Context Driven Tests are more complex and look a lot more like models. The reason for this is twofold.

Metadata 52

Metadata High Quality Data SQL Data Integration 52

Zalando Engineering

APRIL 24, 2024

Deep Dive The actual implementation of the mock server follows similar suggestions from the official docs of mirage.js const isMockServerEnabled = config. The only additional step required is the setup of these edge cases that features could potentially exhibit in the form of multiple mocked data sets for the mock server to consume.

Designing 62

Designing Coding Accessible Accessibility 62

Netflix Tech

MARCH 10, 2021

Access the AWS console ( docs , talk , demo ) ConsoleMe allows users to access the AWS console through the use of temporary IAM role credentials. Retrieve and serve short-lived AWS credentials through Weep ( docs , talk ) Weep is ConsoleMe’s CLI utility. Users have a number of ways they can log in to the AWS console.

AWS 100

AWS Accessible Accessibility Cloud 100

Zalando Engineering

JANUARY 22, 2024

Predictably, this resulted in an overload of Google Docs that limits the concurrent editors for the document who were working on the Post Mortem. As the first incident of its kind and with a large scale impact for our customers, it got high attention across the organization.

AWS 83

AWS Metadata Cloud Process 83

Tweag

OCTOBER 4, 2023

Topiary uses the excellent clap command line argument parser library, which has a feature to derive the parser directly from your types and their annotations (such as doc comments). topiary config outputs the computed configuration, as TOML. config/topiary/languages.toml # 3. All my other CLI modernisation work stems from here.

Coding 63

Coding Designing Engineering IT 63

Grouparoo

DECEMBER 20, 2021

We’ve continued to make this workflow better, introducing tools like our Config UI , which makes it easy to configure Grouparoo through a visual interface, in context with your data, while still outputting declarative configuration in the form of JSON files. If you want to learn more, please check out our docs.

Cloud 52

Cloud Software Engineer Software Engineering Coding 52

Grouparoo

MARCH 3, 2021

Case Insensitive Sting Comparisons Postgres supports both the like and iLike operators for comparing strings, with the i indicating case-insensitive matching ( Postgres Docs ). Instead, if you really want your like function to be made case-sensitive, you would use the case_sensitive_like PRAGMA ( SQLite Docs ).

SQL 52

SQL Database Relational Database Coding 52

Knowledge Hut

MAY 16, 2024

While on the iOS emulator run: npm run ios Expo has docs that you can reference for solutions or even utilize the Expo Forums created for the Expo community to ask questions. First, you need to lock the version you build with to ensure the app still works even after updates, using the following command: npm config set save-exact=true 2.

Java 98

Java Utilities Coding Project 98

Grouparoo

DECEMBER 13, 2021

Most of these files can be generated using our Config UI. You can either copy/paste the app configuration from our app-example-dbt repo and update it with the id, name, and query you would like, or you can utilize Config UI to manually configure your credentials, environment, and refresh query. status ~ " node: " ~ res.

Metadata 52

Metadata Data Warehouse SQL Utilities 52

Towards Data Science

FEBRUARY 19, 2024

As we saw in the script, the job needs to access both the images to embed, as well as Hugging Face to download the model configs and weights. A trust policy that allows the EMR jobs to access other AWS services.

AWS 61

AWS Building Bytes Python 61

dbt Developer Hub

AUGUST 2, 2022

Define our rules and create a pre-commit config file ​ pre-commit-dbt provides a range of tests that can be run on the models of our dbt project. As described before, we need to run a dbt docs generate in order to create updated JSON artifacts used in the pre-commit hooks. To remediate this let’s follow those 4 steps: 1.

Python 52

Python SQL Data Warehouse Project 52

Cloudera

JUNE 25, 2021

The Replication Manager support matrix is documented in our public docs. Configure the required ports to enable connectivity from CDH to CDP Public Cloud (see docs for details). This step involves checking the following configs on source: . For context, the setup used is as follows. Pre-Check: CDH Cluster.

Cloud 69

Cloud Data Lake Cloud Storage Metadata 69

dbt Developer Hub

APRIL 30, 2023

dbt brought code-based tests, integrated CI, efficient development with packages, and global docs. When dbt was first developed, it aimed to bring the best practices in software engineering to the field of analytics—this meant version-controlled, rigorously tested, and collaborative data transformations.

Data Engineering 64

Data Engineering Data Engineer Engineering Finance 64

Scott Logic

APRIL 2, 2024

The package docs, combined with the documentation for axe-core , should help you configure it to your needs. Then it’s just a case of installing the package and updating your Playwright config. By default, it will check against the latest W3C standards, as well as best practices, but you can reduce the requirements if necessary.

Accessible 52

Accessible Accessibility IT Coding 52

Towards Data Science

JUNE 2, 2023

In the example, we tagged our test with two tags: {{ config(tags=['unit-test', 'unit-tests']) }} The first one is a boilerplate tag required by the dbt-unit-testing library. Be also aware of other limitations listed in the library docs. But we want to only run our unit tests.

Coding 73

Coding SQL Software Engineer Software Engineering 73

dbt Developer Hub

MAY 18, 2022

To implement this distribution on our tables in dbt, we would apply this configuration to each of our models: { { config ( materialized = 'table' , dist = 'all' ) } } Here's a visualization of the data stored on our nodes: When to use the all distribution : This type of distribution is great for smaller data which doesn’t update frequently.

Raw Data 59

Raw Data Consulting Datasets IT 59

Rockset

JANUARY 16, 2019

config/connect-standalone.properties , which contains the configuration required for Kafka Connect. config/connect-rockset-sink.properties to configure Rockset-specific properties. config/connect-rockset-sink.properties to configure Rockset-specific properties. config/connect-standalone.properties./config/connect-rockset-sink.properties

Kafka 40

Kafka SQL Python Data 40

dbt Developer Hub

NOVEMBER 28, 2021

For more information about defining sources, take a look at the Sources page in the dbt docs. It ensures that the schemas are included in catalog.json when dbt docs generate is run, which is critical for collecting and tracing data lineage. So it's necessary to do so before running models: % dbt docs generate Running with dbt=0.21.0

Raw Data 52

Raw Data Metadata Database Datasets 52

Grouparoo

MAY 11, 2021

If you're interested in learning more about them for your project, there's a detailed table on this page in the docs. Take a gander through those files (they're under the config subfolder, under the various types of things we just created). Every time you see an ellipsis (that's a.

Data Pipeline 52

Data Pipeline Data Project Utilities 52

Big Data Tools

FEBRUARY 9, 2022

While our plugin does this in the UI right inside the IDE, SeaTunnel works on a different scale and provides users with a way to describe synchronization config declaratively. For example, this is the case for PostgreSQL, and this behavior is even described in the docs. Future improvements Data engineering tools are evolving every day.

Data Engineering 52

Data Engineering Data Engineer Engineering Big Data Tools 52

Big Data Tools

FEBRUARY 9, 2022

While our plugin does this in the UI right inside the IDE, SeaTunnel works on a different scale and provides users with a way to describe synchronization config declaratively. For example, this is the case for PostgreSQL, and this behavior is even described in the docs. Future improvements Data engineering tools are evolving every day.

Data Engineering 52

Data Engineering Data Engineer Engineering Big Data Tools 52

Ripple Engineering

FEBRUARY 2, 2021

After you add the.gitlab-ci.yml file and push it to your repo, you will see the pipeline icon initiate with the contents of your config YAML file. hooks: - id: seed-isort-config - repo: [link] rev: v4.3.21 seed-isort-config - Statically populates the known_third_party isort setting in Python. flake8 - Enforces a pycodestyle.

Building 52

Building Python Google Cloud Coding 52

Cloudera

SEPTEMBER 28, 2020

A new tool ‘fs2cs’ CLI tool is provided to do basic queue structure migration (there are different behaviors between CS and FS, the fs2cs tool only converts part of configurations, see doc), upload to new CM. Atlas ships with a default heap size of 2GB as specified in this Config changes to increase memory configurations resolved the issue.

Cloud 131

Cloud Kafka Professional Services Metadata 131

Zalando Engineering

NOVEMBER 19, 2023

And even if you were to gather all the information from the docs, it's still not enough. Of course, a lot of these files are the configs and tests and test resources, so the actual number of Scala files is much lower. In Zalando, the size of data is pretty massive. This just goes deeper into the rabbit hole, doesn't it?)

Scala 86

Scala Java Coding IT 86

DoorDash Engineering

MARCH 7, 2023

Per their config suggestions, we set an environment variable that connects LocalStack to Docker and a couple of volumes, one of which is required for Docker connectivity while the other specifies where LocalStack should store its data. The configuration we’re using is based on the recommended configuration from LocalStack.

AWS 119

AWS PostgreSQL Database SQL 119

dbt Developer Hub

JUNE 19, 2020

Are you tagging individual models in config blocks? Useful links Packages Docs Package Hub dbt utils package ✅ Code style ​ Do you have a clearly defined code style? Useful links dbt release version Sources Refs tags Jinja docs ✅ Testing & Continuous Integration ​ Do your models have tests? Are you using YAML selectors?

Project 40

Project BI SQL Utilities 40

Confluent

APRIL 24, 2019

The examples you see in the docs are great for getting started. The figure below illustrates the high-level design of the Test Machine: Tests as data. Test authors shouldn’t be concerned with the mechanics of writing test data to Kafka.

Systems 94

Systems Kafka Transportation Coding 94

Mutt Data

APRIL 11, 2023

Read the docs and you will find most use cases explained. Which is why we used that config in the next test. FastAPI Prometheus instrumentator We used prometheus-fastapi-instrumentator to add a /metrics endpoint which exports metrics in a prometheus-friendly format. We had to add latency and requests metrics, but it was a breeze.

Utilities 40

Utilities Python Deep Learning AWS 40

Rockset

SEPTEMBER 21, 2021

Since this was a simple config change, it didn’t require a code deploy and was easy to get out. If you're unfamiliar with how condition variables work, check out the java docs ; most languages will have some sort of condition variable implementation as well. 1 network call, and fewer wasted tokens. private final long batchSize =.;

Systems 52

Systems Java Coding Metadata 52

Cloudera

OCTOBER 15, 2020

"zkConnected":true, "status":0, "QTime":0, "params":{. "q":"*:*", "doAs":"<querying user>", "_forwardedCount":"1", "_":"1599835760799"}}, "response":{"numFound":0,"start":0,"docs":[]. }}. Configuration: Config name. Config value. Configuration: Config name. Config value.

AWS 113

AWS Data Cloud Cloud Storage 113

Rock the JVM

JUNE 20, 2022

scripts.sql volumes: db: driver: local Next, we can add the database configurations to the config file such as application.conf. There are many other ways to connect to the database, which can be found in the docs. Use any free online services such as ElephantSQL A sample docker-compose.yaml file looks like this: version: '3.8'

Scala 40

Scala PostgreSQL Database SQL 40

ProjectPro

DECEMBER 10, 2021

Source: Microsoft Azure docs By launching a cloud service instance, Azure cloud services can be utilized to implement multi-tier web-based apps in Azure. Blob storage is suitable for sharing images/docs directly to a browser, storing files meant for multiple access, streaming audio/video files, backing up data, creating log files, etc.

BI 52

BI Cloud Computing SQL Database 52

Cloudera

FEBRUARY 28, 2024

You can find a more in-depth explanation of pull mode in the official docs. Specify replications with Streams Replication Manager’s Replication Configs. Pull mode is recommended as it is the deployment type that was found to provide the highest amount of resilience against various timeout and network instability issues.

Kafka 68

Kafka Management Big Data Architecture 68

Cloudera

SEPTEMBER 13, 2021

The producer properties file is configured to access the broker, it at least has to contain bootstrap.servers config. So following the docs I added a new Kafka Broker role instance but this broker is empty at this point. This config only marks the broker dead and triggers an alert. Adding Brokers. In Cloudera Manager 7.4.3

Kafka 70

Kafka Bytes Utilities Management 70