Zalando Engineering

NOVEMBER 28, 2018

Introducing the Graph Browser Algorithm The Graph Browser algorithm enables browsing on a graph by generating a unique set of Navigation Tags for a product of interest on the graph, which we call the anchor product. To explain further we provide a diagram of the process in figure 2.

Algorithm 40

Algorithm Computer Science Python Big Data 40

Snowflake

MARCH 30, 2023

Collecting, cleaning, and organizing data into a coherent form for business users to consume are all standard data modeling and data engineering tasks for loading a data warehouse. Feature engineering: Data is transformed to support ML model training. “The features you use influence more than everything else the result.

Engineering 84

Engineering Raw Data Data Science Scala 84

Data Engineering Weekly

JUNE 4, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. Sign up free to test out the tool today. Can we test SQL business logic during the development phase itself?

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Knowledge Hut

MARCH 13, 2024

Wondering what is a big data engineer? Let me introduce you to who is big data engineer and what does a big data engineer do. Who Is a Big Data Engineer? A big data engineer is often confused with a data engineer because they work with data. What Does A Big Data Engineer Do?

Big Data 52

Big Data Data Engineering Data Engineer Engineering 52

Knowledge Hut

MARCH 13, 2024

Wondering what is a big data engineer? Let me introduce you to who is big data engineer and what does a big data engineer do. Who Is a Big Data Engineer? A big data engineer is often confused with a data engineer because they work with data. What Does A Big Data Engineer Do?

Big Data 52

Big Data Data Engineering Data Engineer Engineering 52

Netflix Tech

AUGUST 10, 2022

A Validation Framework For Unreal Engine By Adam Davis, Jimmy Fusil, Bhanu Srikanth and Girish Balakrishnan Game Engines in Virtual Production The use of Virtual Production and real time technologies has markedly accelerated in the past few years. Virtual Production?—?A

Engineering 95

Engineering Utilities Manufacturing Project 95

Knowledge Hut

JUNE 20, 2023

A novice data scientist prepared to start a rewarding journey may need clarification on the differences between a data scientist and a machine learning engineer. Let's find out the differences between a data scientist and a machine learning engineer below to make an informative decision.

Machine Learning 52

Machine Learning Data Engineering Data Engineer Engineering 52

Knowledge Hut

DECEMBER 26, 2023

Various computer systems and applications are designed and created by software engineers to solve real-world problems. The software engineer, also called the software developer, is responsible for developing software for applications and computers. The role of software engineers involves more than just developing their own systems.

Software Engineer 52

Software Engineer Software Engineering Engineering Programming Language 52

The Pragmatic Engineer

MARCH 3, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. Problems were modeled off a real Google engineering challenge. This means that two photos can, but don't have to, have tags in common.” We cover one out of five topics in yesterday's subscriber-only The Scoop issue.

Coding 169

Coding IT Software Engineer Software Engineering 169

DoorDash Engineering

AUGUST 1, 2023

Challenges Faced With StatsD StatsD was a great asset for our early observability needs, but we began encountering constraints such as losing metrics during surge events, difficulties with naming/standardized tags, and a lack of reporting tools. These common tags are useful to create common dashboards and alerts to monitor service health.

AWS 82

AWS Transportation Programming Language Government 82

Knowledge Hut

SEPTEMBER 22, 2023

In the clamour for data scientists, nobody, however, emphasizes the importance of a data engineer. One of the most sought-after career positions in the Microsoft Azure ecosystem is an Azure data engineer. The information that I will share in this article will serve as an illustration of a manual for Azure data engineer certification.

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Knowledge Hut

OCTOBER 29, 2023

The user computer is a client and google search engine act as server. HTML is composed of various tags which defines the document and it's semantic. Semantic add meaning to the content as expressed by enclosing tags and it is not purely for visual presentation purpose. Then we need to use < p > tag.

Project 98

Project Programming Language Java Coding 98

Rockset

APRIL 18, 2023

We’re excited to introduce vector search on Rockset to power fast and efficient search experiences, personalization engines, fraud detection systems and more. Before vector search, search experiences primarily relied on keyword search, which frequently involved manually tagging data to identify and deliver relevant results.

Unstructured Data 105

Unstructured Data Metadata Machine Learning SQL 105

Christophe Blefari

MARCH 1, 2023

dbt is a development framework that combines modular SQL with software engineering best practices to make data transformation reliable, fast, and fun. It was the previous tag line dbt Labs had on their website. Jinja templating — Jinja is a templating engine that seems to exist forever in Python.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Knowledge Hut

SEPTEMBER 29, 2023

This growth is creating a strong demand for data experts, especially Azure data engineers. But who are Azure data engineers, and what do they do? Moreover, what benefits can you expect from a career in Azure Data Engineering? Join us on this journey through the exciting realm of Azure Data Engineering.

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Cloudera

OCTOBER 11, 2021

Modak, a leading provider of modern data engineering solutions, is now a certified solution partner with Cloudera. Customers can now seamlessly automate migration to Cloudera’s Hybrid Data Platform — Cloudera Data Platform (CDP) to dynamically auto-scale cloud services with Cloudera Data Engineering (CDE) integration with Modak Nabu.

Data Engineering 92

Data Engineering Data Engineer Cloud Engineering 92

Monte Carlo

MARCH 12, 2024

For example, some of the first questions data engineers want to know following an incident are: What DAG or workflow is populating this table? Now the data engineer can restart the DAG or take whatever additional steps may be required to resolve the issue. What was the result and duration of the recent task runs for this DAG?

BI 52

BI Metadata Data Pipeline Data Engineering 52

The Pragmatic Engineer

SEPTEMBER 22, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. Bun was mostly built by Jared Sumner , a former Stripe engineer, and recipient of the Thiel Fellowship (a grant of $100,000 for young people to drop out of school and build things, founded by venture capitalist, Peter Thiel). world by storm.

Programming Language 237

Programming Language Coding Project Engineering 237

Tweag

APRIL 26, 2023

Defunctionalization — replacing functional arguments in higher-order functions with tags denoting the context of those functions (Wikipedia has a good article on this). Tags or strings In the above, the different transformations are denoted by different types ( data Prenex , for instance).

Data Process 98

Data Process Process Programming Data 98

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. As authors of the Engineering Blog will (mostly) be software engineers, we decided to switch to a git-based workflow and a static site generator. Article tags should be curated via an explicit allowlist. scripts/new-post.py

Engineering 40

Engineering Bytes AWS Python 40

Towards Data Science

JUNE 2, 2023

How to improve the code quality of your dbt models with unit tests and TDD All you need to know to start unit testing your dbt SQL models Photo by Christin Hume on Unsplash If you are a data or analytics engineer, you are probably comfortable writing SQL models and testing for data quality with dbt tests.

Coding 74

Coding SQL Software Engineer Software Engineering 74

RandomTrees

MARCH 22, 2024

Tagging and Categorization : Delimited string columns are commonly used for tagging or categorizing data. By splitting these tags into rows, we can analyze the distribution of tags, identify trends, and perform targeted analysis based on specific tags.

Media 52

Media Healthcare Datasets Electronics 52

Knowledge Hut

DECEMBER 29, 2023

A Front-End Developer is a type of Software Engineer who works on front-end development. This markup language is made up of tags that label or indicate what the content should appear as. The HTML opening tag (<html>) and the HTML closing tag (</html>) indicate the start and end of a HTML structure.

Portfolio 98

Portfolio Amazon Web Services Programming Language Coding 98

Monte Carlo

AUGUST 16, 2022

Just as an engineer would never deploy code to production without testing it first, or run production software without application performance monitoring and observability, data teams can apply a similar “testing + observability” framework to their data pipelines. Fortunately, there’s a better way forward.

Engineering 52

Engineering Data Lake Retail BI 52

Towards Data Science

MAY 26, 2023

In Hamilton, the way to add extra metadata is through the use of the ` @tag ` and ` @tag_outputs ` decorators. This provides flexibility and the ability for you and your organization to define tags and values that make sense for your context. Use @tag to annotate functions. """ #. contents of function not important.

Metadata 87

Metadata Data Science Coding Machine Learning 87

Cloudera

OCTOBER 18, 2022

This was envisioned as a one-stop solution to serve the different personas around cloud cost awareness: from senior leaders down to the frontline engineer. The solution was prototyped in Cloudera Data Science Workbench (CDSW) , and is built using Python and PySpark, which is scheduled using Cloudera Data Engineering.

Cloud 87

Cloud IT Data Warehouse AWS 87

Hepta Analytics

FEBRUARY 14, 2022

We looked at the following: How do we ingest – ETL vs ELT Where do we store the data – Data lake vs data warehouse Which tool to we use to ingest – cronjob vs workflow engine NOTE : This weeks task requires good internet speed and good compute. Replace the image tag with the lines below. It is called Idempotency.

Data Ingestion 52

Data Ingestion Data Engineering Data Engineer Engineering 52

Monte Carlo

MAY 20, 2024

User Annotations and Tags : Collaborative input from users such as comments, usage tips, and tags to improve discoverability and utility. It’s an essential resource for database administrators, data engineers, and developers who need to ensure that the data within a specific system or database is used consistently and correctly.

Datasets 52

Datasets Government Data Business Analyst 52

Towards Data Science

MARCH 1, 2023

Similarly, our flatMapWithGroupState will accumulate tags (evaluated true/false Sigma expressions) and later release them. Our Flux Capacitor function is easy to configure and let’s the user specify how and when each individual tag is stored and retrieved. This evaluator is a no-op, it simply passes the current tag value through.

Designing 76

Designing Scala Data Science Process 76

LinkedIn Engineering

JULY 27, 2023

Costwiz creates accountability by notifying organization owners to assign these recommendations to engineers or SREs and tracking the workflow of a recommendation. These tools expose the right set of data without us having to worry about engineering efforts for dedicated reporting UI or a workflow for email alerts and reporting dashboards.

Metadata 75

Metadata Utilities Cloud Data Lake 75

Engineering at Meta

FEBRUARY 6, 2024

config: github/workflows/dotslash-config.json # Tag for the release to to target. config: github/workflows/dotslash-config.json # Tag for the release to to target. The other required input is the ID of the release, which in GitHub, is defined by a Git tag. We look forward to hearing from you!

Metadata 117

Metadata Coding Building Project 117

Snowflake

NOVEMBER 13, 2023

After consulting with Atlan, Tide decided to use Atlan Playbooks to automatically identify, tag and classify data. With items from Snowflake appropriately tagged, Tide could mask PII such as account numbers, email addresses and phone numbers in the Atlan UI.

Finance 87

Finance Consulting Data Warehouse Data Governance 87

Netflix Tech

FEBRUARY 9, 2021

Growth Engineering at Netflix?—?Automated In the Growth Engineering team, we refer to this as the top of the signup funnel. For more background on the signup funnel and Growth Engineering’s role in the signup funnel, please read our initial post on the topic: Growth Engineering at Netflix? Growth Engineering at Netflix?—?Automated

Engineering 66

Engineering Entertainment Metadata Designing 66

Zalando Engineering

OCTOBER 18, 2023

Certain directives are built into GraphQL, like @skip , @include , @deprecated , and @specifiedBy , and are supported by all GraphQL engines. In the second step, after execution, we match the error paths and omit the error tag for those extracted paths. See our blog post GraphQL persisted queries and schema stability for more details.

Metadata 86

Metadata Coding Banking Designing 86

Engineering at Meta

FEBRUARY 7, 2023

Recently, we partnered with the Daylight team at Meta to create Earth Tables, an extension of Daylight that simplifies OSM tags. We parsed OSM’s complicated building and building:part tags to refashion our building features from the ground up. The post Improving Meta’s global maps appeared first on Engineering at Meta.

Entertainment 135

Entertainment Transportation Data Schemas AWS 135

Netflix Tech

JUNE 14, 2023

By the summer of 2020, many UI engineers were ready to move to GraphQL. The GraphQL shim enabled client engineers to move quickly onto GraphQL, figure out client-side concerns like cache normalization, experiment with different GraphQL clients, and investigate client performance without being blocked by server-side migrations.

Utilities 141

Utilities Java Systems AWS 141

Lyft Engineering

OCTOBER 3, 2023

This makes it easy for engineers to generate, transform, store, and analyze data at the petabyte scale. The lack of centralization in data quality also made the data discovery process inefficient, making it hard for data scientists and data engineers to identify trustworthy data. Yet this has come at the cost of quality.

Big Data 84

Big Data Metadata Data Warehouse Data 84