Simon Späti

OCTOBER 19, 2022

Will Rust kill Python for Data Engineers? Rust may not replace Python outright, but it has consumed more and more of JavaScript tooling and there are increasingly many projects trying to do the same with Python/Data Engineering. If you only came here to know this, my answer is no. Betteridge’s Law strikes again!

Data Engineering 130

Data Engineering Data Engineer Engineering Programming Language 130

Confessions of a Data Guy

JULY 7, 2023

With the rise of Rust, new tools like DuckDB, Polars, and whatever else, things do seem to shifting at a fundamental level. It seems like there is someone at the base of a titering rock with a crowbar, picking and […] The post Ballista (Rust) vs Apache Spark. A Tale of Woe. appeared first on Confessions of a Data Guy.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

Confessions of a Data Guy

JANUARY 13, 2023

Rust has been on my mind a lot lately, probably because of Data Engineering boredom, watching Spark clusters chug along like some medieval farm worker endlessly trudging through the muck and mire of life.

Data Pipeline 147

Data Pipeline Data Engineering Data Engineer Data 147

Confessions of a Data Guy

OCTOBER 1, 2023

So perhaps you’re thinking it’s time to use Rust on your next project.

Project 130

Project IT Data Data Engineering 130

KDnuggets

OCTOBER 13, 2023

A new deep learning framework built entirely in Rust that aims to balance flexibility, performance, and ease of use for researchers, ML engineers, and developers.

Deep Learning 127

Deep Learning Engineering Machine Learning 127

Confessions of a Data Guy

APRIL 6, 2023

The post QuickSort in Rust! appeared first on Confessions of a Data Guy.

Data 130

Data 130

Tweag

JULY 26, 2023

The vast majority of the Rust projects are using Cargo as a build tool. Cargo is great when you are developing and packaging a single Rust library or application, but when it comes to a fast-growing and complex workspace, one could be attracted to the idea of using a more flexible and scalable build system.

Building 84

Building Metadata Coding Project 84

Tweag

AUGUST 9, 2023

In this blog post, I’ll take you on a deep dive into this performance-enhancing strategy using a real-world Rust project as an example. Are you ready to turbocharge your Rust static executables? Creating Rust static executables based on musl Let’s explore how to compile wasmtime into a static Linux executable that links with musl.

Utilities 90

Utilities Building Coding Project 90

Workfall

FEBRUARY 7, 2023

Reading Time: 6 minutes While Rust is known for its backend web development capabilities, the introduction of WebAssembly (Wasm) has enabled the development of rich front-end apps in Rust. Yew is a Rust Framework for creating web applications that compile to the WebAssembly target that is meant to run on the browser.

Building 87

Building PostgreSQL Coding AWS 87

Workfall

MAY 16, 2023

Reading Time: 5 minutes When Rust compiles code, you get an executable if you created the application using the --bin command. Conclusion In this blog, we looked at how we can deploy a Rocket Rust REST API to an AWS EC2 instance using GitHub Actions and Docker. AWS is in love with RUST and you should be too!

AWS 81

AWS Cloud Computing Coding Cloud 81

Workfall

MAY 2, 2023

Reading Time: 5 minutes After you’ve finished developing your Yew Rust application, it’s time to make it available to your users. This blog will go over how to launch your Yew Rust application on AWS EC2. Conclusion In this blog, we covered deploying a Yew Rust application to AWS EC2 using GitHub Actions and Nginx.

AWS 55

AWS Cloud Certification Utilities 55

Christophe Blefari

FEBRUARY 16, 2024

uv: Python packaging in Rust — I've been using poetry in the last 2 years and I'm quite satisfied about it. sqruff: SQL linter written in Rust — This is the results of the Rust hype, people are now porting more and more tools in Rust for efficiency. And it's for the better.

Food 130

Food Data SQL Government 130

Scott Logic

DECEMBER 7, 2022

The chip can be programmed in a number of different languages but I chose Rust for the hacker news karma. unwrap (); delay.delay_ms ( 500 ); } If you’re familiar with Rust, the calls to unwrap would typically indicate that set_high/low are fallible (can fail) and failure should explicitly be handled.

Building 52

Building Coding Programming Manufacturing 52

Seattle Data Guy

JANUARY 17, 2023

Are data engineers all learning Rust? 2022 is coming to an end. What is the state of data infra? Are Snowflake and Databricks still fighting over total cost of ownership? Is everyone switching to DuckDB? Let’s try to answer these questions.

Data Engineering 211

Data Engineering Data Engineer Engineering Data 211

Scott Logic

OCTOBER 18, 2023

If you want the TL;DR; here are the highlights: Rust and JavaScript usage is continuing to increase, but some more notable changes are happening a little further down - with both Swift and Zig seeing a significant increase in adoption. For the third year running, Rust is the most frequently used language for WebAssembly.

Programming Language 145

Programming Language Coding Java Datasets 145

Zalando Engineering

MARCH 27, 2019

Introducing Rust in an Enterprise Environment Discovery Sometime in 2013, on the internet I stumbled over a new programming language called Rust. When examining Rust, I found most of the features I used every day like Pattern Matching, the “New Type Pattern” and a “Scala like” Iterator API. That was of course correct.

Scala 40

Scala Programming Language Technology Utilities 40

Tweag

NOVEMBER 23, 2022

These examples will be in Rust, but similar trade-offs should apply to other languages compiled to WASM. Since Rust doesn’t have any tooling for checking non-blockingness (see here for some discussion), this might be difficult to ensure. and start running it concurrently.

Architecture 40

Architecture Coding Structured Data Process 40

Confessions of a Data Guy

MAY 7, 2023

It’s gotten so bad, I’ve started to use it in my Rust code on the side, Polars that is. Polars is one of those tools that you just want … no … NEED a reason to use it. I mean you have a problem if you could use Polars Python, and you find yourself using […] The post Polars – Laziness and SQL Context.

SQL 182

SQL Python Coding IT 182

Scribd Technology

JULY 19, 2021

By providing a foundational Delta Lake library in Rust, delta-rs can enable native bindings in Python, Ruby, Golang, and more.We will review what functionality delta-rs supports in its current Rust and Python APIs and the upcoming roadmap.

Python 40

Python Kafka Data Engineering Data Engineer 40

Engineering at Meta

OCTOBER 23, 2023

Fact1: Buck2 is written in Rust The core of Buck2 is written in Rust, with the rules written in Starlark (a Python-like language) and interpreted by our open-source Starlark library. C++, Rust, Python, OCaml, etc.) In our internal tests at Meta, we observed that Buck2 completed builds approximately 2x as fast as Buck1.

Programming Language 109

Programming Language Java Python Programming 109

Christophe Blefari

FEBRUARY 23, 2024

In order to do it you'll have to write Rust and Python code. This is a good way to enter the Rust world I guess. How you can write a Polars plugin — A dedicated website to explain how to write Polars plugins to extend the library capabilities.

Data Lake 130

Data Lake PostgreSQL MongoDB Scala 130

Christophe Blefari

JANUARY 26, 2024

SQLGlot switch to Rust — I really like SQLGlot, this is a SQL parser that gives you back the AST to do stuff. They ported the parser from Python to Rust and got 30-40% performance improvement. The leader submitted a solution running in less than 2s— have fun —while ClickHouse took 19s.

Google Cloud 130

Google Cloud Algorithm Data Java 130

Data Engineering Weekly

MARCH 31, 2024

Polars written in Rust offer 5-10X more performance than Pandas!! The trend is a simple example of the rise of Rust in building data infrastructure. The PyO3 native Python crate for Rust makes building Python binding much simpler, which increases the adoption of an architecture pattern using the mix of Rust and Python.

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Tweag

JULY 5, 2023

High-Level Differences Buck2 is a fresh rewrite in Rust. For example, starlark-rust is used by Buck2 to parse and evaluate Starlark, however, it is also used for Buck2 IDE integration, and could be useful for Bazel IDE integration as well. invoke build buck2 build //starlark-rust/starlark 6. Bazel recording steps: 1.

Systems 139

Systems Building Java Programming Language 139

KDnuggets

NOVEMBER 20, 2023

Ruff's 700+ built-in lint rules, reimplemented in Rust for speed, provide comprehensive linting and formatting to enforce clean and consistent Python code.

Coding 105

Coding Python Programming 105

Workfall

MARCH 7, 2023

Hands-on The code used in this blog is found at the following repository: [link] Dependencies and Installations First and foremost ensure that you have Rust installed. How to Create a REST API with Rust Rocket Framework and Diesel Middleware with PostgreSQL Database? AWS is in love with RUST and you should be too! 2): home.rs(3):

PostgreSQL 79

PostgreSQL Database AWS Data Collection 79

Scott Logic

JANUARY 3, 2024

I’d also noted that the Rust programming language seemed to be a popular choice for working with Wasm and that was something else I was keen to have a play with. With this in mind I set about creating an exploration project, the goal of which would be to write a program in Rust and get it to run in the browser. file via a bootstrap.js

Programming Language 52

Programming Language Coding Programming Project 52

Engineering at Meta

FEBRUARY 26, 2024

Instead of committing large, platform-specific executables to a repository, DotSlash combines a fast Rust program with a JSON manifest prefixed with a #! DotSlash takes the pain out of distributing binaries and toolchains to developers. to transparently fetch and execute the binary.

Software Engineer 91

Software Engineer Software Engineering Programming Engineering 91

Zalando Engineering

MAY 19, 2016

I recently started using Rust – the programming language. It was just a matter of time before I started experimenting with Rust. In this post I’ll talk about my experience on getting started with Rust, with a second post to follow that explains how to include it in a Docker image. Let’s dive into the existing code.

Scala 40

Scala Programming Language Java Programming 40

Data Engineering Podcast

JANUARY 28, 2024

Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com ) with your story. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.

Data Lake 147

Data Lake Building High Quality Data AWS 147

Christophe Blefari

NOVEMBER 18, 2022

— JOIN US My two cents about DuckDB Ok, right now, LinkedIn and Twitter data world are a bit going one-way down the Rust and DuckDB street. While I don't have any opinion on Rust except the fact it's look like a programming language eternal debate I'm bored of, I have one on DuckDB.

Python 130

Python Data Warehouse Data SQL 130

Data Engineering Weekly

FEBRUARY 18, 2024

link] Abhijeet Rathod: Rust: The Rising Star in Data Engineering 🦀 Embracing the Power of Performance, Safety, and Elegance We started to see more and more tools, especially around Python rewritten in Rust, with much better performance.

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Christophe Blefari

JANUARY 14, 2023

Polars—Pandas are freezing Recently influencers are betting that Rust will be the de-facto language in data engineering. But with Rust the approach is different. And it makes sense, for instance ruff a Python linter that is build in Rust that claims to be extremely faster that the usual stuff. Enjoy the Data News.

Python 130

Python Kafka Data Scala 130

Tweag

JUNE 28, 2023

Topiary is a universal formatter engine written in Rust and packaged with Cargo. Packaging it in nixpkgs was a breeze because of nixpkgs’ support for Rust, but mostly because someone else did it for us , for which we have to thank our contributor figsoda. Except that Topiary is a Rust project, so how is it supposed to get into OPAM?

Building 52

Building Project Utilities Systems 52

Netflix Tech

FEBRUARY 26, 2024

The tool is written in Rust, leveraging the libbpf-rs and ratatui crates. This information is displayed in a top-like tabular format or a time series graph over a 10s moving window. Once bpftop terminates, it turns off the statistics-gathering function. Getting started Visit the project’s GitHub page to learn more about using the tool.

Programming 69

Programming Utilities Coding Designing 69

Christophe Blefari

MARCH 17, 2023

Data with Rust — This is a handbook that will showcase how to work in data engineering with Rust. This time it shows that everything can be done in SQL. With recursive CTEs he implemented sklearn linear model and the code is not even that big. At the moment only part 1 and 2 are written but it looks promising.

Data 130

Data SQL Deep Learning Kafka 130

Workfall

MARCH 21, 2023

Rewritten in Rust: SurrealDB is entirely written in Rust. Rust, as we know, has a small memory footprint and low CPU requirements. S upport for multiple different clients and libraries: SurrealDB supports the most popular programming languages, including JavaScript, TypeScript, Rust, Go, and others. src/main.rs(1):

PostgreSQL 68

PostgreSQL NoSQL Database Unstructured Data 68