How PyO3 revolutionizes the integration of Python and Rust. Discover how you can develop performant and secure applications with these tools that combine Python's flexibility with Rust's speed.

Pfuzzer: A Python fuzzy searcher built with Rust

The combination of Rust and Python is becoming increasingly popular - especially for performance-sensitive applications. With PyO3, Rust modules can be seamlessly integrated into Python, allowing you to combine Rust's speed and security with Python's flexibility.

A remarkable example of this synergy is Pfuzzer - a Python Fuzzy Search library based on the high-performance Rust library Nucleo. But Pfuzzer is more than just a practical solution for imprecise searches: It demonstrates how the PyO3 framework can be used to seamlessly integrate Rust code into Python.

In this article, you'll learn how Pfuzzer works, how PyO3 simplifies the development of Python modules in Rust, and how you can implement your own fuzzy search in Python with Rust.

Let's dive together into the world of PyO3 and discover how Rust and Python go Hand in Hand in projects like Pfuzzer!

Would you like to learn more about performant Python development? Check out our Python & Django Technologies page!

What is PyO3?

PyO3 is a Rust framework that enables seamless integration of Rust code into Python. It provides a bridge between both languages, allowing Rust programs to be used as native Python modules. With PyO3, you can

The framework takes care of important technical details such as memory management, handling data structures, and interaction between runtimes of both languages.

The combination of Rust's high performance and safety with Python's flexibility makes PyO3 particularly interesting for computationally intensive tasks like data processing, machine learning or - as in Pfuzzer's case - developing a performant fuzzy search module.

Pfuzzer: A Python Module for Fuzzy Search with Rust

Pfuzzer is a Python Fuzzy Searcher that, with the help of Rust's Nucleo library, enables a fast and efficient fuzzy search.

Why Rust for Fuzzy Search?

Installing and Setting Up Pfuzzer

To make a Rust project usable with PyO3 in Python, Maturin is one of the most popular tools. Maturin simplifies the entire build process by compiling the Rust code and preparing it as a Python wheel package. This package can subsequently be installed and used like any other Python library.

Setting Up a PyO3 Project

1. Create Project

First create a new Python project (In the sense of "Rustification" we of course use the uv Package Manager):

$ mkdir pfuzzer
$ cd pfuzzer
$ uv venv
$ uv add maturin
$ . .env/bin/activate

2. Initialize Maturin

Run maturin and have it create all necessary files:

$ maturin init ✔  📷 What kind of bindings to use? ¸ pyo3
  ✦ Done! New project created pfuzzer

This command will create a Cargo.toml as well as a lib.rs file. These will be used respectively for Rust Dependency Management and the actual Rust Code that will be called from Python.

The command maturin develop will then install your freshly created Rust package in the previously created virtualenv. Since we use uv, the -–uv flag must be added. With maturin build –release, the package can be built for roll-out.

Pfuzzer: Implementation and Functionality

The Pfuzzer Python Package is a wrapper for the Rust Nucleo library. As previously shown, wrapping an existing library is easily done with PyO3. Nevertheless, I will briefly explain the implementation and functionality. The entire code can be viewed here.

Thanks to PyO3, the implementation is quite straightforward. It provides several Rust attributes to make Rust code callable in Python. Here, e.g., the implementation of the fundamental Python module:

mod python_classes;
use pyo3::prelude::*;
/// A Python fuzzy searcher module implemented in Rust.
#[pymodule]
fn pfuzzer(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_class::<python_classes::pfuzzer::Pfuzzer>()?;
    Ok(())
}

This is so little code that one might think it's already written in Python. But briefly explained the most important sections:

Michael Schilonka

We can also accelerate your Python application with Rust and PyO3.

Get in touch

×

Book a meeting with our expert

Michael Schilonka

Find a time slot

or

or

Form is loading...

Close

There are other ways to assign Rust code to the module. Among these is m.add_function. Unlike add_class, the Turbofish syntax is not used here, instead the Rust function must be called with a macro. More about this in the PyO3 docs.

Now let's get to implementing our Pfuzzer class. Since there are no classes in Rust, we use structs:

#[pyclass]
pub struct Pfuzzer {
    pub matcher: Matcher,
}

As an attribute, pyclass is given here. This marks the struct as a Python class and can then, as previously shown, be assigned to the module.

Every good class obviously also needs a constructor:

use nucleus::{Config, Matcher, Utf32Str};

#[pymethods]
impl Pfuzzer {
    #[new]
    pub fn new() -> PyResult<Self> {
        Ok(Pfuzzer {
            matcher: Matcher::new(Config::DEFAULT),
        })
    }

...

Through #[pymethods], we declare the entire implementation block as Python methods of the Pfuzzer class. The constructor itself is marked with #[new]. Currently, PyO3 supports only the __new__ magic method and not __init__.

Finally, the actual "logic" of our wrapper. I present the compare_strings method:

pub fn compare_strings(&self, targets: Vec<String>, query: String) -> Vec<Option<u16>> {
        let mut res = Vec::<Option<u16>>::new();
        for target in targets {
            res.push(self.matcher.to_owned().fuzzy_match(
                Utf32Str::Ascii(target.as_bytes()),
                Utf32Str::Ascii(query.as_bytes()),
            ))
        }
        return res;
    }

As one can quickly recognize, it uses the nucleo Matcher and performs fuzzy matching for each target string based on the given query string. The method's result is then an optional integer per target string, which indicates how well the query string matches the target string. The higher the result, the better the match. A Null / None value indicates that the target and query do not match (or at least are not measurably similar).

And how does this translate to Python? Have a look:

from pfuzzer import Pfuzzer

pf = Pfuzzer()

print(pf.compare_strings(["hello world", "hello blueshoe"], "helo world"))

>>> [257, None]

According to the result matrix, the first target string is the string which best matches the query.

If you still have ideas for new features for Pfuzzer, or even optimization thoughts, then feel free to leave a comment. Or create an Issue on Github!

Conclusion: Rust and Python - An Unbeatable Combination

In this article, we have illuminated the impressive synergy between Rust and Python through the development of the Pfuzzer module. Pfuzzer demonstrates how PyO3 enables integrating the high performance and efficiency of Rust into the user-friendly environment of Python. The seamless connection of both languages opens up new possibilities for developers, especially in areas like data processing and machine learning.

If you have a passion for really fast Python, then write to us in the comments and we'll discuss the deployment possibilities of Rust in your Python project! We're also looking forward to comments about your use of PyO3!

Frequently Asked Questions

1. How can I create a Python module with Rust and PyO3?

To write a Rust module for Python, you'll need PyO3 and Maturin. The fundamental steps:

2. How does Fuzzy Search work in Rust with the Nucleo library?

The Nucleo library offers a high-performance Fuzzy Matching Algorithm to calculate string similarity. Pfuzzer uses this library for fast approximate searches in Python.

3. Can I use Rust modules in existing Python projects?

Yes! Rust modules can be seamlessly integrated into existing Python projects. Thanks to PyO3 you can directly import and use Rust functions as Python modules.