Contributions

Article
Conda installs are very slow, but you can speed them with a much-faster Conda reimplementation called Mamba.
Article
You can write Python extensions with Cython, Rust, and many other tools. Learn which one you should use, depending on your particular needs.
Article
Python has two packaging systems, pip and Conda. Learn the differences between them so you can pick the right one for you.
Article
Python 3.10 is out now, but you won't be able to switch for a while: there's missing packages, missing toolchain updates, and more.
Article
Learn how to scan your Conda package dependencies for security vulnerabilities.
Tutorial
NumPy provides memory views transparently, as a way to save memory. But you need to understand how they work, because if you’re not careful you can also leak memory, or even modify data in ways you didn’t expect.
Article
When you’re loading many strings into Pandas, you’re going to use a lot of memory. If you have only a limited number of strings, you can save memory with categoricals, but that’s only helpful in a limited number of situations.

With Pandas 1.3, there’s a new option that can save memory on large number of strings as well, simply by changing to a new column type.
Article
Learn how to accurately measure memory usage of your Pandas DataFrame or Series.
Article
Measuring your Python program’s memory usage is not as straightforward as you might think. Learn two techniques, and the tradeoffs between them.
Article
Learn the variety of techniques you can use to make your Python application’s Docker image a whole lot smaller.
Article
A compiled language like Rust or C is a lot faster than Python, but it won’t always make your Python code faster. Learn about the hidden overhead you’ll need to overcome.
Tutorial
Pandas can easily load data using a SQL query, but the resulting dataframe may use too much memory. Learn how to process data in batches, and then how to reduce memory usage even further.
Article
Using old versions of pip can result in installing old packages, or needing to recompile packages from scratch. So make sure you upgrade pip before using it.
Library
A Python memory profiler for data processing and scientific computing applications
Article
Every time you change your pip requirements and rebuild your Docker image, you’re going to have download all your packages. Learn how to prevent this with Docker BuildKit’s new caching feature.
Article
There are many ways out-of-memory problems can manifest in Python. Learn how to identify them, as a first step to fixing the problem.
Tutorial
You want your application packaging to be reproducible, but you also want to be able to change dependencies easily without conflicts. Conda doesn’t make this easy, so learn how to do it with a third-party tool: conda-lock.
Article
To make your Python code faster, you should often start with optimizing single-threaded versions, then consider multiprocessing, and only then think about a cluster.
Article
Python 3.9 is out now, but when should you switch? Learn the problems you'll encounter, and when it's time to try it out.
Article
The official Python Docker image is useful, but to actually understand why, and to use it correctly, it’s worth understanding how exactly it’s constructed.
Article
There are a variety of ways of packaging your Python application for distribution, from wheels to Docker to PEX to Conda, and more. This article gives a survey of the different approaches, specifically focusing on distributing internal server applications.
Article
When your server is leaking memory, the Fil memory profiler can help you spot the buggy code.
Tutorial
Objects in Python have large memory overhead; create too many objects, and you’ll use far more memory than you expect. Learn why, and what do about it.
Article
Storing integers or floats in Python has a huge overhead in memory. Learn why, and how NumPy makes things better.
Tutorial
There are many reasons your code might fail to import in Docker. Here's a quick series of checks you can do to figure out the problem.
Article
Python will automatically release memory for objects that aren't being used. But sometimes function calls can unexpectedly keep objects in memory. Learn about Python memory management, how it interacts with function calls, and what you can do about it.
Tutorial
Debugging out-of-memory crashes can be tricky. Learn how the Fil memory profiler can help you find where your memory use is happening.
Article
Fil is a new memory profiler which shows you peak memory usage, and where that memory was allocated. It’s designed specifically for the needs of data scientists and scientists running data processing pipelines.
Tutorial
You don’t want to deploy insecure code to production—but it’s easy for mistakes and vulnerabilities to slip through. So you want some way to catch security issues automatically, without having to think about it.

For your code to your Python dependencies to the system packages you depend on, learn about some tools that will help you catch security vulnerabilities.
Tutorial
If your Docker build isn’t reproducible, a minor bug fix can spiral out of control into a series of unwanted and unnecessary major version upgrades.

There are multiple layers of reproducibility, from operating system to Python dependencies, so this article will cover how to deal with each.

Showing the last 30 only...