conda and Jupyter notebooks for data work

In short: conda manages Python environments and packages including non-Python libraries; Jupyter notebooks mix code, results and text in one interactive document.

In more detail: In data analysis and machine learning many packages with difficult dependencies are needed (NumPy, pandas, PyTorch, CUDA libraries). conda (and the faster variant mamba) installs them together in separate environments. Jupyter provides the working surface for it.

In Depth

conda: environments

conda create -n data python=3.12 pandas matplotlib jupyterlab
conda activate data
conda install scikit-learn
conda env export > environment.yml     # record the environment
conda env create -f environment.yml    # rebuild on another machine
conda deactivate

An environment keeps packages and Python version separate so that projects do not disturb each other. Miniconda is the lean installation, Anaconda brings many packages. For pure Python packages pip with venv also works; conda is strong when non-Python libraries (C, Fortran, CUDA) are to come along. Alternatives: uv, Poetry, pixi.

Jupyter Notebook and JupyterLab

A notebook consists of cells: code (Python, R, Julia and others), text in Markdown and the outputs (numbers, tables, charts). You run cells one at a time and see the result immediately. This makes notebooks suitable for exploring, learning, presenting and reproducible reports. JupyterLab is the extended interface; Google Colab and Kaggle offer notebooks in the browser with free compute resources.

Typical use

import pandas as pd
df = pd.read_csv('sales.csv')
df.groupby('month')['revenue'].sum().plot(kind='bar')

The chart appears directly below the cell. With %timeit, !pip install and further magic commands there are helpers for measuring and shell.

Stumbling blocks

The order of execution matters: cells can run in any order and create hidden state. Before passing it on, run “Restart & Run All”. Notebooks are unwieldy for version control (Git) because they store outputs. Larger programs belong in modules that the notebook calls. Take care not to leave credentials and personal data in notebooks.

See also: pip and venv, pandas, NumPy, Virtual environment