{
"cells": [
{
"cell_type": "markdown",
"id": "c8185b74",
"metadata": {},
"source": [
"# distrdf002_dask_connection\n",
"Configure a Dask connection and fill two histograms distributedly.\n",
"\n",
"This tutorial shows the ingredients needed to setup the connection to a Dask\n",
"cluster (e.g. a `LocalCluster` for a single machine). After this initial\n",
"setup, an RDataFrame with distributed capabilities is created and connected\n",
"to a Dask `Client` instance. Finally, a couple of histograms are drawn from\n",
"the created columns in the dataset. Relevant documentation can be found at\n",
"http://distributed.dask.org/en/stable .\n",
"\n",
"\n",
"\n",
"\n",
"**Author:** Vincenzo Eduardo Padulano \n",
"This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Tuesday, May 19, 2026 at 08:10 PM."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "89f8dbe2",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:10:53.567746Z",
"iopub.status.busy": "2026-05-19T20:10:53.567615Z",
"iopub.status.idle": "2026-05-19T20:10:54.836132Z",
"shell.execute_reply": "2026-05-19T20:10:54.835284Z"
}
},
"outputs": [],
"source": [
"from dask.distributed import LocalCluster, Client\n",
"import ROOT\n",
"\n",
"\n",
"def create_connection():\n",
" \"\"\"\n",
" Setup connection to a Dask cluster. Two ingredients are needed:\n",
" 1. Creating a cluster object that represents computing resources. This can be\n",
" done in various ways depending on the type of resources at disposal. To use\n",
" only the local machine (e.g. your laptop), a `LocalCluster` object can be\n",
" used. This step can be skipped if you have access to an existing Dask\n",
" cluster; in that case, the cluster administrator should provide you with a\n",
" URL to connect to the cluster in step 2. More options for cluster creation\n",
" can be found in the Dask docs at\n",
" http://distributed.dask.org/en/stable/api.html#cluster .\n",
" 2. Creating a Dask client object that connects to the cluster. This accepts\n",
" directly the object previously created. In case the cluster was setup\n",
" externally, you need to provide an endpoint URL to the client, e.g.\n",
" 'https://myscheduler.domain:8786'.\n",
"\n",
" Through Dask, you can connect to various types of cluster resources. For\n",
" example, you can connect together a set of machines through SSH and use them\n",
" to run your computations. This is done through the `SSHCluster` class. For\n",
" example:\n",
"\n",
" ```python\n",
" from dask.distributed import SSHCluster\n",
" cluster = SSHCluster(\n",
" # A list with machine host names, the first name will be used as\n",
" # scheduler, following names will become workers.\n",
" hosts=[\"machine1\",\"machine2\",\"machine3\"],\n",
" # A dictionary of options for each worker node, here we set the number\n",
" # of cores to be used on each node.\n",
" worker_options={\"nprocs\":4,},\n",
" )\n",
" ```\n",
"\n",
" Another common usecase is interfacing Dask to a batch system like HTCondor or\n",
" Slurm. A separate package called dask-jobqueue (https://jobqueue.dask.org)\n",
" extends the available Dask cluster classes to enable running Dask computations\n",
" as batch jobs. In this case, the cluster object usually receives the parameters\n",
" that would be written in the job description file. For example:\n",
"\n",
" ```python\n",
" from dask_jobqueue import HTCondorCluster\n",
" cluster = HTCondorCluster(\n",
" cores=1,\n",
" memory='2000MB',\n",
" disk='1000MB',\n",
" )\n",
" # Use the scale method to send as many jobs as needed\n",
" cluster.scale(4)\n",
" ```\n",
"\n",
" In this tutorial, a cluster object is created for the local machine, using\n",
" multiprocessing (processes=True) on 2 workers (n_workers=2) each using only\n",
" 1 core (threads_per_worker=1) and 2GiB of RAM (memory_limit=\"2GiB\").\n",
" \"\"\"\n",
" cluster = LocalCluster(n_workers=2, threads_per_worker=1, processes=True, memory_limit=\"2GiB\")\n",
" client = Client(cluster)\n",
" return client"
]
},
{
"cell_type": "markdown",
"id": "b41186bc",
"metadata": {},
"source": [
"This tutorial uses Python multiprocessing, so the creation of the cluster\n",
"needs to be wrapped in the main clause as described in the Python docs\n",
"https://docs.python.org/3/library/multiprocessing.html"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "7d732ff9",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:10:54.854380Z",
"iopub.status.busy": "2026-05-19T20:10:54.854031Z",
"iopub.status.idle": "2026-05-19T20:11:00.298866Z",
"shell.execute_reply": "2026-05-19T20:11:00.276080Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Info in : png file distrdf002_dask_connection.png has been created\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Saved figure to distrdf002_dask_connection.png\n"
]
}
],
"source": [
"if __name__ == \"__main__\":\n",
"\n",
" # Create the connection to the mock Dask cluster on the local machine\n",
" connection = create_connection()\n",
" # Create an RDataFrame that will use Dask as a backend for computations\n",
" df = ROOT.RDataFrame(1000, executor=connection)\n",
"\n",
" # Set the random seed and define two columns of the dataset with random numbers.\n",
" ROOT.gRandom.SetSeed(1)\n",
" df_1 = df.Define(\"gaus\", \"gRandom->Gaus(10, 1)\").Define(\"exponential\", \"gRandom->Exp(10)\")\n",
"\n",
" # Book an histogram for each column\n",
" h_gaus = df_1.Histo1D((\"gaus\", \"Normal distribution\", 50, 0, 30), \"gaus\")\n",
" h_exp = df_1.Histo1D((\"exponential\", \"Exponential distribution\", 50, 0, 30), \"exponential\")\n",
"\n",
" # Plot the histograms side by side on a canvas\n",
" c = ROOT.TCanvas(\"distrdf002\", \"distrdf002\", 800, 400)\n",
" c.Divide(2, 1)\n",
" c.cd(1)\n",
" h_gaus.DrawCopy()\n",
" c.cd(2)\n",
" h_exp.DrawCopy()\n",
"\n",
" # Save the canvas\n",
" c.SaveAs(\"distrdf002_dask_connection.png\")\n",
" print(\"Saved figure to distrdf002_dask_connection.png\")"
]
},
{
"cell_type": "markdown",
"id": "f8e5c4e9",
"metadata": {},
"source": [
"Draw all canvases "
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "f7a658fe",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:11:00.329164Z",
"iopub.status.busy": "2026-05-19T20:11:00.328957Z",
"iopub.status.idle": "2026-05-19T20:11:00.613826Z",
"shell.execute_reply": "2026-05-19T20:11:00.592998Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"
\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from ROOT import gROOT \n",
"gROOT.GetListOfCanvases().Draw()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {
"a1e4cd14769746efb456f516d45772ad": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "2.0.0",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "2.0.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "2.0.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border_bottom": null,
"border_left": null,
"border_right": null,
"border_top": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"a2a58b29476b4422bee09961f780c43e": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "2.0.0",
"model_name": "VBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "2.0.0",
"_model_name": "VBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "2.0.0",
"_view_name": "VBoxView",
"box_style": "",
"children": [],
"layout": "IPY_MODEL_a1e4cd14769746efb456f516d45772ad",
"tabbable": null,
"tooltip": null
}
}
},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}