Skip to content

Progress Tracking

Monitor the progress of long-running Bundlebase operations.

Overview

Bundlebase reports progress events for operations like attaching data, indexing, and fetching from sources. If tqdm is installed, progress bars appear automatically — no setup required. You can also register a custom callback to handle progress events yourself.

Progress Module

progress

Progress tracking for long-running Bundle operations.

This module provides integration with progress bar libraries like tqdm, allowing users to see visual progress feedback for operations like indexing, attaching data, and querying large datasets.

By default, if tqdm is installed, progress bars will appear automatically. Users can also provide custom callbacks for progress tracking.

Example

Auto-enabled if tqdm installed

import bundlebase c = await bundlebase.create("/path/to/bundle") await c.attach("large_data.parquet") # Progress bar appears automatically

Custom callback

def my_progress(event, operation, id, current, total, message): if event == 'start': print(f"Starting: {operation}") elif event == 'update': print(f"Progress: {current}/{total or '?'}") elif event == 'finish': print(f"Finished: {operation}")

bundlebase.progress.set_callback(my_progress)

_current_tracker module-attribute 

_current_tracker: Optional[Callable] = None

__all__ module-attribute 

__all__ = ['set_callback', 'enable_tqdm', 'disable', 'install_default_tracker', 'TqdmProgressTracker', 'CustomCallbackTracker']

TqdmProgressTracker 

TqdmProgressTracker()

Progress tracker that uses tqdm for visual progress bars.

This tracker creates and manages tqdm progress bars for Bundle operations. It supports both determinate (known total) and indeterminate (unknown total) progress.

__call__ 

__call__(event: str, operation: str, id: int, current: int, total: Optional[int], message: Optional[str])

Handle a progress event.

PARAMETER DESCRIPTION
event

Type of event ('start', 'update', 'finish')

TYPE: str

operation

Human-readable operation name

TYPE: str

id

Unique identifier for this progress operation

TYPE: int

current

Current progress value

TYPE: int

total

Total expected value (None for indeterminate)

TYPE: Optional[int]

message

Optional status message

TYPE: Optional[str]

CustomCallbackTracker 

CustomCallbackTracker(callback: Callable)

Progress tracker that forwards events to a custom callback.

This tracker allows users to provide their own progress handling logic.

PARAMETER DESCRIPTION
callback

Function with signature: fn(event, operation, id, current, total, message)

TYPE: Callable

__call__ 

__call__(event: str, operation: str, id: int, current: int, total: Optional[int], message: Optional[str])

Forward the event to the custom callback.

_has_tqdm 

_has_tqdm() -> bool

Check if tqdm is available.

set_callback 

set_callback(callback: Optional[Callable] = None)

Set a custom progress callback.

PARAMETER DESCRIPTION
callback

Function with signature: fn(event: str, operation: str, id: int, current: int, total: Optional[int], message: Optional[str]) If None, disables progress tracking.

TYPE: Optional[Callable] DEFAULT: None

Example

def my_callback(event, operation, id, current, total, message): if event == 'start': print(f"Starting: {operation}") elif event == 'update': pct = (current / total * 100) if total else 0 print(f"Progress: {pct:.1f}%") elif event == 'finish': print(f"Finished: {operation}")

bundlebase.progress.set_callback(my_callback)

enable_tqdm 

enable_tqdm()

Explicitly enable tqdm progress bars.

This is called automatically on module import if tqdm is available. Users only need to call this if they disabled progress and want to re-enable it.

RAISES DESCRIPTION
RuntimeError

If tqdm is not installed

disable 

disable()

Disable all progress tracking.

Operations will run silently without any progress feedback.

install_default_tracker 

install_default_tracker()

Install the default progress tracker.

If tqdm is available, enables tqdm progress bars. Otherwise, does nothing (silent operation).

This is called automatically when the bundlebase module is imported.

Examples

Automatic Progress Bars (tqdm)

If tqdm is installed, progress bars appear automatically for long-running operations:

import bundlebase

c = await bundlebase.create("/path/to/bundle")
c = await c.attach("large_data.parquet")  # Progress bar appears automatically

Install tqdm if you don't have it:

pip install tqdm

Custom Progress Callback

Register your own callback to handle progress events. The callback receives an event type ('start', 'update', or 'finish'), the operation name, a unique operation id, the current progress value, the total (or None if unknown), and an optional message:

import bundlebase

def my_progress(event, operation, id, current, total, message):
    if event == 'start':
        print(f"Starting: {operation}")
    elif event == 'update':
        pct = (current / total * 100) if total else 0
        print(f"Progress: {pct:.1f}%")
    elif event == 'finish':
        print(f"Finished: {operation}")

bundlebase.progress.set_callback(my_progress)

Disabling Progress

Turn off all progress feedback:

import bundlebase

bundlebase.progress.disable()

Re-enable tqdm progress bars later:

bundlebase.progress.enable_tqdm()

Logging Progress Events

Forward progress events to Python logging instead of the terminal:

import bundlebase
import logging

logger = logging.getLogger("bundlebase.progress")

def log_progress(event, operation, id, current, total, message):
    if event == 'start':
        logger.info("Starting %s", operation)
    elif event == 'finish':
        logger.info("Finished %s", operation)

bundlebase.progress.set_callback(log_progress)

Callback Reference

A progress callback has the signature:

def callback(event: str, operation: str, id: int, current: int,
             total: Optional[int], message: Optional[str]) -> None: ...
Parameter Description
event 'start', 'update', or 'finish'
operation Human-readable operation name
id Unique identifier for this progress operation (multiple operations can run concurrently)
current Current progress value
total Total expected value, or None for indeterminate progress
message Optional status message

See Also