Skip to Content

Tracking progress of Python's asyncio coroutines with tqdm

A neat little trick with tqdm that allows you to track the progress of asynchronous coroutines in Python:

# aws is any set of awaitables - coroutines in this case
for future in tqdm.tqdm(asyncio.as_completed(aws)):
    result = await future
    # Handle results as they are completed

asyncio.as_completed schedules the execution of the coroutine objects and returns a iterable of Future objects, each representing the earliest result from the remaining set of coroutines.

Since tqdm.tqdm takes any iterable and generates a progress meter from it, we can simply wrap the result of asyncio.as_completed with tqdm.tqdm and await the result of the next earliest Future object in each iteration, effectively providing a progress meter representing the execution progress of the set of coroutines.

progress bar in action

Note: asyncio.as_completed takes a set of any awaitable (i.e. any object that can be used in an await expression), so this would also work with Future or Task objects in addition to coroutines.

comments powered by Disqus