Tracking progress of Python's asyncio coroutines with tqdm
A neat little trick with tqdm that allows you to track the progress of asynchronous coroutines in Python:
# aws is any set of awaitables - coroutines in this case
for future in tqdm.tqdm(asyncio.as_completed(aws)):
result = await future
# Handle results as they are completed
asyncio.as_completed schedules the execution of the coroutine objects and returns a iterable of Future objects, each representing the earliest result from the remaining set of coroutines.
Since tqdm.tqdm takes any iterable and generates a progress meter from it, we can simply wrap the result of asyncio.as_completed with tqdm.tqdm and await the result of the next earliest Future object in each iteration, effectively providing a progress meter representing the execution progress of the set of coroutines.
Note: asyncio.as_completed takes a set of any awaitable (i.e. any object that can be used in an await expression), so this would also work with Future or Task objects in addition to coroutines.