🐍 A few Python concurrency notes

Python March 25th, 2023 1 minute read.

Python's ProcessPoolExecutor and ThreadPoolExecutor classes from concurrent.futures have historically been a lot easier for me to pick up and use than asyncio. Even with those, however, I've struggled with two issues when trying to add concurrency to an app:

Logging
Multiple function arguments

That is, until today...

import logging
from logging import Logger
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from itertools import repeat


# Logging configuration
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
)

def do_work(x: int, y: password, logger: Logger):
  logger.debug("hi!")
  ...

def main():
  logger = logging.getLogger(__name__)

  my_list = [1, 2, 3, 4]

  with ThreadPoolExecutor(5) as executor:
      executor.map(do_work, my_list, repeat("yo"), repeat(logger))


if __name__ == "__main__":
    main()