Coding Q1 — Image Processing Pipeline

The other Q1 prompt: apply a sequence of image transformations (grayscale, flip, scale, blur, rotate) to a directory of images using a Python imaging library, then optimize with multiprocessing/multithreading for a target wall-clock budget. Requirements You are given four folders and their absolute paths up front: small_images/ — used for the correctness pass. big_images/ — used for the performance pass. transformations/ — JSON files describing transformation sequences. out/ — write outputs here. The canonical function signature is: def process_images( image_dir: str, transformation_dir: str, output_dir: str, get_output_path: Callable[[str, str], str], # (image_name, json_name) -> abs path ) -> None: ... # For each (image, transformations.json) pair: load image, apply the # listed transformations in order, write to get_output_path(...). # No return value — outputs are observed by the grader on disk. Each transformations JSON has the shape {"transformations": [{"type": ..., <param>: ...}, …]}. Six transformation types: No-parameter: grayscale, flip_horizontally (a.k.a. flip_horizontal), flip_vertically (a.k.a. flip_vertical). Parameterised: scale (factor: float), blur (radius: int), rotate (angle: float, degrees). For every (image, transformation_json) pair, apply the listed transformations sequentially and write the result. The supplied get_output_path (sometimes named util.get_output_path) supplies output paths so candidates do not have to do their own os.path work. Phases Correctness pass: apply transformations on small_images/. Match against expected outputs. Performance pass: apply the same pipeline to big_images/ under a wall-clock target. Naive sequential implementations always miss the target; candidates are expected to introduce ProcessPoolExecutor (preferred — image transforms are CPU-bound) or ThreadPoolExecutor (acceptable; some interviewers explicitly want to see threads). Follow-ups: thread vs. process tradeoffs, sharing pre-decoded images across workers, distributing across multiple machines, dealing with images that don't fit in memory. Notes pillow (PIL) is the most common choice; scikit-image works too. Familiarity with at least one is required — interviewers will not coach API details. A common rewrite during the round: stop re-reading the same source image inside each worker. Split the pipeline so the decode happens once and only the transform list runs per (image, json) pair in parallel. One sibling variant adds TPS / throughput estimation as a follow-up — quantify FPS achieved, identify the bottleneck (CPU, memory bandwidth, disk IO). For the multi-process path, beware of pickling cost: passing decoded PIL images across process boundaries can dominate wall-clock unless you pass paths and decode inside the worker. The interviewer states up front that searching online is allowed and expected — this round tests how you read a new imaging-library API in real time. AI assistants are forbidden. Some rotations name the folders large_images/ and output/ (instead of big_images/ and out/); the helper util exposes both the listing and the output-path builder. Preparation Write a sequential transform pipeline against pillow from scratch in under 10 minutes (open, apply, save). Build a ProcessPoolExecutor wrapper that maps (image_path, transformation_json) pairs to workers and returns when all complete. Internalize the thread-vs-process answer: image transforms release the GIL inside PIL's C code in some cases, so threads can help; but most candidates have better luck arguing for processes due to predictable CPU parallelism. Cross-train the web-crawler sibling prompt in parallel — same recruiter blurb, different problem.