URL Maze — Recursive API Crawl to "Congrats"

Call a given URL; the response contains the next URLs to follow. Keep crawling until a response says "Congrats", returning only that final URL. Follow-ups add HTTP error handling (503/504/404, retries) and an auth-passkey scavenger hunt. Requirements You are given a starting URL. Call it; the response may contain a new set of URLs (or other data). Continue calling the returned URLs, recursing/iterating, until a response returns "Congrats". Return only the final URL — you do not need to return the path taken. You must first print the HTTP response and inspect it to learn the format before parsing; the prompt intentionally does not specify the response schema. Node shape (as discovered by exploring): each response roughly contains go_next: ["...", "..."] — the next URLs to visit (the canonical next-URL key), a terminal marker like "Congrats" (handle case-insensitively, and search recursively — it may appear nested in a dict/list value, e.g. {"message": "Congrats, you found the exit!"}), and (in the follow-up) a key field used for authentication. The canonical two-part structure exposes two signatures — a clean traversal, then a resilient version: from typing import Any, Callable, Optional def find_exit_url(start_url: str, fetch_json: Callable[[str], Any]) -> Optional[str]: ... # Part 1: clean graph search. fetch_json(url) returns a JSON-like Python object. # Assume every request succeeds and every body is valid JSON. # A response "containing congrats" (case-insensitive, checked recursively) means url is the exit. # Otherwise the next URLs live under the "go_next" key. The graph may contain cycles. # Return the exit URL, or None if no exit is reachable. def find_exit_url_resilient( start_url: str, fetch: Callable[[str, dict[str, str]], Any], # fetch(url, headers) -> response with .status_code and .body max_retries: int = 5, ) -> Optional[str]: ... # Part 2: same traversal, real-HTTP behavior. Statuses: 503/504 retry, 404 dead-end, # 401 needs an auth header (do NOT mark permanently visited), 2xx may expose a passkey. # Body may be plain text or non-JSON; "Congrats" can appear in JSON or raw text. Follow-up 1 — error handling. The interviewer swaps in a new starting URL whose nodes return HTTP error statuses — commonly 503 / 504 and 404. Your Part-1 solution will break; you must add retry logic. (One interviewer specifically wanted immediate retries without sleep/backoff, and needed the retry count raised — around 5 — before the chain passed.) Follow-up 2 — passkey auth. Some reachable nodes expose a passkey; some nodes require one. Collect passkeys into a dictionary as you discover them and, for any node that needs authentication, attach the correct key/value pairs as request headers so the request succeeds. Passkeys are returned by successful responses in one of these exact shapes — normalize each to header key/value(s): {"passkeys": {"X-Maze-Key": "abc123"}} # header dict → merge every pair into headers {"key": {"X-Maze-Key": "abc123"}} # same, under the "key" field {"passkey": "abc123"} # bare string → send as {"X-Passkey": "<value>"} Notes A BFS/DFS over the URL graph both work; one passing solution used BFS over go_next. BFS is convenient because it sidesteps recursion-depth limits and makes visited-tracking natural. This is graded as a practical, fiddly round, not an algorithm round. The interviewer may say outright that code style doesn't matter — the priority is getting the core crawl working, then layering in error handling. Print sparingly while debugging. A common failure: so much debug output that you miss the node where a key is returned, then waste time hunting a "missing key" bug. Inspect responses deliberately. For the response body, read the raw text first (e.g. resp.text) before attempting JSON parsing — formats are inconsistent across nodes. Time management is the killer. Multiple candidates left a known bug unfixed when time expired. Resilient-crawler design invariants The follow-up tests whether you can harden the crawler without breaking the Part-1 traversal invariants. The subtle points, all of which are checkable: Retry vs dead-end by status: retry 503/504 a fixed number of times (≈5, no sleep) before giving up on that URL; treat 404 (and any other >= 400 non-401) as a dead end and mark it processed. Only 401 is special. Do NOT mark 401 URLs permanently visited. A URL that fails auth before the relevant passkey is discovered may become reachable later — record it as blocked (with the current auth "version"), not as visited. Requeue blocked URLs when headers change. When a successful response yields a new passkey that actually changes the header set, bump an auth version and re-enqueue previously-blocked URLs so they get retried with the new headers. (A no-op passkey update must not trigger requeues.) Inspect raw body before assuming JSON. Parse defensively: if the body is already a dict/list use it; if it's a string, json.loads it and fall back to the raw string on JSONDecodeError. Success text may be plain text. Keep cycle protection. Successfully processed URLs (and dead ends) stay marked visited so the graph's cycles don't loop forever; guard both at enqueue time and again when dequeuing. Complexity: Part 1 is O(V + E) time / O(V) space over reachable URLs and links. The resilient version is O((V + E) · (R + A)) worst case, where R is the retry limit and A is the number of header changes; with small constant R and A bounded by the passkey count it degrades to O(V + E) in practice. Preparation Build a small crawler against a mock server: follow go_next links until a terminal marker, returning the final URL. Add a retry wrapper that handles 5xx/404 with a configurable, no-sleep retry count, and a header-injection step that looks up a per-host passkey from a dict you populate during the crawl. Rehearse inspecting a response (resp.status_code, resp.text, then resp.json()) so exploring an unknown payload under time pressure is automatic. Drill the 401-requeue path specifically: block a node behind a passkey that's only revealed by a later node, and confirm your visited-set logic still reaches the exit.