← 返回 scale.ai 的题目列表LLM API Practical and Output Validation
类型:qbank
Build a small backend or notebook function that calls an LLM API, writes a prompt for a given scenario, parses the output, and validates whether the model response satisfies the task. The API key may be provided during the interview.
Requirements
Call an LLM API from code; one loop explicitly provides the API key.
Write a prompt for the given scenario.
Implement the first function that sends the prompt and handles the model's response.
Normalize or parse the output into the requested format.
Implement a second function that validates whether the previous output is correct.
Follow-up discussion covers:
Output-format validation.
What can go wrong in production.
How to make the feature reliable beyond a demo.
Senior FDE variant: fill in chunks of Python / notebook code to perform LLM evaluations.
Notes
The coding signal is API integration plus defensive engineering, not just prompt wording.
Treat the model output as untrusted. Validate schema, required fields, allowed values, and failure modes before returning it to the caller.
Mention retries and timeouts, but avoid blindly retrying non-idempotent calls. Keep request ids or trace ids if the prompt produces side effects.
For production discussion, cover logging, prompt/version management, cost controls, response validation, fallback behavior, and test fixtures with deterministic mocked responses.
Alternate canonical variant — CSV classification service
Build an HTTP server that accepts a CSV file path and writes the converted data as JSON to the filesystem.
Extend the JSON records with a category field generated through an LLM API; the API key is provided.
The round is AI-enabled, but the candidate is still expected to author the prompt and drive the implementation rather than delegate the entire task.
The scaling follow-up focuses on sharding the input and issuing LLM calls in parallel.
Preparation
Write a minimal Python wrapper around an LLM-style HTTP API: request payload, headers, timeout, response parse, and exception handling.
Practice validating JSON-like model output with a strict schema and clear error paths.
Prepare a short production-readiness checklist: retries, rate limits, prompt versioning, observability, eval set, and rollback strategy.