Design ChatGPT

Design a conversational AI system like ChatGPT: users send text prompts and receive AI-generated responses streamed back token-by-token, with persistent conversation history across sessions. Problem Statement Design a conversational AI system like ChatGPT: users send text prompts and receive AI-generated responses streamed back token-by-token, with persistent conversation history across sessions. This was an Apple onsite and phone screen for a Software Engineer. Interviewers focused on how GPU inference fits into the broader system, how streaming responses work end-to-end, and how conversation context is managed at scale without blowing up latency.