← 返回 anthropic 的题目列表Web Crawler with Multithreading
类型:online_judge
Build a multithreaded web crawler to retrieve up to 100 distinct URLs. The initial URL will be provided as a command line argument. The crawler should visit sites concurrently and stop when either 100 URLs have been retrieved or no further links can be followed. Requirements:
Use multithreading to improve crawling efficiency.
Each thread should handle a different URL.
The data structure should prevent duplicate URL visits.
Output each visited URL to the console.
Test cases:
Input: https://example.com
Input: https://xyz.io
Input: https://mytest.org
Example
Input
https://example.com