Web Crawler Challenge

Implement a web crawler that, given a starting URL, retrieves the content of the page and all directly linked pages up to a depth of 3. The crawler should ignore duplicate pages. Assume limited time and memory for requests. Write efficient code to minimize the number of requests and optimize speed. Provide a few URLs as input and return all retrieved pages' titles and URLs. Example test case: When the input is the starting URL 'http://example.com', and this page has two direct links 'http://example.com/page1' and 'http://example.com/page2', where 'http://example.com/page1' further links to 'http://example.com/page3'. The output should include the titles and URLs of these pages. Example Input 'http://example.com'