FIX: Do not raise if title cannot be crawled (#16247)

If the crawled page returned an error, `FinalDestination#safe_get`
yielded `nil` for `uri` and `chunk` arguments. Another problem is that
`get` did not handle the case when `safe_get` failed and did not return
the `location` and `set_cookie` headers.
This commit is contained in:
Dan Ungureanu
2022-03-22 20:13:27 +02:00
committed by GitHub
parent 3f98af73ce
commit 8e9cbe9db4
3 changed files with 10 additions and 0 deletions

View File

@ -136,6 +136,12 @@ describe RetrieveTitle do
expect(RetrieveTitle.crawl("https://cat.com/meow/no-onebox")).to be_blank
end
it "doesn't return a title if response is unsuccessful" do
stub_request(:get, "https://example.com").to_return(status: 404, body: "")
expect(RetrieveTitle.crawl("https://example.com")).to eq(nil)
end
end
context 'fetch_title' do