github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-06-06 02:24:47 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

FIX: Detect Wayback Machine using user agent (#9777)

Browse Source
This commit is contained in:
Dan Ungureanu
2020-05-14 14:10:07 +03:00
committed by GitHub
parent 321879a619
commit 3ed6a0e904
2 changed files with 5 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
spec/components/crawler_detection_spec.rb
View File

@ -51,7 +51,9 @@ describe CrawlerDetection do
end
it "returns true when VIA header contains 'web.archive.org'" do
crawler!("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36", "HTTP/1.0 web.archive.org (Wayback Save Page)")
crawler! "Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot)"
crawler! "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36", "HTTP/1.0 web.archive.org (Wayback Save Page)"
crawler! "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36", "Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; http://archive.org/details/archive.org_bot), 1.1 warcprox"
end
it "returns false for non-crawler user agents" do