HSTS Protocol, 307 Redirects, and the Implications for SEO
Around the early 2010s, SEOs began talking about SSL (Secure Sockets Layer) and debating the implications it may have on search engine performance. Google had been hinting at the necessity for a while, but formally announced that SSL certificates would become mandatory in January 2017. Websites should employ SSL for a number of reasons beyond security--Google Chrome has released a number of different ways they will penalize non-https sites in their browser, such as displaying a hard-to-ignore red ‘X’ in front of the site in the browser. Not great for user experience.
As a full-time SEO professional, I have encountered my fair share of HTTPS migrations. One of my common recommendations as a "last step" in the process is to set up HSTS (HTTP Strict Transport Security). This is a request made at the browser level, ensuring that browsers always "remember" that a website is now HTTPS, and to never request the HTTP version. Adding this extra layer is another way to prevent users, search engines, and hackers from accessing the HTTP version of your website, which is what would make you vulnerable to security compromises.
Technically speaking, the redirect that is happening within the browser when a site has HSTS enabled is temporary--you will see a 307 in Google Chrome network panel.
There is clearly some confusion around 307s. As John Mueller states in his post titled, “A search-engine guide to 301, 302, 307 & other redirects)”:
"307 redirects: Wait, isn’t this a server-side redirect? No, this is actually your browser trolling you. If you set up HTTPS, 301 redirect from HTTP to HTTPS, and enable HSTS, when you try to access the HTTP version in your browser, it’ll automatically access the HTTPS version, but record it as a 307 redirect. The 307 is a lie :)."
Without context this can be confusing and unclear, but after experiencing this situation firsthand, this post now makes complete and total sense. I do however disagree with the statement that the 307 is a lie—there is a 307 redirect occurring, but it’s happening at the browser level, so you do not need to be concerned about it on the server side for SEO.
To further explore, I ran the same page through an HTTP Header Checker, the redirect is a 301.
I recently ran into this issue with a client migrating their website from non-http to HTTPS on the Shopify platform, which enables the HSTS protocol. Like any good SEO, as soon as the site was migrated, I immediately took to Screaming Frog to crawl the legacy URLs I had collected for my 301 Plan. And there they were, a whole host of 307s--sound the alarms!
However, these redirects had been hard coded as 301s through the Shopify platform. The only places reporting 307s were Google Chrome and Screaming Frog. I ran the site through Deep Crawl and only saw 301s. Therefore, I started to form my theory that Screaming Frog crawls sites more closely to how a browser would, while Deep Crawl crawls at the HTTP Header level. Upon further investigation, I confirmed that Screaming Frog is not caching the HSTS protocol, so every time they hit an HTTP page they are going through that 307 redirect again.
I reached out to support for both tools and received the following response:
Screaming Frog:
"As it's a crawler it inherently behaves differently from a browser, we are not passing referrer information when the spider visits links it finds in a page. We also do not 'cache' the HSTS protocol and only make HTTPS requests."
Basically, what Screaming Frog is saying here is that while their crawler does not behave exactly like a browser, it is still making the same requests that a browser would, which is why we are seeing these 307 status codes. And because they do not cache the protocol,they make the HTTPs request each time, therefore triggering the temporary redirect from HTTP to HTTPS.
Deep Crawl:
"We extract the redirect code from the response header when the page comes back from the server. So if its a 301, we report it as 301; and if its an HSTS response header we report in the "Pages in HSTS" report."
Therefore, we are correct in ascertaining that Deep Crawl makes requests at the header level. So while there are certainly reasons to use both Screaming Frog and Deep Crawl to get a complete and accurate understanding of the technical health of your website, in this case the server response header was really the only element we were concerned about and Deep Crawl helped us to fully understand the severity of this situation.
What This Means For You
If your migration involves moving from HTTP to HTTPS AND enabling HSTS Protocol, then be sure to check all of your pages with a HTTP header checker, or the Deep Crawl tool, along with Screaming Frog, before you jump to any conclusions. This will save you hours of panic and sending “false alarm” emails to clients and developers.
I thought it would be helpful to share this experience and our findings with the SEO community, as at the time I was unable to find many resources directly addressing this occurrence. Reach out to Stella Rising today to learn more about our SEO migration services.
Comments