Duplicate Content Issues and How to Solve Them

Let’s talk about the issue of Duplicate Content. It’s estimated that duplicate content makes up about 29% of the web, according to a study by Raven.

You may be thinking “Well, I know plagiarism is wrong and I don’t copy someone else’s content.” However, duplicate content is unintentionally created in most cases. The most common scenario is when a topic is written about more than once by the same company or organization. Another instance when duplicate content occurs is when one webpage can be accessed by multiple URLs. Highly similar content is viewed as duplicate content by Google and greatly affects SEO ranking. Content that has duplicates is unlikely to ever rank as the top search result even when searched for with relevant keywords.

What constitutes duplicate content to humans and bots?

Take the scenario where there are multiple blog posts on the same topic for example. A human reading two highly similar posts may see that there are similarities and differences in each post. However, to a bot, two highly similar posts are equivalent to two of the same post.

In the other scenario where a website has different variations on the URL, bots count each variation as a separate entity. Take a look at this example given by Moz in the article Canonicalization:

http://www.marketing.com
https://www.marketing.com
http://marketing.com
http://marketing.com/index.php
http://marketing.com/index.php?r…

In this example, the URLs above would take us to the same location and as a human, we would likely consider this as one destination, one page. However, a search engine bot would consider this as five individual pages of duplicate content. Another problematic situation arises with product pages for e-commerce websites, where each variation of a product appears to be duplicate content.

When all of this “duplicate” content compete against each other for search engine ranking, this becomes a serious problem.

How Do Duplicate Content Issues Affect SEO?

Search engine bots become confused when there is too much content that appears the same. In the article Canonical Tags, Moz states some of the ways that duplicate content affects SEO:

How Do I Fix Duplicate Content Issues?

One of the ways that some have attempted to fix the duplicate content problem is by giving bots instructions in the robots.txt file not to crawl a page. Although it might sound reasonable, this is not an effective solution.

Canonicalization is the Key to Managing Duplicate Content Issues

What is canonicalization?
“Fixing duplicate content issues all comes down to the same central idea: specifying which of the duplicates is the “correct” one.” – Duplicate Content by Moz

Identifying the canonical version is equivalent to telling search engine bots that a specific page or URL is the one that should be indexed and ranked. There are different approaches to canonicalizing duplicate content. The two methods we are going to cover is the 301 Redirect and the Rel Canonical Tag. Each method has different uses and benefits.

Two Ways to Canonicalize:

When to Use Rel Canonical
The Rel Canonical Tag is especially useful in the situation we listed previously where you have multiple blog posts on the same topic. Since you still want the duplicate blog posts to be accessible to visitors, you would use the Rel Canonical tag on each duplicate to direct rank power and indexing to the page you designate as the canonical. Rel Canonical can even be used across different domains. For example, if you own two websites and have a similar blog post on each site, you can canonicalize the blog post from one website to the other.

Additionally, Hubspot lists in the article Canonicalization 101 some other common situations to use Rel Canonical:

• Product Variation Pages
• Mobile Specific URLs & Subdomain
• Region/Country Specific URLs (if page content is in same language)

How to Use Rel Canonical

1. Identify which of the duplicate pages is the canonical
2. On the duplicate pages, the canonical tag should point to the URL of the canonical page
3. On the master copy, the canonical tag should be self-referential- pointing back to itself with its own URL as the canonical.
4. The tag should look like this, with the absolute URL** of the canonical page replacing where it says http://www.example.com:

<link rel=”canonical” href=”http://www.example.com” />

**Absolute URL: an absolute URL is when the full path is specified, whereas a relative URL is only partially specified. The absolute and relative URL of the page SEO within the folder News might look like this:

Absolute: “http://www.example.com/news/seo”

Relative: “/news/seo”

The canonical tag should be used in the <head> section within the html of the page. Correct placement of any html element including the canonical tag is important for a website to function properly. Improper placement of an html tag can cause serious problems. If you don’t have the html knowledge to manually input the canonical tag, Yoast explains how to set the canonical URL by using the Yoast SEO plugin in the article rel=canonical: the ultimate guide.

Additional Tips to Canonicalize Duplicate Content:

• Canonicalize Exact and Very-Near Duplicates: Remember, bots see very near duplicates as exact duplicates
• Aggregate Unique Content: Consolidate rank power of two similar posts by moving unique content from the duplicate to the canonical post and using the canonical tag accordingly.
• Don’t Be Confusing and Give Clear Directions
  • Specify no more than one rel=canonical for a page. When more than one is specified, all rel=canonicals will be ignored.
  • Don’t use the canonical tag in conjunction with 301 redirects in the same set of duplicate content. For example, don’t use the canonical tag on post 1 to canonicalize post 2, then use 301 redirects on post 2 to redirect back to post 1. (Very confusing.)
• Make Sure Your Target Page Exists and Is Not:
  • A 404 Error (Page Not Found)
  • Blocked by robots.txt
  • Set to “noindex” (blocked from search results)
• Check Auto-Populated Tags: Some CMS, SEO Plugins and E-commerce systems will auto-populate canonical tags. Be sure to check that these tags point to where you actually want them to.

There are a variety of methods to manage duplicate content and we’ve covered the two main ones in this post. The best method that D4 uses to combat duplicate content from multiple posts on the same topic is simply to not create multiple posts. In creating a single post with unique content that exhaustively covers a single topic, the rank power is already consolidated in that one post. When there is new information available on a topic that has already been covered, it can be more advantageous to update the existing post rather than create a new post that may compete for rank power.

Duplicate content is a complicated issue that can have significant impact on SEO. D4 can help with managing duplicate content issues and improving SEO for your website. If you need help creating and managing engaging content, call us at 775-636-9986!