Customize URL Rewrites

Rewrites transform known URLs right after they are resolved and before they are fetched or compared. They also run on the URLs a server answers with, a response URL or a redirect target. Feedcanon has not fetched that rewritten form, so it fetches it once and uses it only if it serves the same feed. Otherwise it keeps the URL the server sent. When the rewritten form redirects permanently, the redirect target is used, and cleanUrlFn may only edit its query. A target a rewrite would change again is ignored, so the rewritten form is kept. Rewrites are useful for:

• Consolidating domain aliases to a single canonical domain
• Transforming platform-specific URL patterns
• Handling URL redirects at the domain level

Interface

Each rewrite has two methods:

type Rewrite = {
  match: (url: URL) => boolean
  rewrite: (url: URL) => URL
}

match

Return true if this rewrite should process the URL:

match: (url) => url.hostname === 'feeds.example.com'

rewrite

Transform the URL and return it. The URL object is mutable:

rewrite: (url) => {
  url.hostname = 'canonical.example.com'
  return url
}

Built-in

WARNING

No URL rewrites are enabled by default. To use built-in rewrites, you must explicitly pass them via the rewrites option.

FeedBurner

Transforms various FeedBurner domain aliases. Since FeedBurner URL structure is well-known, the rewrite also normalizes URLs (strips query params, trailing slashes, etc.):

From	To
https://feedproxy.google.com/example?format=xml	https://feeds.feedburner.com/example
https://feeds2.feedburner.com/example/	https://feeds.feedburner.com/example

Blogger / Blogspot

Transforms Blogger and Blogspot URLs to canonical form. Since Blogger/Blogspot URL structure is well-known, the rewrite also normalizes URLs:

From	To
https://example.blogspot.co.uk/feeds/posts/default	https://example.blogspot.com/feeds/posts/default
https://example.blogspot.de/atom.xml	https://example.blogspot.com/feeds/posts/default
http://www.example.blogspot.com/feeds/posts/default	https://example.blogspot.com/feeds/posts/default
http://blogger.com/feeds/123/posts/default	https://www.blogger.com/feeds/123/posts/default

Transformations applied:

• Country-specific TLDs (.blogspot.co.uk, .blogspot.de) → .blogspot.com
• Legacy paths: /atom.xml → /feeds/posts/default, /rss.xml → /feeds/posts/default?alt=rss
• HTTP → HTTPS
• blogger.com → www.blogger.com
• www.example.blogspot.com → example.blogspot.com, since the Blogspot certificate does not cover the www form
• Strips Blogger API params that do not change which feed is served: alt=atom, alt=json, an empty alt and v
• Keeps alt=rss, which selects the RSS version of the feed
• Keeps redirect: without redirect=false, a blog that set up FeedBurner redirects its feed there, and the FeedBurner feed can carry different content
• Keeps orderby, pagination (max-results, start-index) and date filters (published-min, published-max, updated-min, updated-max): a filtered view serves a different set of posts

Examples

Using Built-in Rewrites

import { findCanonical, feedburnerRewrite, bloggerRewrite } from 'feedcanon'

const url = await findCanonical('https://feedproxy.google.com/example', {
  rewrites: [feedburnerRewrite, bloggerRewrite],
})

Custom Domain Alias

import { findCanonical, feedburnerRewrite } from 'feedcanon'

const url = await findCanonical('https://old.example.com/feed', {
  rewrites: [
    feedburnerRewrite,
    {
      match: (url) => url.hostname === 'old.example.com',
      rewrite: (url) => {
        url.hostname = 'new.example.com'
        return url
      },
    },
  ],
})