Below are two jobs almost every site needs, written for all four, plus a quick check you can run on your redirect list today.

If you'd rather not copy anything by hand, our free .htaccess, Nginx & Caddy Rule Generator writes all of these for you.

1. Force HTTPS and drop www, in one redirect

The goal: http://www.example.com/page goes straight to https://example.com/page in a single hop, not two.

Apache / LiteSpeed (.htaccess)

RewriteEngine On
RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,R=301]

The third condition captures the domain without www., so one rule fixes both problems at once. Behind Cloudflare or another proxy, %{HTTPS} can read off even when the visitor is on HTTPS; use RewriteCond %{HTTP:X-Forwarded-Proto} !https [OR] instead.

Nginx

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

You also need a second server block for https://www.example.com that returns the same redirect, with your certificate on it. No if, no rewrite: a separate server block is the fastest and clearest way.

Caddy

www.example.com, http://www.example.com, http://example.com {
    redir https://example.com{uri} permanent
}

Caddy redirects HTTP to HTTPS on its own, so you only list the addresses that should land on the main one.

Cloudflare

No rule needed: turn on SSL/TLS > Edge Certificates > Always Use HTTPS, and create a redirect rule from the Redirect from WWW to root template under Rules.

2. Block AI training crawlers

The crawlers that collect training data identify themselves in the User-Agent. (List shortened here; the full, current list is in the generator.)

Apache / LiteSpeed

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (GPTBot|ClaudeBot|CCBot|Bytespider) [NC]
RewriteRule ^ - [F,L]

Nginx (the map goes once inside http {}, the if inside your server {})

map $http_user_agent $blocked_ua {
    default 0;
    "~*(GPTBot|ClaudeBot|CCBot|Bytespider)" 1;
}

server {
    # ...
    if ($blocked_ua) { return 403; }
}

Caddy

example.com {
    @ai_train header_regexp User-Agent "(?i)(GPTBot|ClaudeBot|CCBot|Bytespider)"
    respond @ai_train 403
}

Cloudflare (Security > Security rules > Create rule > Edit expression, action: Block)

(lower(http.user_agent) contains "gptbot")
or (lower(http.user_agent) contains "claudebot")
or (lower(http.user_agent) contains "ccbot")
or (lower(http.user_agent) contains "bytespider")

Add the same names to robots.txt as well: well-behaved crawlers stop there, and the server rule catches the rest.

3. The redirect-list mistake: chains

After a few redesigns, a redirect list ends up like this:

/old-page   /new-page
/2019/sale  /sale
/sale       /offers

A visitor to /2019/sale now takes two hops: to /sale, then to /offers. Search engines follow chains, but each hop is wasted time, and long chains get abandoned. Fix the list so every old URL points at its final address:

/2019/sale  /offers
/sale       /offers

Worse is a loop (/a → /b, /b → /a), which redirects forever until the browser gives up. Check for both every time you add lines.

Skip the copy-pasting

We put all of this, and a lot more, into a free tool at PaidHosting.com: the .htaccess, Nginx & Caddy Rule Generator. Flip the switches once and it writes matching rules for Apache, Nginx, Caddy and Cloudflare. It also fixes redirect chains and loops for you, splits AI crawlers into training and search bots, and gives you a share link to your exact setup.

Every output was tested on real Apache, Nginx, Caddy, LiteSpeed and Cloudflare setups before release, and the code is open source (MIT): https://github.com/PaidHost/server-rules-generator. If something misbehaves on your server, open an issue and we'll fix it.