Regex for URLs: what to match and what to leave to a parser

"URL regex" usually means one of two things:

1. Finding URLs inside a block of text, like a chat app that turns links into something you can click.
2. Validating one input, like a "Website" field on a profile form.

They need different patterns, and the second one is mostly a job for a URL parser. This post covers both, plus the traps that catch people in each. Every example is live, so edit the patterns and see what changes.

01Finding URLs in text

The quick version is a scheme followed by anything that isn't a space:

\S+ grabs everything up to the next space, and in real writing a URL is often followed by punctuation. Link that and the reader gets a 404.

02Dropping trailing punctuation

The fix is to say what the last character can be. The body can be anything except spaces, angle brackets and quotes. The final character also can't be common punctuation:

This works because the greedy [^\s<>"]* runs to the end of the URL, then backs off one character at a time until the final class fits. (Our post on greedy and lazy quantifiers walks through that backing off.)

There's no perfect rule here. Wikipedia links like https://en.wikipedia.org/wiki/Regex_(disambiguation) really do end in ), and this pattern trims it. Linkify libraries handle that by checking whether the parentheses are balanced, which is more logic than one regex should carry.

Notice that www.example.com isn't found, because there's no scheme. If you want bare www. links too, start the pattern with (https?://|www\.) instead.

03Validating a URL: the parts

A web URL has up to six parts. Here they are in https://regexhunter.com:443/blog/regex-urls?ref=home#top:

Part	Example	Required?
scheme	https	yes
host	regexhunter.com	yes
port	:443	no
path	/blog/regex-urls	no
query	?ref=home	no
fragment	#top	no

A pattern that checks each part's shape looks like this:

Read it part by part:

1. ^(https?):// the scheme, http or https, and the ://
2. ([^/:?#\s]+) the host: anything up to the next /, :, ? or #
3. (:\d+)? an optional port, digits only
4. (/[^?#\s]*)? an optional path, starting with /
5. (\?[^#\s]*)? an optional query, starting with ?
6. (#\S*)?$ an optional fragment, then the end

It's a good shape check. But it says nothing about whether the host makes sense. https://999.999.999.999 passes, and so does https://-. You could keep adding rules for hostnames, IP addresses, IPv6 brackets, usernames and passwords, and international domain names. People have, and the results run to hundreds of characters.

04Let a parser do the parsing

Every mainstream language ships a URL parser that already knows those rules. In JavaScript (browsers and Node), the URL constructor throws on anything it can't parse:

function isHttpUrl(s) {
  try {
    const u = new URL(s);
    return u.protocol === 'http:' || u.protocol === 'https:';
  } catch {
    return false;
  }
}

new URL("https://exa mple.com"), new URL("https://999.999.999.999") and new URL("example.com") all throw. The protocol check matters: new URL("javascript:alert(1)") parses just fine, and you don't want that in a link.

You also get the pieces for free: u.hostname, u.port, u.pathname, u.search, u.hash, and u.searchParams.get("x") for one query value. And the URL comes back normalized: HTTPS://Example.COM:443/a/../b becomes https://example.com/b.

In Python, urllib.parse.urlsplit splits a URL into the same parts:

from urllib.parse import urlsplit

u = urlsplit("http://localhost:3000/api?x=1#top")
(u.scheme, u.hostname, u.port, u.path, u.query, u.fragment)
# ('http', 'localhost', 3000, '/api', 'x=1', 'top')

But urlsplit is a splitter, not a validator. It rarely raises an error. urlsplit("example.com") returns an empty scheme and host with example.com as the path, and urlsplit("https://exa mple.com") happily reports exa mple.com as the host. So check u.scheme in ("http", "https") and that u.netloc isn't empty, and add your own host rules if you need them.

05Trap: the unescaped dot

In a pattern, . means "any character". Forget the backslash in a domain name and you've allowed lookalike domains:

Write www\.example\.com. Every literal dot in a pattern needs a backslash.

06Trap: the missing end

This one matters for redirect and allow-list checks, like "only send people back to URLs on our own site":

The host has to stop right after .com. Require one of the characters that can end it, or the end of the string:

Better still for security checks: parse with new URL() or urlsplit, then compare the hostname with === or ==. An exact string comparison has no edge cases to forget.

07Try one

Every monster is a web address. The innocents use the wrong scheme, a broken ://, a typo, or no scheme at all:

08Checklist

• To find URLs in text: https?://[^\s<>"]*[^\s<>".,;:!?)], so trailing punctuation stays out.
• To validate one URL: use new URL() in JavaScript and check the protocol, or urlsplit in Python and check scheme and host.
• Escape every literal dot: example\.com.
• In allow-lists, anchor both the start and the end of the host, or better, compare parsed hostnames.
• Add the i flag if capitals can turn up. HTTPS://EXAMPLE.COM is a valid URL.

09Keep practising

• The Real-world validators track in the Academy includes URL slugs, emails, IP addresses and more.
• The escaping section of the guide covers the backslash rules that the dot trap is about.
• Try today's hunt, or browse past boards in the archive.