Regular Expressions with re
What regex is actually for, when it's overkill, the re module built up systematically, and a real log-parsing example.
Advanced Python Starts Here
This module opens Phase 5 — Advanced Python, the most demanding phase of this track before you move into production-readiness and career topics in Phase 6. Everything up to this point — variables, control flow, functions, data structures, files, exceptions, object-oriented Python — was about writing correct programs. Phase 5 is about writing programs that hold up under real-world pressure: text that does not arrive in a clean shape, time zones that do not line up, work that needs to happen concurrently instead of one line at a time, and code whose types and behaviour need to survive contact with other engineers and other systems. Regular expressions are the right place to start, because pattern matching in text turns out to be a dependency of almost everything else in this phase — log parsing, API response cleanup, and data validation all lean on it constantly.
A regular expression (regex, for short) is a small, dense language for describing a pattern that text either matches or does not. Instead of writing loops and conditionals to inspect a string character by character, you describe the shape you are looking for once, and Python's re module does the character-by-character work for you. Regex shows up constantly in real engineering work: validating that a string looks like an email address before accepting it, pulling a request ID out of a log line, splitting a messy CSV field on multiple possible delimiters, or finding every URL inside a block of text.
import re
zip_code = "80202"
if re.match(r"^\d{5}$", zip_code):
print("Looks like a valid 5-digit zip code")
else:
print("Not a valid zip code shape")
# "Looks like a valid 5-digit zip code"That single pattern — ^\d{5}$ — says "exactly five digits, and nothing else." By the end of this module you will be able to read that immediately. Regex syntax looks intimidating in bulk, but it is built from a genuinely small set of building blocks combined together — this module introduces them one at a time, in order, so nothing feels like it appeared from nowhere.
An Honest Take Before You Learn the Syntax
Before diving into syntax, it is worth being direct about something most tutorials skip: regex is frequently the wrong tool, reached for out of habit rather than necessity. You already know Module 04's string methods — .startswith(), .endswith(), .split(), in — and Module 14's string-processing techniques. For a huge share of everyday text checks, those are faster to write, faster to read, and faster to execute than a regex.
# Checking a prefix — regex is unnecessary here
if filename.startswith("invoice_"): # clear
...
# vs
if re.match(r"^invoice_", filename): # works, but adds no value over the line above
# Checking for a substring — same story
if "error" in log_line.lower(): # clear and fast
...
# vs
if re.search(r"error", log_line, re.IGNORECASE): # heavier than it needs to beReach for regex specifically when the pattern you are matching has real structure that plain string methods cannot express — "one or more digits, optionally followed by a decimal point and more digits," or "a sequence of letters, then a dash, then exactly four digits." That is a genuinely different kind of problem than "does this string start with a fixed prefix," and it is exactly the kind of problem this module is about.
re.match, search, findall, sub, split
The re module is part of Python's standard library — no installation required. Five functions cover the overwhelming majority of real regex usage, and it is worth learning the difference between them precisely, because mixing them up is one of the most common regex mistakes in real code.
import re
re.match(r"\d+", "42 apples") # matches — "42" is found at position 0
re.match(r"\d+", "I have 42 apples") # None — the string does not START with digitsre.search(r"\d+", "I have 42 apples") # matches — finds "42" anywhere in the string
re.search(r"\d+", "no numbers here") # None — no digits anywherere.findall(r"\d+", "I have 42 apples and 7 oranges")
# ['42', '7'] — a plain list of strings, every match foundre.sub(r"\d+", "#", "I have 42 apples and 7 oranges")
# "I have # apples and # oranges"
# Redacting something sensitive — a real, common use
re.sub(r"\d{3}-\d{2}-\d{4}", "XXX-XX-XXXX", "SSN on file: 123-45-6789")
# "SSN on file: XXX-XX-XXXX"re.split(r"[,;]\s*", "apples, oranges;bananas, grapes")
# ['apples', 'oranges', 'bananas', 'grapes']
# str.split(",") alone could not handle the mix of commas AND semicolonsNotice the recurring shape: match and search return a Match object (or None if nothing matched) — not the matched text directly. findall and split return plain lists. sub returns a new string. Getting the actual matched substring out of a Match object is covered in Part 06 below, once groups are introduced.
if re.search(r"\d+", text): works because a Match object is always truthy and None is always falsy — but printing that match object directly gives you something like <re.Match object; span=(7, 9), match='42'>, not "42". You need .group() to extract the actual text, shown in Part 06.From Literal Characters to Character Classes
Literal characters
Most characters in a regex pattern simply match themselves. The pattern cat matches the literal text "cat" wherever it appears — nothing special is happening yet.
re.search(r"cat", "concatenate") # matches — "cat" appears inside "conCATenate"The dot . — any single character
. matches exactly one character of any kind (except a newline, by default).
re.findall(r"c.t", "cat cot cut cят c t")
# ['cat', 'cot', 'cut', 'c t'] — "cят" is skipped, since я and т are TWO characters, not oneCharacter classes [ ] — any ONE character from a set
Square brackets define a set of acceptable characters for one position. A hyphen inside brackets defines a range, and a leading ^ inside the brackets negates the set.
re.findall(r"[aeiou]", "hello world") # ['e', 'o', 'o'] — any single vowel
re.findall(r"[a-z]", "Hi 123") # ['i'] — lowercase a through z only
re.findall(r"[A-Za-z0-9]", "Hi 123!") # ['H','i','1','2','3'] — letters and digits, not '!' or the space
re.findall(r"[^0-9]", "abc123") # ['a','b','c'] — ^ inside [] means NOT these charactersShorthand classes: \d \w \s and their negations
Because digit, word-character, and whitespace classes are so common, re provides shorthand for them — and for their opposites.
\d any digit — equivalent to [0-9]
\D any NON-digit — equivalent to [^0-9]
\w any "word" character — letters, digits, and underscore, equivalent to [A-Za-z0-9_]
\W any NON-word character
\s any whitespace — space, tab, newline
\S any NON-whitespace characterre.findall(r"\d", "Room 204, Suite 5B") # ['2','0','4','5']
re.findall(r"\w+", "user_name-42 field!") # ['user_name', '42', 'field'] — the dash and ! break \w+How Many Times, and Where in the String
On their own, character classes only match one character. Quantifiers say how many times the preceding element is allowed to repeat, and anchors pin a match to a specific position in the string rather than letting it appear anywhere.
* zero or more
+ one or more
? zero or one (optional)
{n} exactly n times
{n,} n or more times
{n,m} between n and m times, inclusivere.findall(r"\d+", "room 4, hall 12, gate 007")
# ['4', '12', '007'] — \d+ greedily grabs runs of one or more digits
re.match(r"colou?r", "color") # matches — the 'u' is optional
re.match(r"colou?r", "colour") # matches too
re.match(r"\d{3}-\d{4}", "555-1234") # matches — exactly 3 digits, dash, exactly 4 digits
re.match(r"\d{3}-\d{4}", "55-1234") # None — only 2 digits before the dashAnchors — ^ and $
^ anchors a match to the start of the string, and $ anchors it to the end. Without anchors, a pattern is free to match anywhere inside the string — which is a common source of bugs when a full-string validation was actually intended.
# WITHOUT anchors — this "validation" actually just checks for digits ANYWHERE
re.match(r"\d{5}", "hello 80202 world extra text")
# matches "80202" — but re.match only pins the START, not the end,
# so trailing junk after a valid-looking prefix is silently accepted
# WITH both anchors — this genuinely validates the ENTIRE string
re.match(r"^\d{5}$", "80202") # matches — the whole string is exactly 5 digits
re.match(r"^\d{5}$", "80202-extra") # None — correctly rejectedre.match(r"\d{5}", value) validates that value is exactly five digits. It does not — it only confirms the string starts with five digits, and anything can follow. Always add a trailing $ when the intent is full-string validation, as shown above.Pulling Structured Pieces Out of a Match
Parentheses ( ) in a pattern create a capturing group — a piece of the overall match that you can extract individually afterward, rather than only getting the full matched text back as one block. This is where regex stops being a yes/no check and starts being a real parsing tool.
import re
text = "Order #4471 placed on 2026-08-13"
match = re.search(r"Order #(\d+) placed on (\d{4}-\d{2}-\d{2})", text)
if match:
print(match.group(0)) # the FULL match: "Order #4471 placed on 2026-08-13"
print(match.group(1)) # the FIRST group: "4471"
print(match.group(2)) # the SECOND group: "2026-08-13"Named groups — clearer than counting parentheses
Counting group(1), group(2), and so on becomes error-prone once a pattern has more than two or three groups, especially after the pattern gets edited later and the numbering shifts. Named groups, written (?P<name>...), solve this by letting you retrieve each piece by name.
pattern = r"Order #(?P<order_id>\d+) placed on (?P<order_date>\d{4}-\d{2}-\d{2})"
match = re.search(pattern, text)
print(match.group("order_id")) # "4471"
print(match.group("order_date")) # "2026-08-13"
# .groupdict() returns everything at once, as a regular dict
print(match.groupdict())
# {'order_id': '4471', 'order_date': '2026-08-13'}match.group("order_id") is self-documenting; one that reads match.group(3) forces the reader back up to the pattern to figure out what group 3 even is.Why * and + Grab More Than You Might Expect
By default, quantifiers are greedy — they match as much text as possible while still allowing the overall pattern to succeed. This is a genuinely common source of confusing bugs the first time it bites, especially with HTML-like or delimiter-heavy text.
text = '<b>bold</b> and <i>italic</i>'
re.findall(r"<.+>", text)
# ['<b>bold</b> and <i>italic</i>']
# The greedy .+ matched from the FIRST < all the way to the LAST > —
# almost certainly not what was intended.Adding a ? immediately after a quantifier makes it non-greedy (also called "lazy") — it matches as little text as possible instead.
re.findall(r"<.+?>", text)
# ['<b>', '</b>', '<i>', '</i>']
# .+? stops at the FIRST > it can, giving four separate, correct matchesBeautifulSoup, not re.re.compile() — Reuse and Performance
Every call to a module-level function like re.search(pattern, text) compiles pattern into an internal matching engine representation before running it. Python caches recently-used compiled patterns automatically, so calling the same pattern repeatedly is not catastrophic — but when a pattern is used many times in a loop, or the same pattern is reused across a codebase, compiling it once explicitly with re.compile() is both faster and clearer about intent.
import re
ZIP_CODE = re.compile(r"^\d{5}(-\d{4})?$") # optional +4 extension
candidates = ["80202", "80202-1234", "8020", "abc12"]
for c in candidates:
if ZIP_CODE.match(c):
print(f"{c}: valid")
else:
print(f"{c}: invalid")A compiled pattern object exposes the same methods you have already used — .match(), .search(), .findall(), .sub(), .split() — just called directly on the compiled object instead of passing the pattern string to the module-level function each time.
Why Every Pattern in This Module Starts With r"..."
Back in Module 04, you met raw strings — r"..." — which tell Python not to interpret backslash escape sequences like \n or \t. Regex patterns use the backslash constantly for their own purposes (\d, \w, \s), and those meanings are completely unrelated to Python's own string escape sequences — which creates exactly the kind of collision raw strings exist to prevent.
# WITHOUT r-prefix — Python's OWN string escaping interferes first
pattern = "\d+" # Python sees \d, does not recognize it as a known escape,
# and (in modern Python) leaves it as the two characters \ and d —
# but this is fragile and inconsistent across escape sequences
# For example, \s is fine, but some sequences ARE meaningful to Python itself:
"\t" # this is an actual TAB character to Python, not the two characters \ and t —
# if you meant the regex whitespace escape \s and mistyped \t, you'd get a
# literal tab character in your pattern instead of what you intended
# WITH r-prefix — completely unambiguous, exactly what you typed, character for character
pattern = r"\d+" # r"\d+" is guaranteed to be backslash, d, plus — nothing elser prefix, certain digit/letter combinations after a backslash (like \b, which is a backspace character to Python but a word-boundary anchor in regex) produce silently wrong patterns instead of an error, which makes the bug genuinely hard to spot.Extracting Structured Data From Log Lines
Here is a realistic, complete example that pulls together everything above: parsing semi-structured application log lines into structured data, a task that comes up in nearly every backend engineering role.
2026-08-13 09:14:02 ERROR [order-service] Failed to process order 4471: timeout after 30s
2026-08-13 09:14:07 INFO [order-service] Order 4472 processed successfully
2026-08-13 09:15:33 ERROR [payment-service] Failed to process order 4473: card declinedimport re
LOG_PATTERN = re.compile(
r"^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+"
r"(?P<level>\w+)\s+"
r"\[(?P<service>[\w-]+)\]\s+"
r"(?P<message>.+)$"
)
log_lines = [
"2026-08-13 09:14:02 ERROR [order-service] Failed to process order 4471: timeout after 30s",
"2026-08-13 09:14:07 INFO [order-service] Order 4472 processed successfully",
"2026-08-13 09:15:33 ERROR [payment-service] Failed to process order 4473: card declined",
]
errors = []
for line in log_lines:
match = LOG_PATTERN.match(line)
if match and match.group("level") == "ERROR":
order_match = re.search(r"order (\d+)", match.group("message"))
errors.append({
"timestamp": match.group("timestamp"),
"service": match.group("service"),
"order_id": order_match.group(1) if order_match else None,
"message": match.group("message"),
})
for e in errors:
print(e)
# {'timestamp': '2026-08-13 09:14:02', 'service': 'order-service', 'order_id': '4471',
# 'message': 'Failed to process order 4471: timeout after 30s'}
# {'timestamp': '2026-08-13 09:15:33', 'service': 'payment-service', 'order_id': '4473',
# 'message': 'Failed to process order 4473: card declined'}Notice how the pattern is built from exactly the pieces this module covered, composed together: named groups for the fields you need to extract, \d and \w shorthand classes, quantifiers for repeated digits, and a nested re.search() call to pull the order ID out of the already-extracted message text — a genuinely common pattern, where one regex extracts a broad structure and a second, narrower regex digs further into one piece of it.
A Denver Insurance Platform's Silent Data-Quality Bug
An insurance-quoting startup accepts phone numbers from a web form and needs to validate them before storing them, since downstream systems (SMS notifications, an agent dialer) assume a clean, consistent shape. An engineer writes a quick validation function under deadline pressure.
import re
def is_valid_phone(number):
return re.match(r"\d{3}-\d{3}-\d{4}", number) is not None
is_valid_phone("303-555-0192") # True — correct
is_valid_phone("303-555-0192 ext 4") # True — ALSO accepted, incorrectlyWhat breaks, three weeks later
The dialer system starts throwing errors on a small but growing fraction of stored numbers. The root cause, once found: exactly the missing-anchor bug from Part 05. re.match only pins the pattern to the start of the string, never the end, so any input that merely begins with a valid-looking phone number — with arbitrary trailing text — was silently accepted and stored as-is.
The fix, and the second issue it uncovers
The immediate fix is exactly Part 05's lesson: add a trailing $ anchor, r"^\d{3}-\d{3}-\d{4}$", so the entire string must match, not just a prefix of it. But the fix also surfaces a second, more interesting question in code review: should the validator reject (303) 555-0192 and 3035550192, both of which are genuinely valid phone numbers in a different but common format? The team ultimately normalizes input first — stripping non-digit characters with re.sub(r"\D", "", number) — and validates the normalized 10-digit result, rather than trying to write one pattern that accepts every real-world formatting style directly.
The lesson generalizes well beyond phone numbers: a regex validator that is not fully anchored on both ends does not fail loudly — it fails by silently accepting more than intended, which is exactly the kind of bug that survives testing and only shows up once real, messy user input reaches it in production.
Four Misconceptions About Regular Expressions
5 Interview Questions — With Complete Answers
Regex Mistakes Beginners Make Constantly
Errors You Will Hit With Regex — And Exactly Why
🎯 Key Takeaways
- ✓Regex is for patterns with real structure — repetition, optional parts, alternation. For simple prefix/suffix/substring checks, plain string methods are simpler and faster.
- ✓re.match anchors to the START of the string only; re.search finds the first match anywhere; re.findall returns every match as a list. Know which one you actually need.
- ✓Character classes ([], \d, \w, \s) match one character from a set. Quantifiers (*, +, ?, {n,m}) say how many times the preceding element repeats.
- ✓Anchors ^ and $ pin a match to the start and end of the string. Full-string validation requires BOTH — a missing $ is one of the most common real regex bugs.
- ✓Capturing groups (...) extract pieces of a match individually via .group(n). Named groups (?P<name>...) do the same by name, and stay correct even after the pattern is edited.
- ✓Quantifiers are greedy by default, matching as much as possible. Add ? after a quantifier (+?, *?) for non-greedy matching.
- ✓re.compile() compiles a pattern once for reuse — clearer and faster than repeatedly calling a module-level re function with the same pattern string.
- ✓Always write regex patterns as raw strings (r"...") — Python's own string escaping can otherwise silently corrupt certain backslash sequences before re ever sees them.
- ✓Regex is a weak tool for deeply nested formats like HTML and XML — use a dedicated parser for those instead.
What comes next
Module 33 covers dates and times — the datetime module, timezone-aware datetimes, and the formatting codes that trip up almost everyone the first time they need them.
Module 33 → Working with Dates and TimesDiscussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.