Text Cleaner

About the Text Cleaner

Text copied from web pages, PDFs, Word documents or chat apps often carries baggage: HTML tags, & entities, curly quotes, emoji, links, invisible zero-width characters, double spaces and hard line wrapping. The Text Cleaner applies the clean-ups you tick, always in the same order, and shows how many changes each one made, so you know exactly what happened to your text.

HTML is converted to text by a small built-in parser: tags are removed, scripts and styles are dropped, and paragraphs, line breaks, list items and table cells become line breaks and tabs. Nothing in the HTML is run or loaded, and your text never leaves the browser.

Common uses

• Turning an HTML email, newsletter or web page source into readable plain text.
• Preparing copy for a CMS, spreadsheet or code that only accepts plain ASCII quotes and dashes.
• Removing hidden zero-width and direction-control characters that break search, comparisons and code.
• Unwrapping hard line breaks from PDFs and emails before editing or translating.
• Stripping emoji, links or email addresses before analysing or publishing text.

Order of operations

1. Remove HTML tags
2. Decode HTML entities
3. Normalise Unicode (NFC or NFKC)
4. Remove zero-width and control characters
5. Smart quotes and dashes → plain ASCII
6. Remove emoji
7. Remove URLs
8. Remove email addresses
9. Collapse repeated spaces
10. Trim spaces at the start and end of lines
11. Join lines (keeping paragraphs, or into one line)
12. Remove blank lines

The order matters. Tags are removed before entities are decoded, so an encoded <b> in your text ends up as the visible text <b> instead of being treated as a tag. Space and line clean-ups come last, so they also tidy the gaps left by removed emoji, links and tags.

Which invisible characters are removed

Zero-width spaces (U+200B), word joiners (U+2060), byte-order marks (U+FEFF), soft hyphens (U+00AD), left-to-right and right-to-left marks, bidirectional embedding, override and isolate controls, and ASCII and C1 control characters. Tabs and line breaks are kept, and Unicode line and paragraph separators become normal line breaks.

Hidden tag characters (U+E0000–U+E007F), which can carry invisible text, are removed too — except inside the flag emoji for England, Scotland and Wales, which need them. Zero-width joiners and non-joiners are kept where they are part of an emoji or of a word in a script that uses them, such as Devanagari or Persian, and removed everywhere else.

Bidirectional controls are the characters behind "Trojan Source" attacks (CVE-2021-42574), which make source code display differently from how it runs.

NFC or NFKC?

NFC (canonical composition) only merges letters and combining accents into single characters. The text looks exactly the same, but searching and comparing become reliable.

NFKC also replaces compatibility characters with plain equivalents: the ligature fi becomes fi, full-width Ａ becomes A, ① becomes 1, superscript ² becomes 2 and the no-break space becomes a normal space. That is useful for search keys, usernames and data cleaning, but it can change meaning — x² becomes x2 — so use NFC for text that people will read.

Limitations

• Remove HTML tags works on HTML source. It does not apply CSS or run scripts, so text hidden with CSS is kept and text that a page adds with JavaScript is not there to keep.
• Entity decoding knows every named entity from HTML 4 plus the commonly used HTML5 names; rare HTML5 names such as ⪢̸ are left unchanged. Numeric references like ’ and ’ always work.
• Emoji are recognised with your browser's Unicode data. Emoji newer than that data may be missed.
• URL removal recognises links that start with http://, https://, ftp:// or www.. Bare domains such as example.com are kept.
• Smart-quote conversion cannot tell an apostrophe from a closing single quote; both become '.

Privacy

Everything happens in your browser. What you enter or open here is not uploaded or stored by MySmartCoPilot.