Text/Clean

Clean invisible characters and quotes

Cleans up invisible differences such as NBSP, zero-width characters, BOM, curly quotes and decomposed Korean characters.

Example

  • Options used
  • Default options
Input
“Smart quotes” and⍽non-breaking◌space
한글
Output
"Smart quotes" and non-breakingspace
한글

Symbols in examples: ⇥ tab · · trailing space · ⍽ NBSP · □ full-width space · ◌ zero-width character

Open this operation in the tool

What it does

Cleans up characters that look identical on screen but have different character codes, which gets in the way of searching, comparing and deduplicating. Each item can be switched on and off separately.

Options

  • Curly quotes → straight quotes: converts the “ ” ‘ ’ that word processors and messengers insert automatically into " '. Necessary before pasting into code, CSV or JSON.
  • Special spaces → regular space: converts the no-break space (NBSP, U+00A0) common in web pages, the narrow no-break space (U+202F) and spaces of various widths (U+2000–U+200A) into ordinary spaces. This is a common reason Excel's VLOOKUP cannot find a matching value.
  • Remove zero-width characters and BOM: removes the zero-width space (U+200B), zero-width non-joiner and joiner (U+200C, U+200D), word joiner (U+2060), BOM (U+FEFF) and soft hyphen (U+00AD). These are often the reason a copied URL or hashtag does not work. The zero-width joiner is also used in some emoji sequences, so turn this option off if you need to keep emoji intact.
  • All dashes → hyphen: unifies – — ‒ − into -. Off by default.
  • Ellipsis → three dots: converts the single character … into .... Off by default.
  • Unicode NFC normalization: file names created on macOS and some text store Korean in decomposed form (NFD), with a syllable split into its letters like ㅎ ㅏ ㄴ. NFC normalization recombines them into the precomposed 한. Accented Latin letters are recombined the same way.

When to use it

The first thing to suspect when other cleanup steps give odd results. It is safe to put it at the top of most recipes. Full-width letters and digits are handled by Full-width ↔ half-width.