Locale-aware

Remove duplicate linesVN · en-VN

Choose exactly what counts as equal and which occurrence remains. Original spelling and spacing stay intact unless you request trimming.

Options
Input lines
0
Output lines
0
Removed lines
0

Locale used: en-VN. Your text stays in this tab.

Ready.

How to use it

Clean copied columns, tags, identifiers, or logs while keeping comparison rules and output transformations visible.

  1. Enter one logical item per line.
  2. Choose case, whitespace, normalization, empty-line, occurrence, and output line-ending options.
  3. Run the tool and review the input, output, and removed-line counts.

Worked example

Input
Alpha
alpha
Beta
Output
Alpha
Beta

With case-insensitive comparison and “keep first,” the original first spelling is retained.

Method and definitions

  • Lines are split on CRLF, CR, or LF; empty input has zero lines and a trailing break creates a final empty line.
  • The comparison key applies only the options you select. Stable input order is preserved.
  • NFC and NFKC normalization are opt-in because normalization can change equality.

Limits and edge cases

  • This is exact rule-based matching, not fuzzy similarity detection.
  • Locale-insensitive matching uses locale lowercase conversion, not language understanding.
  • The limit is 100,000 lines and 2 MiB of UTF-8 text.

Privacy

All comparison and output generation happen in this browser tab; nothing is uploaded or retained automatically.