Text Runs in your browser

Duplicate Lines Remover

Remove duplicate lines from a list — keep first or last, case-insensitive, trim before comparing, sort, and report removed lines. 100% private.

100% Private Works Offline Instant

Start from a preset

Include my text in the link

What counts as the same line

Case-sensitive

Turn off so Apple and apple match.

Fold case properly

German ß matches ss.

Ignore accents

Café matches Cafe.

Trim whitespace

Ignore spaces at each end.

Collapse inner spaces

Runs of spaces or tabs count as one.

Ignore punctuation

A trailing comma stops mattering.

Ignore this leading text

Ignore this trailing text

Compare from character

0 means the start.

Compare to character

0 means the end.

Compare only column

0 means the whole line.

Column delimiter

Leave empty to detect it.

Compare using a pattern

A regular expression. Invalid ones are reported, not applied.

Pattern group

0 means the whole match.

Show what is being compared

See the key for the first few lines.

What to do with matches

Action

Keep one of each and drop the rest.

Work on

Which one survives

Matters when your rules ignore case or spacing.

Where the survivor sits

Blank lines

Paragraph spacing is preserved. This is the default.

Line endings

Your text uses LF.

Strip trailing whitespace

Cleans the output, not just the comparison.

Sort the result

Replaces the original order.

Sort by

Plain A to Z. item10 comes before item2.

Direction

Language for sorting

Leave empty for your browser's language.

Your text

0 characters. LF line endings.

Result

Your cleaned text will appear here…

What happened

In: 0
Duplicates removed: 0
Blank lines removed: 0
Out: 0
Distinct values: 0
Appear once only: 0
Duplicate rate: 0%
Characters: 0 → 0
Bytes: 0 → 0
Line endings: LF

0 in − 0 duplicates − 0 blank lines = 0 out. Every line is accounted for.

Take it away

Format

Every download carries a short note of the settings used, so a colleague can tell what was done.

Private by design. Your text is processed in this tab. There is no upload, no tracking and no network request in this tool, and it works offline. Settings, presets and history are stored in this browser only.

How to use it

1.Paste or drop your text. Anything with repeated lines: a list, an export, a CSV.
2.Say what counts as the same. Ignore case, spaces, accents or punctuation, or compare only one column or part of the line.
3.Choose what to do. Remove the repeats, or keep only the repeats, count them, group them, or just mark them.
4.Check and take it. The report shows exactly what was removed. Copy it, or download it in any of six formats.

What this tool assumes

Blank lines are not duplicates of each other. Paragraph spacing survives unless you ask for it to go.
The line endings you gave are the line endings you get back, unless you choose otherwise.
Input minus what was removed always equals the output, and the tool shows that arithmetic.
Every removal is reported with its original line number, so nothing disappears without a trace.
A matching rule that makes visibly different lines match will say so.

Questions

Why did my blank lines disappear before?

The old version treated every blank line as a duplicate of the first blank line, so all but one were removed from the whole document, and that was the default. Now blank lines are kept unless you ask otherwise.

Some switches used to do nothing. Why?

The interface read option names the engine did not have, so the trim and keep-empty switches were wired to nothing and stayed permanently off. Every control is now connected to a real option.

Will my Windows file still be a Windows file?

Yes. CRLF line endings are detected and restored. The old version silently converted everything to LF.

What is the difference between keep first and keep last?

When your matching rule ignores case or spacing, the repeats are not identical, so it decides which text survives. You can also choose whether the survivor sits in the first or the last position.

Why does sorting warn me about keep first?

Sorting replaces the original order, so where a line came from stops mattering. The choice still decides which text survives, just not where it lands.

Alphabetical or natural, which do I want?

Alphabetical is plain A to Z, so item10 comes before item2. Natural reads the digits as numbers, so item2 comes first. The old tool labelled natural sort as alphabetical.

What happens to lines that are not numbers in a numeric sort?

They are grouped at the end and counted in a message. The old version turned them into zero, so text landed silently among your numbers.

Can I dedupe a CSV on one column?

Yes. Choose CSV rows, pick the column, and keep the header out of the comparison. Quoted commas are handled properly, so a field like "Smith, John" is not split.

Does anything leave my browser?

No. Everything runs in this tab, there is no network request in this tool, and it works offline.

Related tools

Text Sorter — Order the cleaned list, with more sort keys.
Word & Character Counter — Measure the list before and after.
Diff Checker — Compare the original against the cleaned version.
Text Trimmer — Strip whitespace before deduplicating.

Also known as: duplicate line remover, remove duplicate lines, dedupe text, deduplicate list, unique lines only, find duplicate lines, remove repeated lines, uniq online, list deduplicator, clean duplicate entries.

About Duplicate Lines Remover

Remove duplicate lines from a list — keep first or last, case-insensitive, trim before comparing, sort, and report removed lines. 100% private. Everything runs locally in your browser — your data never leaves your device.

How to use

Enter your input in the tool above.
Adjust any options to your preference.
Use the Copy or Download buttons to save the result.
Everything happens locally — your data never leaves your browser.

FAQ

Can I keep the last occurrence instead of the first?

Yes. Select 'Keep last' in the options — the tool keeps the last occurrence of each duplicate line instead of the first.

Related tools

Abstract Generator

Generate extractive summaries from long text with IMRaD-aware scoring. 100% private. 100% private.

Open

Acronym Expander

Expand common acronyms and abbreviations with a configurable dictionary. 100% private. 100% private.

Open

Acronym Generator

Generate backronyms from keywords with custom dictionaries and random modes. 100% private. 100% private.

Open

All tools

Duplicate Lines Remover

What counts as the same line

What to do with matches

Result

What happened

Take it away

How to use it

What this tool assumes

Questions

Related tools

About Duplicate Lines Remover

How to use

FAQ

Can I keep the last occurrence instead of the first?

Related tools

Abstract Generator

Acronym Expander

Acronym Generator

Search tools and actions

What counts as the same line

What to do with matches

Result

What happened

Take it away

How to use it

What this tool assumes

Questions

Related tools