Notepad++ remove duplicate lines

You have a given list, and want to remove duplicates, quickly and easily ?

One of the possible solutions, free and effective, is to use Notepad++ [1], a free text editor, but, on top of it, it is light, and offers lot of interesting functionalities.

You will also have to install TextFX plugin. To do so, go on SourceForge [2], and download the latest plugin version (Fig 10). You then just have to extract downloaded archive in your Notepad++ installation folder (Fig 11).

Notepad++ remove duplicate lines and sort : Fig 10 : TextFX Plugin download on SourceForge

Notepad++ remove duplicate lines and sort : Fig 11 : Plugin installation for Notepad++

Once Notepad++ is launched, you can have a file containing duplicates (Fig 1).

In order to delete them, in a first time, verify that the option “+Sort outputs only UNIQUE (at column) lines” (Fig 2) is set, and then select your data (Fig 3).

Notepad++ remove duplicate lines and sort : Fig 2 : Notepad++ sort unique

Notepad++ remove duplicate lines and sort : Fig 3 : Notepad++ select data

You now got two choices:

- delete duplicates in case insensitive mode (Fig 4), as result you will get lines containing same letters as other lines deleted (Fig 5),

- delete duplicates in case sensitive mode (Fig 6), as result you will get lines identical to other ones deleted (Fig 7).

Notepad++ remove duplicate lines and sort : Fig 4 : Notepad++ sort case insensitive

Notepad++ remove duplicate lines and sort : Fig 5 : Notepad++ duplicate lines removed

Notepad++ remove duplicate lines and sort : Fig 6 : Notepad++ sort case sensitive

Notepad++ remove duplicate lines and sort : Fig 7 : Notepad++ identical lines removed

You can then, with the two newly created list, after having them copied in different files, compare them (Fig 8), with, as result, the display in first file of lines that do not exist in the second one, and, in the second one, the display of lines that do not exist in the first file (Fig 9).

Notepad++ remove duplicate lines and sort : Fig 8 : SENotepad++ compare files

Notepad++ remove duplicate lines and sort : Fig 9 : Notepad++ compare results

Notepad++ sort unique

To sort unique lines in Notepad++, simply install the TextFX plugin, select the text in which you only want to keep unique lines and to sort them, and select the option TextFX > TextFX tools > Sort lines case sensitive (at columns).

Make sure the sort output only unique (at columns) option is checked to keep only unique records and have a full Notepad++ sort remove duplicates operation.