You have a given list, and want to remove duplicates, quickly and easily ?

One of the possible solutions, free and effective, is to use Notepad++ [1], a free text editor, but, on top of it, it is light, and offers lot of interesting functionalities :

• files are opened in tabs,
• syntax highlight (files in a known format are coloured),
• ability to cancel a lot of operations,
• duplicates removing,
• comparison of several files, …

You will also have to install TextFX plugin. To do so, go on SourceForge [2], and download the latest plugin version (Fig 10). You then just have to extract downloaded archive in your Notepad++ installation folder (Fig 11).

Once Notepad++ is launched, you can have a file containing duplicates (Fig 1).

In order to delete them, in a first time, verify that the option “+Sort outputs only UNIQUE (at column) lines” (Fig 2) is set, and then select your data (Fig 3).

You now got two choices :

• delete duplicates in case insensitive mode (Fig 4), as result you will get lines containing same letters as other lines deleted (Fig 5),
• delete duplicates in case sensitive mode (Fig 6), as result you will get lines identical to other ones deleted (Fig 7).

You can then, with the two newly created list, after having them copied in different files, compare them (Fig 8), with, as result, the display in first file of lines that do not exist in the second one, and, in the second one, the display of lines that do not exist in the first file (Fig 9).