## General

This tool can be very useful to cleanup a wordlist containing duplicate entries.Might be handy for CTF challenges where wordlists are intentionally made large and contain lots of duplicate entries.


## Command

Tool
```Terminal
>$ ./dup-rem.py input-file output-file {Optional arguments: asc, dsc}
```

- asc - dsc                            ---> Sorts the output in a Ascending or descending order


## Other Possibility

Terminal
```
# Combine files together
cat file1.txt file2.txt file3.txt > combined_list.txt

# Sort and remove duplicate
sort combined_list.txt | uniq -u > cleaned_combined_list.txt