uniq

Uniquify files, write out the unique lines from the given InputFile
uniq only removes adjacent matching lines.
If an InputFile of - (or nothing) is given, then uniq will read from standard input.

Syntax
      uniq [options]... [InputFile [OutputFile]]

Options

   -N
   -f N
   --skip-fields=N
       Skip N fields on each line before checking for uniqueness.  Fields
       are sequences of non-space non-tab characters that are separated
       from each other by at least one spaces or tabs.

   +N
   -s N
   --skip-chars=N
       Skip N characters before checking for uniqueness.  If you use both
       the field and character skipping options, fields are skipped over
       first.

   -c
   --count
       Print the number of times each line occurred along with the line.

   -i
   --ignore-case
       Ignore differences in case when comparing lines.

   -d
   --repeated
       Print only duplicate lines.

   -D
   --all-repeated
       Print all duplicate lines and only duplicate lines.  This option
       is useful mainly in conjunction with other options e.g., to ignore
       case or to compare only selected fields.  This is a GNU extension.

   -u
   --unique
       Print only unique lines.

   -w N
   --check-chars=N
       Compare N characters on each line
       (after skipping any specified fields and characters).
       By default the entire rest of the lines are compared.

By default, `uniq' prints the unique lines in a sorted file, i.e., discards all but one of identical successive lines. Optionally, it can instead show only lines that appear exactly once, or lines that appear more than once.

To remove all duplicate lines, the input must first be sorted. For non-sorted data, an alternative command is sort -u (sort unique)

If no OutputFile file is specified, uniq writes to standard output.

Examples

Count the frequency of some words:

echo "one two three one three" | tr -cs "A-Za-z" "\n" | sort | uniq -c | sort -n -r

“The unique the complex, the extraordinary and irreplaceable Diana, whose beauty, both internal and external, will never be extinguished from our minds” ~ Earl Spencer

Related:

sort - Sort text files.
tr - Translate, squeeze, and/or delete characters.


© Copyright SS64.com 1999-2014
Some rights reserved