regex - Unable to create sed substitution to deduplicate file -
i have file many duplicates of form
a b b c c
which need reduce to
a b c
so wrote sed command: sed -r 's/^(.*)$\n^(.*)$/\1/mg' filename
, file still showing duplicates. i'm sure regex works because tested here.
so doing wrong?
i suspect may related -r
option, i'm not sure (but without invalid reference \1 on
s' command's rhs` error).
either of 2 simpler approaches should work you.
a simple awk
command print line first time maintaining array of printed lines:
awk '!seen[$0]++' file b c
since file sorted can use uniq
also:
uniq file b c
edit: newer gnu-awk versions support in place editing using:
awk -i 'inplace' '!seen[$0]++' file
Comments
Post a Comment