Shell脚本实战20-单词及字母去重排序

发表于 | 更新于: | 分类于

1. 需求

用Shell处理以下的内容：

the squid project provides a number of resources to assist users design,implement and support squid installations. Please browse the documentation and support sections for more infomation,by theshu training.

要求：

  • 按单词出现的频率降序排序
  • 按字母出现的频率降序排序

提示：这道题适合用awk进行处理。

2. 数据准备

1
2
# cat test.txt
the squid project provides a number of resources to assist users design,implement and support squid installations. Please browse the documentation and support sections for more infomation,by theshu training.

3. 参考脚本

3.1. 按单词出现频率降序排序的参考脚本

3.1.1. 方法1：awk数组法

1
# awk -F "[,. ]" '{for(i=1;i<+NF;i++) array[$i]++} END {for(key in array) print array[key],key|"sort -nr"}' test.txt | column -t

3.1.2. 方法2：sort排序法

1
# tr "[ ,.]" "\n" < test.txt | grep -v "^$" | sort | uniq -c | sort -rn

3.1.3. 方法3：tr配合awk数组法

1
# tr "[ ,.]" "\n" < test.txt | sed '/^$/d' | awk '{++$[$0]} END {for(key in S) print S[key]" " key | "sort -rn"}'

3.2. 按字母出现频率降序排序的参考脚本

3.2.1. 方法1：awk数组法

1
# tr "{ |,|.}" "\n" < test.txt | awk -F "" '{for(i=1;i<=NF;i++)array[$i]++}END{for(key in array)print array[key],key | "sort -nr"}'

3.2.2. 方法2：grep及sort排序法

1
# grep -o "[^ ]" test.txt | sort | uniq -c | sort -rn -k1
0%