Know Your Pipes and Filters

Pipes and filters are everywhere. Not just in your homes but also in the terminal.

Piyush Kochhar
Aug 4 · 8 min read
Image for post
Image for post
Know your pipes and filters — Created by Piyush Kochhar

What is a Filter? ䷜

Why filter data? 🗂

What is a Pipe? 🚰

$ command 1 | command 2 |… command n
Adriaens Dishman
Anna Tichner
Bank Guerro
Phil Cuniam
Reagen Dobrovolny
Cicily Veale
Coop Joysey
kendrick Liggons
micheal Fray
benn Cork
Micheal Fray

Let’s filter some data!

$ cat sample.txtAdriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorMicheal Fray
$ cat sample.txt | less Adriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily Veale:
$ cat sample.txt | grep -v “Anna Tichner" Adriaens DishmanBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | grep -c "Anna Tichner" 1
$ cat sample.txt | grep -n "Anna Tichner" 2:Anna Tichner
$ cat sample.txt | grep -i "aNNa TiCHner"Anna Tichner
$ cat sample.txt | grep -A2 “Anna Tichner”Anna TichnerBank GuerroPhil Cuniam
$ cat sample.txt | grep -B1 "Anna Tichner"Adriaens DishmanAnna Tichner
$ cat sample.txt | grep -C1 "Anna Tichner"Adriaens DishmanAnna TichnerBank Guerro
$ cat sample.txt | sortAdriaens DishmanAnna TichnerBank GuerroCicily VealeCoop JoyseyMicheal FrayPhil CuniamReagen Dobrovolnybenn Corkkendrick Liggonsmicheal Fray
$ cat sample.txt | sort -rmicheal Fraykendrick Liggonsbenn CorkReagen DobrovolnyPhil CuniamMicheal FrayCoop JoyseyCicily VealeBank GuerroAnna TichnerAdriaens Dishman
$ cat sample.txt | sort -nAdriaens DishmanAnna TichnerBank GuerroCicily VealeCoop JoyseyMicheal FrayPhil CuniamReagen Dobrovolnybenn Corkkendrick Liggonsmicheal Fray
$ cat sample.txt | sort -fAdriaens DishmanAnna TichnerBank Guerrobenn CorkCicily VealeCoop Joyseykendrick LiggonsMicheal Fraymicheal FrayPhil CuniamReagen Dobrovolny
$ cat sample.txt | uniqAdriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | uniq -c1 Adriaens Dishman1 Anna Tichner1 Bank Guerro1 Phil Cuniam1 Reagen Dobrovolny1 Cicily Veale1 Coop Joysey1 kendrick Liggons1 micheal Fray1 benn Cork1 Micheal Fray
$ echo "New text here" | tee newfile.txt newfile2.txtNew text here
$ echo "New text here" | tee -a newfile.txtNew text here
$ echo "New text here" | tee -a newfile.txt >/dev/null
$ cat sample.txt | cut -b 1,4,8AisAacBkePlnRgDCiVCpykdkmhbnrMh
$ cat sample.txt | cut -b 1-4,9-11Adri DiAnnahneBankrroPhiliamReagobrCiciealCoopseykend LimichFrabennkMichFra
$ cat sample.txt | cut -c 1-3AdrAnnBanPhiReaCicCookenmicbenMic
$ cat sample.txt | cut -d " " -f 2DishmanTichnerGuerroCuniamDobrovolnyVealeJoyseyLiggonsFrayCorkFray
$ cat sample.txt | fmtAdriaens Dishman Anna Tichner Bank Guerro Phil Cuniam ReagenDobrovolny Cicily Veale Coop Joysey kendrick Liggons micheal Fraybenn Cork Micheal Fray
$ cat sample.txt | fmt -w 1AdriaensDishmanAnnaTichnerBankGuerroPhilCuniamReagenDobrovolnyCicilyVealeCoopJoyseykendrick
...
$ cat sample.txt | headAdriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn Cork
$ cat sample.txt | head -n 5Adriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen Dobrovolny
$ cat sample.txt | tailAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | tail -n 5Coop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | tr [:lower:] [:upper:]ADRIAENS DISHMANANNA TICHNERBANK GUERROPHIL CUNIAMREAGEN DOBROVOLNYCICILY VEALECOOP JOYSEYKENDRICK LIGGONSMICHEAL FRAYBENN CORKMICHEAL FRAY
$ cat sample.txt | tr a-zA-Z n-za-mN-ZA-MNqevnraf QvfuznaNaan GvpuareOnax ThreebCuvy PhavnzErntra QboebibyalPvpvyl IrnyrPbbc Wblfrlxraqevpx Yvttbafzvpurny Senloraa PbexZvpurny Senl
$ cat sample.txt | sed 's/micheal/Michelle/'Adriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick LiggonsMichelle Fraybenn CorkMicheal Fray
$ cat sample.txt | sed 's/Micheal/Michelle/1'Adriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMichelle Fray
$ cat sample.txt |tr [:upper:] [:lower:] |sed 's/micheal/michelle/g'adriaens dishmananna tichnerbank guerrophil cuniamreagen dobrovolnycicily vealecoop joyseykendrick liggonsmichelle fraybenn corkmichelle fray
$ cat sample.txt | sed '2d'Adriaens DishmanBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | sed '1,5d'Cicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | sed '$d'Adriaens DishmanAnna TichnerBank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn Cork
$ cat sample.txt | sed '/A/d'Bank GuerroPhil CuniamReagen DobrovolnyCicily VealeCoop Joyseykendrick Liggonsmicheal Fraybenn CorkMicheal Fray
$ cat sample.txt | od0000000 062101 064562 062541  071556  042040  071551  066550  0671410000020 005015 067101 060556  052040  061551  067150  071145  0050150000040 060502 065556 043440  062565  071162  005157  064120  0661510000060 041440 067165 060551  006555  051012  060545  062547  0201560000100 067504 071142 073157  066157  074556  005015  064503  0645430000120 074554 053040 060545  062554  005015  067503  070157  0450400000140 074557 062563 006571  065412  067145  071144  061551  0201530000160 064514 063547 067157  006563  066412  061551  062550  0661410000200 043040 060562 006571  061012  067145  020156  067503  0655620000220 005015 064515 064143  060545  020154  071106  074541  0050150000240
$ cat sample.txt | od -b0000000 101 144 162 151 141 145 156 163 040 104 151 163 150 155 ... 0000020 015 012 101 156 156 141 040 124 151 143 150 156 145 162 ...0000040 102 141 156 153 040 107 165 145 162 162 157 012 120 150 ...0000060 040 103 165 156 151 141 155 015 012 122 145 141 147 145 ...0000100 104 157 142 162 157 166 157 154 156 171 015 012 103 151 ...0000120 154 171 040 126 145 141 154 145 015 012 103 157 157 160 ...0000140 157 171 163 145 171 015 012 153 145 156 144 162 151 143 ...0000160 114 151 147 147 157 156 163 015 012 155 151 143 150 145 ...0000200 040 106 162 141 171 015 012 142 145 156 156 040 103 157 ...0000220 015 012 115 151 143 150 145 141 154 040 106 162 141 171 ...
$ cat sample.txt | od -c0000000  A   d   r  i  a   e   n   s       D   i   s   h   m   a   n0000020 \r  \n   A  n  n   a       T   i   c   h   n   e   r  \r  \n0000040  B   a   n  k      G   u   e   r   r   o  \n   P   h   i   l0000060      C   u  n  i   a   m  \r  \n   R   e   a   g   e   n0000100  D   o   b  r  o   v   o   l   n   y  \r  \n   C   i   c   i0000120  l   y      V  e   a   l   e  \r  \n   C   o   o   p       J0000140  o   y   s  e  y  \r  \n   k   e   n   d   r   i   c   k0000160  L   i   g  g  o   n   s  \r  \n   m   i   c   h   e   a   l0000200      F   r  a  y  \r  \n   b   e   n   n       C   o   r   k0000220 \r  \n   M  i  c   h   e   a   l       F   r   a   y  \r  \n0000240
$ cat sample.txt | wc11      22     160

The Startup

Medium's largest active publication, followed by +733K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store