Otdfbt

What is the highest possible temporary AC at level 1, without any help from others?

Second (easy access) account in case my bank screws up

Do simulator games use a realistic trajectory to get into orbit?

How to return a security deposit to a tenant

What is the highest possible permanent AC at character creation?

Generate a Graeco-Latin square

English word for "product of tinkering"

How to signal to my players that the following part is supposed to be played on fast forward?

Why is one of Madera Municipal's runways labelled with only "R" on both sides?

Is using haveibeenpwned to validate password strength rational?

Soft question: Examples where lack of mathematical rigour cause security breaches?

How to handle self harm scars on the arm in work environment?

Share calendar details request from manager's manager

Did Milano or Benatar approve or comment on their namesake MCU ships?

Cycle through MeshStyle directives in ListLinePlot

System.StringException: Unexpected end of expression

How Often Do Health Insurance Providers Drop Coverage?

Winning Strategy for the Magician and his Apprentice

What's up with this leaf?

How to tell your grandparent to not come to fetch you with their car?

What is the actual quality of machine translations?

Overlapping String-Blocks

Where Mongol herds graze

What is wrong with this proof that symmetric matrices commute?

Filter YAML file content using sed/awk

Sed + how to set parameters in sed command in order to display lines from text filefilter based on a field value in awkHow can I add a column to a specific position in a csv file using cat, sed, awk or cut?How well does grep/sed/awk perform on very large files?sed/awk replace a specific pattern under another patternRegex: using awk and sed to match websites by IPvalidate file content with bash Regular Expressionsecho line with var that contains few linesextract field from file using sed or awkHow to print one line below the matching RegEx in AWK or SED

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

2

I have a text file with the following content in it.

$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
dev2_bar.example.com

I would like to filter hosts listed under [prod_env] file in bash.
I tried with sed, awk and grep with my limited knowledge, but I am not sure how to filter them.

Example: I would like to print hosts under prod_env and dev_env. so the final output would be:

foo.example.com 
bar.example.com 
foo_dev1.example.com 
dev2_bar.example.com

awk sed yaml

share|improve this question

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

add a comment | 

2

I have a text file with the following content in it.

$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
dev2_bar.example.com

I would like to filter hosts listed under [prod_env] file in bash.
I tried with sed, awk and grep with my limited knowledge, but I am not sure how to filter them.

Example: I would like to print hosts under prod_env and dev_env. so the final output would be:

foo.example.com 
bar.example.com 
foo_dev1.example.com 
dev2_bar.example.com

awk sed yaml

share|improve this question

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

add a comment | 

I have a text file with the following content in it.

$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
dev2_bar.example.com

I would like to filter hosts listed under [prod_env] file in bash.
I tried with sed, awk and grep with my limited knowledge, but I am not sure how to filter them.

Example: I would like to print hosts under prod_env and dev_env. so the final output would be:

foo.example.com 
bar.example.com 
foo_dev1.example.com 
dev2_bar.example.com

awk sed yaml

share|improve this question

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
dev2_bar.example.com

foo.example.com 
bar.example.com 
foo_dev1.example.com 
dev2_bar.example.com

share|improve this question

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

share|improve this question

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

edited May 21 at 18:36

Jeff Schaller♦

46.4k1166150

asked May 21 at 15:48

smcsmc

205210

asked May 21 at 15:48

smcsmc

205210

4 Answers
4

active

oldest

votes

3

An sed solution:

sed -nEe '/[(prod|dev)_env]/!d;N;:loop' -e 's/.*n//;$p;d;;N;P;/n[/D;bloop' hosts.yml

• 
  /[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found


• 
  N;:loop adds the next line and starts a loop


• inside the loop we remove the first of the two lines with s/.*n//, because it is either the [...env] line or we already printed it in the last loop cycle


• 
  $p;d; prints the remaining lines if we reached the last line while printing


• 
  N;P adds the next line and prints the current one


• 
  /n[/D looks if the next line starts with a [. In this case the first line in the buffer (already printed) can be discarded and we start over with that [ line


• 
  bloop otherwise loop

Instead of adding the next line to the buffer, printing and removing the old one, you can go line by line, but this would require another loop, because you can't start over with D

share|improve this answer

edited May 21 at 17:28

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks for the awesome explanation, this once again reminds me how powerful sed is.
  
  – smc
  May 21 at 17:39
  

  
  
  
  

add a comment | 

2

You could do (on a mac):

tr "n" "t" < hosts.yml | sed $'s%t\[%\n[%g' | grep 'prod|dev' | sed $'s%.*]\t%%g' | tr "t" "n"

1. 
   tr will get everything on one line


2. The first sed will
   break line before each section


3. 
   grep will cut out the lines
   (sections) you want


4. Second sed will remove the [] part.


5. Last tr will break line so you get one hostname per line.

share|improve this answer

answered May 21 at 17:20

Søren FalchSøren Falch

212

add a comment | 

2

awk 'BEGINRS="[";FS="]n" 
 $1 ~/(prod|dev)_env/ print $2' ex1

share|improve this answer

answered May 21 at 17:34

JJoaoJJoao

7,5791930

add a comment | 

0

You can do this using GNU sed as shown:

$ sed -nEe '
 /[/!H;$!d;
 x;1!s/^[(prod|dev)_env]n//p
' hosts.yml

share|improve this answer

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f520209%2ffilter-yaml-file-content-using-sed-awk%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

3

An sed solution:

sed -nEe '/[(prod|dev)_env]/!d;N;:loop' -e 's/.*n//;$p;d;;N;P;/n[/D;bloop' hosts.yml

• 
  /[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found


• 
  N;:loop adds the next line and starts a loop


• inside the loop we remove the first of the two lines with s/.*n//, because it is either the [...env] line or we already printed it in the last loop cycle


• 
  $p;d; prints the remaining lines if we reached the last line while printing


• 
  N;P adds the next line and prints the current one


• 
  /n[/D looks if the next line starts with a [. In this case the first line in the buffer (already printed) can be discarded and we start over with that [ line


• 
  bloop otherwise loop

Instead of adding the next line to the buffer, printing and removing the old one, you can go line by line, but this would require another loop, because you can't start over with D

share|improve this answer

edited May 21 at 17:28

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks for the awesome explanation, this once again reminds me how powerful sed is.
  
  – smc
  May 21 at 17:39
  

  
  
  
  

add a comment | 

3

An sed solution:

sed -nEe '/[(prod|dev)_env]/!d;N;:loop' -e 's/.*n//;$p;d;;N;P;/n[/D;bloop' hosts.yml

• 
  /[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found


• 
  N;:loop adds the next line and starts a loop


• inside the loop we remove the first of the two lines with s/.*n//, because it is either the [...env] line or we already printed it in the last loop cycle


• 
  $p;d; prints the remaining lines if we reached the last line while printing


• 
  N;P adds the next line and prints the current one


• 
  /n[/D looks if the next line starts with a [. In this case the first line in the buffer (already printed) can be discarded and we start over with that [ line


• 
  bloop otherwise loop

Instead of adding the next line to the buffer, printing and removing the old one, you can go line by line, but this would require another loop, because you can't start over with D

share|improve this answer

edited May 21 at 17:28

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks for the awesome explanation, this once again reminds me how powerful sed is.
  
  – smc
  May 21 at 17:39
  

  
  
  
  

add a comment | 

An sed solution:

sed -nEe '/[(prod|dev)_env]/!d;N;:loop' -e 's/.*n//;$p;d;;N;P;/n[/D;bloop' hosts.yml

• 
  /[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found


• 
  N;:loop adds the next line and starts a loop


• inside the loop we remove the first of the two lines with s/.*n//, because it is either the [...env] line or we already printed it in the last loop cycle


• 
  $p;d; prints the remaining lines if we reached the last line while printing


• 
  N;P adds the next line and prints the current one


• 
  /n[/D looks if the next line starts with a [. In this case the first line in the buffer (already printed) can be discarded and we start over with that [ line


• 
  bloop otherwise loop

Instead of adding the next line to the buffer, printing and removing the old one, you can go line by line, but this would require another loop, because you can't start over with D

share|improve this answer

edited May 21 at 17:28

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

sed -nEe '/[(prod|dev)_env]/!d;N;:loop' -e 's/.*n//;$p;d;;N;P;/n[/D;bloop' hosts.yml

share|improve this answer

edited May 21 at 17:28

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

answered May 21 at 17:23

PhilipposPhilippos

6,46511951

2

You could do (on a mac):

tr "n" "t" < hosts.yml | sed $'s%t\[%\n[%g' | grep 'prod|dev' | sed $'s%.*]\t%%g' | tr "t" "n"

1. 
   tr will get everything on one line


2. The first sed will
   break line before each section


3. 
   grep will cut out the lines
   (sections) you want


4. Second sed will remove the [] part.


5. Last tr will break line so you get one hostname per line.

share|improve this answer

answered May 21 at 17:20

Søren FalchSøren Falch

212

add a comment | 

2

You could do (on a mac):

tr "n" "t" < hosts.yml | sed $'s%t\[%\n[%g' | grep 'prod|dev' | sed $'s%.*]\t%%g' | tr "t" "n"

1. 
   tr will get everything on one line


2. The first sed will
   break line before each section


3. 
   grep will cut out the lines
   (sections) you want


4. Second sed will remove the [] part.


5. Last tr will break line so you get one hostname per line.

share|improve this answer

answered May 21 at 17:20

Søren FalchSøren Falch

212

add a comment | 

You could do (on a mac):

tr "n" "t" < hosts.yml | sed $'s%t\[%\n[%g' | grep 'prod|dev' | sed $'s%.*]\t%%g' | tr "t" "n"

1. 
   tr will get everything on one line


2. The first sed will
   break line before each section


3. 
   grep will cut out the lines
   (sections) you want


4. Second sed will remove the [] part.


5. Last tr will break line so you get one hostname per line.

share|improve this answer

answered May 21 at 17:20

Søren FalchSøren Falch

212

tr "n" "t" < hosts.yml | sed $'s%t\[%\n[%g' | grep 'prod|dev' | sed $'s%.*]\t%%g' | tr "t" "n"

share|improve this answer

answered May 21 at 17:20

Søren FalchSøren Falch

212

answered May 21 at 17:20

Søren FalchSøren Falch

212

answered May 21 at 17:20

Søren FalchSøren Falch

212

2

awk 'BEGINRS="[";FS="]n" 
 $1 ~/(prod|dev)_env/ print $2' ex1

share|improve this answer

answered May 21 at 17:34

JJoaoJJoao

7,5791930

add a comment | 

2

awk 'BEGINRS="[";FS="]n" 
 $1 ~/(prod|dev)_env/ print $2' ex1

share|improve this answer

answered May 21 at 17:34

JJoaoJJoao

7,5791930

add a comment | 

awk 'BEGINRS="[";FS="]n" 
 $1 ~/(prod|dev)_env/ print $2' ex1

share|improve this answer

answered May 21 at 17:34

JJoaoJJoao

7,5791930

awk 'BEGINRS="[";FS="]n" 
 $1 ~/(prod|dev)_env/ print $2' ex1

share|improve this answer

answered May 21 at 17:34

JJoaoJJoao

7,5791930

answered May 21 at 17:34

JJoaoJJoao

7,5791930

answered May 21 at 17:34

JJoaoJJoao

7,5791930

0

You can do this using GNU sed as shown:

$ sed -nEe '
 /[/!H;$!d;
 x;1!s/^[(prod|dev)_env]n//p
' hosts.yml

share|improve this answer

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

add a comment | 

0

You can do this using GNU sed as shown:

$ sed -nEe '
 /[/!H;$!d;
 x;1!s/^[(prod|dev)_env]n//p
' hosts.yml

share|improve this answer

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

add a comment | 

You can do this using GNU sed as shown:

$ sed -nEe '
 /[/!H;$!d;
 x;1!s/^[(prod|dev)_env]n//p
' hosts.yml

share|improve this answer

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

$ sed -nEe '
 /[/!H;$!d;
 x;1!s/^[(prod|dev)_env]n//p
' hosts.yml

share|improve this answer

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682

answered May 22 at 8:14

Rakesh SharmaRakesh Sharma

682