need help with a simple bash script

[syntax]

Hey guys,
So this is not really my thing, but I need help with a bash script that will take a whole bunch of text files in a directory, search through them for specific (or exclude) text and put the output to a file.

This is what I have
for f in * ;
do
cat *.txt | grep textiwant1 > /var/output/test.txt ;
cat *.txt | grep textiwant2 >> /var/output/test.txt ;
done

The problem with this, is that it goes through all the files using the first statement, then goes through all the files using the second statement.
I need it to go through the first file with both statements, then go through second file with both statements and so on.

/EDIT/

I see why its going through each file, its because of my cat *.txt statement, how do i tell it to do the first file, then increment.
Im sure i can use a while loop and then increment a variable to show a file incrementation, but i dont know the statements

 

[nand]

Here you go:

grep -liP "(first term|second term)" *.txt > /var/output/test.txt

This will perform a regular expression on each file. The regular expression itself contains a group, indicated with parentheses, of strings to search for separated with a piping symbol.
The parameters passed to the grep command says that it should only print file names(-l), ignore case(-i) and perform a PCRE regular expression(-P).
It then redirects standard out into a file.

Hope it helps.

 

[syntax]

Here you go:

grep -liP "(first term|second term)" *.txt > /var/output/test.txt

This will perform a regular expression on each file. The regular expression itself contains a group, indicated with parentheses, of strings to search for separated with a piping symbol.
The parameters passed to the grep command says that it should only print file names(-l), ignore case(-i) and perform a PCRE regular expression(-P).
It then redirects standard out into a file.

Hope it helps.

thanks, but i dont think i quite understand or i didnt explain properly

so

for f in * ;
do
grep -liP "(hostname|ip)" *.txt > /var/get_log/test.txt
done

Just spits out the filenames?

Let me give a better example.

File A contains
carA
houseA
junk
rubbish

File B contains
junk
houseB
carB
rubbish

I want to search both files for the words house and car. I want to output this to a text file, but this must be in order, so the output file should look like
houseA
carA
houseB
carB

It will put it out in this order, because i will specifically state that those are the files i want in that order.
I also need to be able to include grep -v to exclude certain text from the files.

so manually i would do the below (and for more elaborate reasons im not going to really go into here, i also need to exclude certain things)
cat FileA | grep house | grep -v junk > test.txt
cat FileA | grep car | grep -v rubbish >> test.txt
cat FileB | grep house >> test.txt
cat FileB | grep car >> test.txt

 

[syntax]

ok,

so messing around im getting an idea that i need to do something like
for dir;
do
cat $dir | grep xxxxxxxxx > xxxx.txt
cat $dir | grep yyyy >> xxxx.txt
done

or something similar, which would then cycle through the first file, run the cat and greps that i need, then move onto the next file?

 

[nand]

egrep -ih "house|car" *.txt | egrep -v "junk|rubbish" > /tmp/test

However, the sorting is unusual - the file's contents will be outputted as they matched(left to right).
The best I can do is to sort it alphabetically:

for file in $(ls *.txt); do
    grep -ih "house|car" $file | sort > /tmp/test
done

 

[syntax]

Ok,

so i managed to do it

for i in $(dir *.txt); 
do cat "$i" | grep xxxxx >> xxx.txt;
cat "$i" | grep yyyy >> xxx.txt;
done;

Thanks for the help anyways

 

[icyrus]

grep can read the file just fine, why bother with cat?

If the you wanted to matches displayed in the order they're found you could've just used an or flag:

grep "house\|car" *.txt

But with your requirement I don't think that'd work.

 

[Nephew_]

grep can read the file just fine, why bother with cat? ...

+1

And I think it has a memory and performance penalty to first do a cat.

Sure by default when you use grep on a wildcard for file names, it shows the filename, but you just add the switch "-h" to not display the filename.

 

[RNJudas]

So your saying that this:

egrep 'xxx|yyy' *.txt

is not what you want?

Your bash script is passing all those files twice because you tell it to.

If you want to learn Regex PM me. I will teach it to you in 3-5 hours. Or you could take a 6month programming course....

Occam's razor young grasshopper.

As for excluding matches, grep and egrep are the same file. In all cases first match wins unless explicitly indicated. That being said your example has one statement per line so it wont print things you dont want. but lets say it does, well then its time to include SED and AWK, then run the good old sort and uniq where needed

 

[biena]

If you want to learn Regex PM me. I will teach it to you in 3-5 hours

@a=(Lbzjoftt,Inqbujfodf, Hvcsjt); $b="Lbssz Wbmm" ;$b =~ y/b-z/a-z/ ; $c = " Tif ". @a ." hsfbu wj" ."suvft pg b qsphsbnnfs" . ":\n";$c =~y/b-y/a-z/; print"\n\n$c ";for($i=0; $i<@a; $i++) { $a[$i] =~ y/b-y/a-z/;if($a[$i]eq$a [-1]){print"and $a[$i]." ;}else{ print"$a, "; }}print"\n\t\t--$b\n\n";

You must be a very good teached if you can get someone to write the above in 3-5 hours

Actually this one is more fun

http://ex-parrot.com/~pdw/Mail-RFC822-Address.html

 

[RNJudas]

You must be a very good teached if you can get someone to write the above in 3-5 hours

And thats why so many noobs run, sadly programmers assume themselves better than everyone. I am not sure in what context the code above is used but I will say that even PERL has it place. That being said in everyday knowledge people fail to see that PERL just heavily relies on the underlying system where its regex is similar to the SEARCH and REPLACE from SED.

So to clarify my statement. I can teach you regex in 3 - 5 hours. Programming at any level if your dedicated should be done in 10 years! not 21 days.

PS biena, dont be the kind of person that waves their EPenis around simply to intimidate NUBS... it gives the rest of the rest willing to help a harder time...

I am sure you understand