exim command line sender verification (script)

[fixx]

Hi Everyone

I need to be able to verify a few email addresses to make sure they still exist. Now what I would like to do is run a script from command line, read an email address from a MySQL db and do a verification of that email address and then check if it passes.

Is there a way I can use exim's sender verification from command line like:

exim --verify_sender someone@somewhere.com

Or something to that extend?

I'm sure you guys get the picture. :wtf:

It doesn't have to be exim necessarily, any other command line tool will be great.

Thanks a mil...

 

[fixx]

The following I found does verification of the domain only, but I want to verify a whole email address:

exim -bvs asdasd@somedomain.co.za

 

[ambo]

is the email address on the local server or on a remote server?

 

[fixx]

is the email address on the local server or on a remote server?

A remote server.

 

[ambo]

A remote server.

That's not generally possible. The server will see this as a dictionary attack and this normally gets you blacklisted pretty quickly.

 

[Tinuva]

Well Callback verification does exist, however I would advise against using it. I believe this is what you are looking for, and there are mail servers out there that can do it for you, however it does have it's own problems.

Now lets see, in what case will this not work. I work at an ISP, and we run a cluster of anti spam servers infront of our mailservers, which deal with filtering out virus's and spam before the mails are sent to the actual mail servers. These servers have a list of domains for which they will accept mail, but they don't know the actual email address. So in this case, your callout verification will always find the sender as valid.

Another example is, if someone enables a catchall account on a domain, in which case the email server again will always give a valid return for a callout verification.

So while this in theory will work, there are most definitely instances in which I won't work.

Hope that helps.

 

[Logo]

That's not generally possible. The server will see this as a dictionary attack and this normally gets you blacklisted pretty quickly.

It is called sender callout verification or callback verification. There are a lot of mail servers that does this. All it does is Exim makes an SMTP connection to the remote mail server and starts the normal SMTP transaction. If it accepts the mail address the sender is verified. If it returns something like User Unknown then it is seen as an email address that can't be verified.

 

[ambo]

It is called sender callout verification or callback verification. There are a lot of mail servers that does this. All it does is Exim makes an SMTP connection to the remote mail server and starts the normal SMTP transaction. If it accepts the mail address the sender is verified. If it returns something like User Unknown then it is seen as an email address that can't be verified.

Sender callout verifications are very unreliable. Most modern mail server configs will rate limit them and start sending out false negatives after a while. Many configs I've seen will also appear to accept the rcpt address all the way until the end of the data block before rejecting and this will give a false positive.

All this is done specifically so that spammers can't simple do sender verifications on random lists of addresses to decide who they should send mail to.

 

[Logo]

Sender callout verifications are very unreliable. Most modern mail server configs will rate limit them and start sending out false negatives after a while. Many configs I've seen will also appear to accept the rcpt address all the way until the end of the data block before rejecting and this will give a false positive.

All this is done specifically so that spammers can't simple do sender verifications on random lists of addresses to decide who they should send mail to.

I think you don't understand the true use of call outs. It is a Sender verification not a recipient verification. On receiving a mail from user x your mail server will verify that the sending address exists. You don't verify that the receiving address exists before trying to send it. That is the job of the receiving server and leads in to point two.

Sites that do accept a mail before checking if the receiving address exists are, to put it bluntly, stupid. Why would you want to accept data that you can't route ? At the end of the data block the sending server can disconnect, think pipelining here, I have sent the data through and ended the transaction because you gave me no errors. So now you are sitting with a mail that you have to generate a bounce for and send it back to the original server. If it was spam you now have a frozen message in your exim queue because the sending server probably accept incoming connections. That bounce message will sit there for a number of days as it retries to deliver the mail. Multiply this by a couple 100 000 spams a day then you are generating unnecessary traffic. Deny the mail immediately if the address is wrong and make it the sending servers problem to deal with it.

Nobody that I have seen rate limits on call outs because they can't see the difference between a call out and a normal SMTP connection. If you are being rate limited then it more than likely means you did not setup caching and are doing a call out for every email which again is wrong. You should only do a call out if you have not seen the address before. And certain well known domains you can exclude from call outs. You can deal with their addresses being spoofed by using DKIM or SPF. These popular domains rate limits their own users so it is not effective for spammers to use those SMTP server, they will however try to spoof the address. Sender Call Outs should also be the last step before accepting data so that you don't do unnecessary call outs. Things like recipient address, open relays, no reverse records, rbl checks, DKIM or SPF should be done first then you actually end up doing call outs on less than 20% of all mail received. And @Tinuva only then should you accept the data and have whatever appliance you use check the data for spam. If the data section fails x amount of times in y minutes from the same domain then it should added to a local blacklist automatically and you will reject it at an earlier step and won't have to handle it until the problem is resolved.

Lastly spammers don't use call outs because their goal is to send as much as possible as quickly as possible. They don't care if the address exists or not they just send to thousands of addresses at a time. A call out will increase the time it takes to send such large volumes and they will be blacklisted on RBLs before their mail run is complete.

 

[ambo]

I think you don't understand the true use of call outs. It is a Sender verification not a recipient verification. On receiving a mail from user x your mail server will verify that the sending address exists. You don't verify that the receiving address exists before trying to send it.

Are you thinking of the VRFY command in the SMTP standards? All current SMTP implementations that I know block this command due to its abuse. Sender call-outs are normally done by opening a new connection back to the sending domain and then issuing the RCPT command and watching for the response.

Sites that do accept a mail before checking if the receiving address exists are, to put it bluntly, stupid. Why would you want to accept data that you can't route ? ..... Deny the mail immediately if the address is wrong and make it the sending servers problem to deal with it.

Backscatter is bad - I agree with you. However the RCPT command is quite early in the transaction. The server can send a '250 OK' to the sender in response to the recipient list and then still reject the mail later in the transaction. There are also a number of potential configs (like many antispam gateways) where a frontend server accepts all mail and only then passes it onwards to the server that has knowledge of what mailboxes are valid.

You can deal with their addresses being spoofed by using DKIM or SPF. These popular domains rate limits their own users so it is not effective for spammers to use those SMTP server, they will however try to spoof the address.

I'm confused. If they fail SPF and DKIM then the MAILFROM is spoofed. Why are you still needing to do the call out after that?

Lastly spammers don't use call outs because their goal is to send as much as possible as quickly as possible. They don't care if the address exists or not they just send to thousands of addresses at a time. A call out will increase the time it takes to send such large volumes and they will be blacklisted on RBLs before their mail run is complete.

I don't understand this comment. The call-out is done on the receiving side and not the sending side so the spammer doesn't have a choice whether its used. Spammers are not stupid though - they don't waste their time sending from invalid source addresses because too much of their mail would be rejected. Sender call-outs are not likely to put a big dent in the spam volumes.

 

[Tinuva]

I can think of something that will severely screw around with Callback verification, its called grey listing, and enough servers make use of that to have it cause major problems with servers using Callback verification.

If you use Callback verification have fun with it on your mail server, you will probably cause more problems than solutions for yourself.

As for servers accepting mails for address that potentially don't exist, well they exist, and for good reason too. For example our spam-filter servers does this, not because we want to, but we do have clients that we deliver filtered mail to, who doesn't support Callback verification from these spam-filter servers. So what do we do? We accept the mail and archive it. As for the bounce-backs, not really a problem for me causing unwanted traffic, the mail can sit there for 24 hours and then disappear. At least in the event our client does want that email for whatever reason, we archived it and can easily resend it to them once they for example have created that inbox on their end or just to another inbox of their choice.

There comes a point where you have to unfortunately play hard ass, and choose your clients over being the best net-citizen possible. Clients pay, being a good net-citizen not always. As long as you keep your network clean enough to not irritate too many people, you are usually fine. I;ve never seen a mail server being blacklisted for bounce-backs, only if they spit out actual spam non-stop.