No valid SPF record for included Google SPF records

It works here. That leads me to suspect this is a local DNS issue rather than a bug in the program, particularly since (as you suggest) the _spf.google.com SPF record is quite widely used and no one else has complained.

Can you log into the host where you have the policy server installed (ssh or locally, doesn't matter) and see what the output of:

dig txt _spf.google.com

is?

You should get an answer like:

;; ANSWER SECTION:
_spf.google.com. 14 IN TXT "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

If you don't, that confirms a local DNS problem. It may also just be a transient issue where lookups occasionally fail.

Thanks Scott,

I checked all that. DNS seems fine and there are no problems verifying other SPF records. I've restarted nscd and rebooted the server several times. The problem has persisted for several weeks now.

Here's the dig output:

  [13:40:35] andre@proxy-103:~$ dig txt _spf.google.com +short
  "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

One more thing that might be of interest. While most Gogole related SPF checks fail with:

  No valid SPF record for included domain: _spf.google.com

I also get the occasional:

  No valid SPF record for included domain: _netblocks.google.com

which appears to indicate a race condition/timeout somewhere. Is there a DNS lookup timeout for policyd that can be configured?

Andre

Yes, the parameter is Lookup_Time. You can add something like this to the config file:

Lookup_Time = 20

20 seconds is the default.

That's a red herring then as there was never a noticeable delay in DNS lookups according to the logs. I've increased it to 60 seconds but am still getting the

  No valid SPF record for included domain: _spf.google.com

errors.

What platform is this on and what version of pydns (Python DNS) are you using?

Ubuntu 14.04 on x86_64, python3-dns version 3.0.4.

I'm mystified. That's exactly what I'm testing with here.

If you log into the server and manually execute the policy server, you should be able then feed it the same input as postfix is sending it (everything after Read line and between the quotes in your original question). Then a trailing emtpy line.

It would be interesting to see if you get any different results or error messages when you run it by hand.

Same result:

[16:15:22] root@proxy-103:~# sudo -u nobody /usr/bin/python3 /usr/bin/policyd-spf /etc/postfix-policyd-spf-python/policyd-spf.conf
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=ESMTP
client_address=209.85.212.178
client_name=mail-wi0-f178.google.com
reverse_client_name=mail-wi0-f178.google.com
helo_name=mail-wi0-f178.google.com
<email address hidden>
<email address hidden>
recipient_count=0
queue_id=
instance=1a1.54746599.c5b7a.0
size=1999
etrn_domain=
stress=
sasl_method=
sasl_username=
sasl_sender=
ccert_subject=
ccert_issuer=
ccert_fingerprint=
ccert_pubkey_fingerprint=
encryption_protocol=TLSv1
encryption_cipher=ECDHE-RSA-RC4-SHA
encryption_keysize=128

action=prepend Received-SPF: Permerror (SPF Permanent Error: No valid SPF record for included domain: _spf.google.com: include:_spf.google.com) identity=mailfrom; client-ip=209.85.212.178; helo=mail-wi0-f178.google.com; <email address hidden>; <email address hidden>

Also the same result if I run it as root.

Thanks. Let's move down the stack and see what happens if pyspf tries to retrieve the record. Please try:

python3 /usr/lib/python3/dist-packages/spf.py _spf.google.com

You might also install python-spf (if you haven't already) and try:

python /usr/lib/python2.7/dist-packages/spf.py _spf.google.com

If one of those works and the other doesn't, then we'll have a clue.

Thanks for your perseverance, results as follows:

  [16:33:08] root@proxy-103:~# python3 /usr/lib/python3/dist-packages/spf.py _spf.google.com
  None

  [16:33:11] root@proxy-103:~# python /usr/lib/python2.7/dist-packages/spf.py _spf.google.com
  None

OK. Both those absolutely work here.

That at least gets the policy server off the hook.

You don't have any python related stuff in /usr/local do you?

Nothing in /usr/local, but I've done a bit more testing and it turns out that when I switch the name server in /etc/resolv.conf from our local one to 8.8.8.8 all lookups work as expected.

Baffled why this is only a problem when using Python and seemingly only for the lookups in _spf.google.com, but I'll try to dig a bit deeper.

Many thanks for your help,

Andre

Quick update, this turned out to be caused by case-sensitice handling of cached Bind replies by pyspf versions prior to 2.0.11.

http://bmsi.com/pipermail/pymilter/2014-December/000377.html