spf_installer behaviour

Haym

Active Member
May 12, 2017
39
5
8
UK
cPanel Access Level
Root Administrator
Hello,

I'm seeing strange behaviour with the spf_installer script.

Existing account SPF records for primary domain and alias domain:

Code:
v=spf1 mx ip4:1.1.1.1 ip4:2.2.2.2 ~all
v=spf1 +a mx ip4:1.1.1.1 +include:spf.domain.net -all
Then I run:

/usr/local/cpanel/bin/spf_installer username 'include:spf.domain.net' 0 1 1

and both of the above SPF entries are replaced with the following:

Code:
v=spf1 +a +mx +ip4:1.1.1.1 +include:spf.domain.net +include:spf.domain.net -all
v=spf1 +a +mx +ip4:1.1.1.1 +include:spf.domain.net +include:spf.domain.net -all
Am I using it wrong? The preserve option doesn't appear to work at all and the script is adding the new entry multiple times. It seems very buggy!

Thanks
 

rpvw

Well-Known Member
Jul 18, 2013
1,100
472
113
UK
cPanel Access Level
Root Administrator
I would guess your problems stem from the use of the final '1' (the preserve flag) in your command which is appending to the the original SPF value as opposed to overwriting (final '0') it.

The errors that you have with the alias domain seem to stem from the the SPF records being treated as having the same value. I can see no way of defining distinct SPF for a domain alias as opposed to a parent domain, so manual editing might be required after running the script if necessary.

Full instructions from The spf_installer Script - Documentation - cPanel Documentation
 
Last edited:

Haym

Active Member
May 12, 2017
39
5
8
UK
cPanel Access Level
Root Administrator
Thanks for your response. I want to append to the original SPF value though.

I want to append "include:spf.domain.net" to every domain's SPF record on the server without altering any other portion of the existing record, ideally where that portion doesn't already exist but I guess it's fine if it's appended twice.

Is this possible?

Right now, the first record is being overwritten with a completely different record, unrelated to the original record and not even the server's default SPF record (+a added, ip4:2.2.2.2 removed, -all changed to ~all, the policy being appended twice for some reason). If I set the overwrite option to "0", the record doesn't get touched at all.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463
I want to append "include:spf.domain.net" to every domain's SPF record on the server without altering any other portion of the existing record, ideally where that portion doesn't already exist but I guess it's fine if it's appended twice.
Hello @Haym,

You are using "1" for the "overwrite" variable, which will replace the user's current SPF records. You should use "0" for this value if you want to append an entry to the existing SPF record. For example:

Initial SPF record:

Code:
cptest01.tld. IN TXT "v=spf1 +a +mx +ip4:10.1.1.1 ~all"
Run the following command:

Code:
/usr/local/cpanel/bin/spf_installer cptest01 "+include:123.tld" 0 0 1
New SPF record:

Code:
cptest01.tld. IN TXT "v=spf1 +a +mx +ip4:10.1.1.1 +include:123.tld ~all"
Thank you.
 

rpvw

Well-Known Member
Jul 18, 2013
1,100
472
113
UK
cPanel Access Level
Root Administrator
Sorry to query this Michael, but the docs page states
Whether to add the new set of keys to the existing records.

  • A value of 1 indicates that you wish to append the policy option's value to the user's current SPF records.
  • A value of 0 indicates that you wish to replace the user's current SPF record with the policy option's value.
Are you suggesting this is actually the other way round ?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463
Hi @rpvw,

I'm seeing the behavior described in the document when testing. When the "preserve" variable is set to "1", it appends the policy option's variable to the user's current SPF records. If the "preserve" variable is set to "0", it replaces the entry instead of appending it.

Thank you.
 

Haym

Active Member
May 12, 2017
39
5
8
UK
cPanel Access Level
Root Administrator
Hello @Haym,

You are using "1" for the "overwrite" variable, which will replace the user's current SPF records. You should use "0" for this value if you want to append an entry to the existing SPF record. For example:

Initial SPF record:

Code:
cptest01.tld. IN TXT "v=spf1 +a +mx +ip4:10.1.1.1 ~all"
Run the following command:

Code:
/usr/local/cpanel/bin/spf_installer cptest01 "+include:123.tld" 0 0 1
New SPF record:

Code:
cptest01.tld. IN TXT "v=spf1 +a +mx +ip4:10.1.1.1 +include:123.tld ~all"
Thank you.
Hello,

This doesn't work.

Command:
Code:
/usr/local/cpanel/bin/spf_installer demouser 'include:spf.domain.net' 0 0 1
Alias record before running command:

Code:
v=spf1 mx ip4:1.1.1.1 ip4:2.2.2.2 ~all
Primary domain record before running command:

Code:
v=spf1 +a mx ip4:1.1.1.1 +include:spf.domain.net -all
Alias record unchanged after running command:

Code:
v=spf1 mx ip4:1.1.1.1 ip4:2.2.2.2 ~all
Primary domain record has the "include" part unnecessary appended after running command:

Code:
v=spf1 +a +mx +ip4:1.1.1.1 +include:spf.domain.net +include:spf.domain.net -all
For clarity, I also tried running the command using '+include:spf.domain.net' but this didn't work either.

Anyway, I'm 99% of the way to resolving the issue using the API, just running into an issue which I'll open another thread to discuss.

Thanks
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463
Hello @Haym,

Are you using a custom zone template with an SPF txt entry on this system? If so, could you verify if you notice the same behavior without the custom zone template enabled? Also, what version of cPanel is installed on this server?

Primary domain record has the "include" part unnecessary appended after running command:
There's no condition to prevent appending a new policy rule if the same rule already exists. You'd have to enable the "overwrite" and "complete" variables while entering the full SPF policy that you want to use in the zone to prevent that from happening.

Thank you.
 

Haym

Active Member
May 12, 2017
39
5
8
UK
cPanel Access Level
Root Administrator
Hello @Haym,

Are you using a custom zone template with an SPF txt entry on this system? If so, could you verify if you notice the same behavior without the custom zone template enabled? Also, what version of cPanel is installed on this server?
I am using a custom template with the SPF entry and it's cPanel v64.

I am unable to test with the custom zone file disabled at this time unfortunately but will report back to this thread if I get a chance so that other's experiencing the problem can see if this was the cause.

In the mean time, I've written a quick script to add the policy I want for any SPF records which don't have the policy already.

Thanks!
 
  • Like
Reactions: cPanelMichael

Haym

Active Member
May 12, 2017
39
5
8
UK
cPanel Access Level
Root Administrator
So as mentioned, I've written a quick script to inject the required policy to all SPF records for each zone on our servers. I'm sharing it here for anyone else who needs to append a new SPF record across their server and has trouble using cPanel's spf_installer script.

It's PHP (no comments please!) and quickly hacked together for our own use case but should do the job for appending any policy. The script simply checks over each zone file for SPF records and if the required policy doesn't exist, adds it. The changes are done using the WHM API rather than editing any zone files directly, so it should be safe for the future and with the different DNS server options.

Of course check over the script yourself and run in a test environment etc first. I don't take any responsibility if the script messes up zone files on your system because your use case has been overlooked!

PHP:
<?php

// Should theoretically work with any type of policy, e.g. ip4:1.2.3.4
define('POLICY_REQUIRED', 'include:spf.yourdomain.net');

// This script loops through every zone file on the server and appends the POLICY_REQUIRED to each SPF
// record which doesn't already contain the policy. Just set the POLICY_REQUIRED and run.

// Fetch the zones present on this server.
exec('whmapi1 listzones --output=jsonpretty', $zones);

$zones = implode(PHP_EOL, $zones);
$zones = json_decode($zones, true);
$zones = array_column($zones['data']['zone'], 'domain');

// Loop over the zones to process.
foreach ($zones as $domain) {
    $zone = null;
    exec("whmapi1 dumpzone domain={$domain} --output=jsonpretty", $zone);

    $zone = implode(PHP_EOL, $zone);
    $zone = json_decode($zone, true);
    $zone = $zone['data']['zone'][0]['record'];
   
    // Loop over the zone's lines to check for an SPF record.
    foreach ($zone as $record) {
        if ($record['type'] == 'TXT' && strpos($record['txtdata'], 'v=spf1 ') === 0) {
            echo "Found SPF for {$domain} on Line {$record['Line']} ({$record['name']}):" . PHP_EOL;
            echo '  - Existing:' . PHP_EOL;
            echo '    ' . $record['txtdata'] . PHP_EOL;

            // Skip if the required policy already exists.
            if (strpos($record['txtdata'], ' ' . POLICY_REQUIRED. ' ')) {
                echo '  - The required policy exists, skipping.' . PHP_EOL . PHP_EOL;
                continue;
            }

            // Inject the required policy.
            $updated_record = preg_replace('/(.+) ([\?~-]{1}all)/', '$1 ' . POLICY_REQUIRED. ' $2', $record['txtdata']);
            echo '  - Applying policy:' . PHP_EOL;
            echo '    ' . $updated_record . PHP_EOL . PHP_EOL;

            // The record must be urlencoded first,
            $updated_record = urlencode($updated_record);

            // Edit the zone file.
            $command = "whmapi1 editzonerecord domain='{$domain}' line={$record['Line']} name='{$record['name']}' class={$record['class']} ttl={$record['ttl']} type={$record['type']} txtdata='{$updated_record}' unencoded=1";
            exec($command);

            // Not necessary but nice to leave a breather for monitoring!
            sleep(1);
        }
    }
}
 

Attachments

  • Like
Reactions: cPanelMichael

Shawn Paul

Member
Apr 17, 2017
7
0
1
Nairobi
cPanel Access Level
Root Administrator
I would like to change SPF records for a select number of domains saved at '/etc/antispamdomains'. Any way of doing this? The domains are in hundreds so editing individual domains in WHM is not feasible.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463
I would like to change SPF records for a select number of domains saved at '/etc/antispamdomains'. Any way of doing this? The domains are in hundreds so editing individual domains in WHM is not feasible.
There's no existing custom command for this action, as the SPF installer script is used on a per-account basis as opposed to a per-domain basis. Are you able to populate a file that includes a list of account usernames as opposed to domain names?

Thank you.
 

Shawn Paul

Member
Apr 17, 2017
7
0
1
Nairobi
cPanel Access Level
Root Administrator
Thanks for the reply.
I would still achieve my objective if I got a command or script which changes spf for selected cpanel users only, with the respective usernames saved in a directory called '/etc/antispamdomainusers' or something of the sort. Any guide on that? I know the challenge will be the addon and parked domains which i don't wish to include. I only want to change for main account domains.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463