Exim supported options

[sideways]

Hello,

I am trying to setup haproxy in front of 2 CentOS 7 cPanel servers with SSL termination. I have hit the following problem with the cPanel provided exim package.

It seems like the exim version provided with cPanel is not built with the option enabled for proxy support.

on a CentOS 7 server (no cPanel):
[[email protected] ~]# exim --version
Exim version 4.92.3 #3 built 30-Sep-2019 11:50:17
Copyright (c) University of Cambridge, 1995 - 2018
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2018
Berkeley DB: Berkeley DB 5.3.21: (May 11, 2012)
Support for: crypteq iconv() IPv6 PAM Perl Expand_dlfunc OpenSSL Content_Scanning DANE DKIM DNSSEC Event OCSP PRDR PROXY SOCKS TCP_Fast_Open
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb dsearch ldap ldapdn ldapm nis nis0 nisplus passwd sqlite
Authenticators: cram_md5 cyrus_sasl dovecot gsasl plaintext spa tls
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Malware: f-protd f-prot6d drweb fsecure sophie clamd avast sock cmdline
Fixed never_users: 0
Configure owner: 0:0
Size of off_t: 8
Configuration file is /etc/exim/exim.conf

On a CentOS 7 server with cPanel:
[[email protected] ~]# exim --version
Exim version 4.93 #2 built 23-Dec-2019 18:18:10
Copyright (c) University of Cambridge, 1995 - 2018
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2018
Berkeley DB: Berkeley DB 5.3.21: (May 11, 2012)
Support for: crypteq iconv() IPv6 PAM Perl OpenSSL Content_Scanning DANE DKIM DNSSEC Event I18N OCSP PRDR SPF Experimental_SRS
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb passwd
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir autoreply lmtp pipe smtp
Malware: f-protd f-prot6d drweb fsecure sophie clamd avast sock cmdline
Configure owner: 0:0
Size of off_t: 8
2020-02-13 16:00:54 cwd=/root 2 args: exim --version
Configuration file is /etc/exim.conf

Anyone have any ideas on how I should go about getting around this? I would prefer not breaking updates etc.

I could remove exim* from /etc/yum.conf excludes and try to install it from the provided CentOS 7 repo but I'm not sure if it is going to break stuff and I was looking for a better solution.

Thanks in advance for any input you may have.

 

[cPanelLauren]

I would not make modifications to the exim installation cPanel uses - it could severely break things.

I know this is possible to set up (at least HAProxy is) but this is very out of the scope of items we're able to provide support for you might get some help from the haproxy community here: HAProxy community

Of course, anyone who is familiar with this or has suggestions is always welcome to chime in on this thread.

 

[sideways]

Thanks for the ideas. I will reach out to the HAProxy community to see if Exim can work properly without "PROXY" support enabled.

 

[cPanelLauren]

@sideways

If you can, respond here if they're able to assist you with a solution, I'd be curious to know as would others I'm sure

 

[sideways]

I will definitely follow up with the solution. I may also post the entire setup and configuration after I get it working. Dovecot is configured using the replication plugin on both cpanel servers and seems to be working perfectly so far for IMAP when behind HAProxy with SSL termination. Now I just need to fix the Exim issues.

I'm not sure if it is possible for me to rebuild the cPanel Exim RPM using something like rpmrebuild and add the Exim "PROXY" support myself? If this is possible then I would just have to repeat the rebuild when there is an update. I have never done any RPM rebuilding so I will research and run some tests. If anyone has done something like this with cPanel please share. It can't be harder than using Gentoo lol!

I don't mind doing some manual updates I just want to make sure that everything is going to work well together.

 

[sideways]

I am happy to share I have built exim with PROXY support using the cPanel Exim RPM and it has fixed my issue. For anyone wondering how I did this:

DOCUMENTATION FOR RPM BUILD! - How to Build and Install Custom RPMs | cPanel & WHM Documentation
--
cd

mkdir -p rpmbuild/{BUILD,SPECS,SOURCES,BUILDROOT,RPMS,SRPMS}
/scripts/update_local_rpm_versions --edit target_settings.cpanel-devel installed
/scripts/check_cpanel_rpms --targets cpanel-devel --fix
wget http://httpupdate.cpanel.net/RPM/11.86/src/exim-4.93-1.cp1186.src.rpm
rpm -Uvh exim-4.93-1.cp1186.src.rpm
cd rpmbuild/SPECS/
vi exim.spec

**** FIND THE LINES BELOW AND ADD SUPPORT_PROXY

****** MODIFY FILE *****

  # Put the custom makefile in place.
  mkdir -p Local
  cp src/EDITME Local/Makefile
  ##MODDED LNM_SUPPORT FOR HAPROXY
  echo "SUPPORT_PROXY=yes" >> Local/Makefile

****** MODIFY FILE *****

rpmbuild -bb /root/rpmbuild/SPECS/exim.spec
cd rpmbuild/RPMS/x86_64/
rpm -Uvh exim-4.93-1.cp1186.x86_64.rpm --force
/scripts/update_local_rpm_versions --add srpm_versions.exim 4.93-1.cp1186

##NOT SURE IF THAT IS GOOD BUT MOVING ON ##

WILL HAVE TO CHECK IF UPDATES BLOCK ON EXIM... I WILL NEED TO DOWNLOAD, MODIFY, AND INSTALL EXIM WHEN UPDATES ARE DUE! NO BIG DEAL AT ALL SINCE IT IS THE CPANEL RPM AND ONLY A SMALL MOD.

exim --version | grep PROXY
##SUPPORT IS ADDED!!! WOOT WOOT!

• And to share a bit about the setup. The goal was to keep cPanel for mail (preferably in a cluster also with failover and LB) and DNS cluster but to offload websites and mysql to another cluster as well.

• There is 2 haproxy VMs sitting in front of 2 cpanel servers. One is mail1 the other mail2. I have modified the default dovecot config template. I was using this as a guide. - Replication - Dovecot Wiki

EDIT FILE : /var/cpanel/templates/dovecot2.3/main.local

#CUSTOM DOVECOT REPLICATION MODS
.... below this
# Quota support must be enabled globally for the quota-status
# service to work
haproxy_trusted_networks = xx.xx.xx.xx
haproxy_timeout = 3s
mail_plugins = quota quota_clone zlib notify replication [% IF fts_support %]fts fts_solr[% END %]
mail_debug = yes
service replicator {
  process_min_avail = 1
}
dsync_remote_cmd = ssh -i /root/.ssh/id_rsa %{host} /usr/local/bin/dsync-in-wrapper.sh
plugin {
   mail_replica = remote:[EMAIL][email protected][/EMAIL]
}
service aggregator {
  fifo_listener replication-notify-fifo {
    user = vmail
    mode=0666
  }
  unix_listener replication-notify {
    user = vmail
    mode=0666
  }
}
service replicator {
  unix_listener replicator-doveadm {
    mode = 0666
    user = vmail
  }
}
replication_max_conns = 10
service doveadm {
inet_listener {
    port = 12345
    ssl = no
  }
}
doveadm_port = 12345
doveadm_password = xxxxxxxx
plugin {
  mail_replica = tcp:xxxxx.domain.com:12345 # use port 12345 explicitly
  replication_sync_timeout = 3
}
...... further down....
[%- ELSE %]
#    inet_listener imap {
#      address =
#    }
[%- END %]
  ##REPLICATION MODS
  inet_listener imap_haproxy {
    port = 10143
    haproxy = yes
  }
  inet_listener imaps_haproxy {
    port = 10993
    ssl = yes
    haproxy = yes
  }

• In exim I went to advanced editor and added a custom config line : hosts_proxy = xx.xx.xx.xx

• There is probably some minor details I am leaving out but this should get someone pretty close.

• I still need to build tons of hooks so when there is an domain/account/email created or modified on one server, the changes push to the other server. I am also going to offload webmail to the www/mariadb cluster not included in this post.

• clients domain's dns will be set to:

domain.com MX 0 mail1.serverdomain.com (this points to mail server direct for port 25 submissions with no auth)
domain.com MX 10 mail2.serverdomain.com (this points to mail server direct for port 25 submissions with no auth)

• client's email config:

IMAP/SMTP = mail.serverdomain.com (this points to haproxy floating ip)

Here is the important part of the haproxy config (still to be tweaked/tuned):

frontend SMTP_25
    bind xx.xx.xx.xx:25
    mode tcp
    option tcplog
    default_backend SMTP_25
frontend SMTPS_465
    bind xx.xx.xx.xx:465 name smtpssl ssl crt /etc/haproxy/ssl/
    mode tcp
    option tcplog
    default_backend SMTPS_465
frontend SMTPS_587
    bind xx.xx.xx.xx:587 name smtpssl ssl crt /etc/haproxy/ssl/
    mode tcp
    option tcplog
    default_backend SMTPS_587
frontend ft_imap
    bind xx.xx.xx.xx:143
    mode tcp
    option tcplog
    default_backend bk_imap
frontend ft_imap_ssl
    bind xx.xx.xx.xx:993 name imapssl ssl crt /etc/haproxy/ssl/
    mode tcp
    option tcplog
    default_backend bk_imap
frontend mail_ssl
    bind xx.xx.xx.xx:443,xx.xx.xx.xx:80
    ##LETS ENCRYPT TO LOCAL
    acl letsencrypt-acl path_beg /.well-known/acme-challenge/
    use_backend letsencrypt-backend if letsencrypt-acl


#---------------------------------------------------------------------
# BackEnd round robin as balance algorithm
#---------------------------------------------------------------------

backend SMTP_25
    mode tcp
    server Exc1 xx.xx.xx.xx:25 check port 465 fall 3 rise 2 on-marked-down shutdown-sessions verify none send-proxy-v2
    server Exc2 xx.xx.xx.xx:25 check port 465 fall 3 rise 2 on-marked-down shutdown-sessions verify none send-proxy-v2

backend SMTPS_465
    mode tcp
    server Exc1 xx.xx.xx.xx:25 check port 465 fall 3 rise 2 on-marked-down shutdown-sessions verify none send-proxy-v2
    server Exc2 xx.xx.xx.xx:25 check port 465 fall 3 rise 2 on-marked-down shutdown-sessions verify none send-proxy-v2

backend SMTPS_587
    mode tcp
    server Exc1 xx.xx.xx.xx:25 check port 587 fall 3 rise 2 on-marked-down shutdown-sessions verify none send-proxy-v2
    server Exc2 xx.xx.xx.xx:25 check port 587 fall 3 rise 2 on-marked-down shutdown-sessions verify none

backend app-main
    balance roundrobin                                     #Balance algorithm
    option httplog
    option httpchk HEAD / HTTP/1.1\r\nHost:\ localhost    #Check the server application is up and healty - 200 status code
    server web-server-1 xx.xx.xx.xx:443 check ssl verify none                 #Nginx1
    server web-server-2 xx.xx.xx.xx:443 check ssl verify none                #Nginx2
    server web-server-3 xx.xx.xx.xx:443 check ssl verify none                #Nginx2

backend webmail-main
    balance roundrobin
    option httplog                                    #Balance algorithm
    option httpchk HEAD / HTTP/1.1\r\nHost:\ localhost    #Check the server application is up and healty - 200 status code
    server web-server-1 xx.xx.xx.xx:443 check ssl verify none                #Nginx1
    server web-server-2 xx.xx.xx.xx:443 check ssl verify none                #Nginx2
    server web-server-3 xx.xx.xx.xx:443 check ssl verify none                #Nginx2


listen mysql-cluster xx.xx.xx.xx:3306
    mode tcp
    balance roundrobin
    option mysql-check user haproxycheckuser post-41
    server mysql1 10.11.xx.xx:3306 check
    server mysql2 10.11.xx.xx:3306 check
    server mysql3 10.11.xx.xx:3306 check

backend bk_imap
    mode tcp
    balance leastconn
    stick store-request src
    stick-table type ip size 200k expire 30m
    server s1 xx.xx.xx.xx:10143 send-proxy-v2
    server s2 xx.xx.xx.xx:10143 send-proxy-v2

backend bk_imap_ssl
    mode tcp
    balance leastconn
    stick store-request src
    stick-table type ip size 200k expire 30m
    server mail1 xx.xx.xx.xx:10993 send-proxy-v2 verify none
    server mail2 xx.xx.xx.xx:10993 send-proxy-v2 verify none


backend letsencrypt-backend
    server letsencrypt 127.0.0.1:8888



and keepalive config:

vrrp_script chk_haproxy {
  script "killall -0 haproxy" # check the haproxy process
  interval 2 # every 2 seconds
  weight 2 # add 2 points if OK
}

vrrp_instance sites {
  interface ensxxx # interface to monitor
  state MASTER # MASTER on haproxy1, BACKUP on haproxy2
  virtual_router_id 51
  priority 101 # 101 on haproxy1, 100 on haproxy2
  virtual_ipaddress {
    xx
    xx
    xx
    xx
    xx
  }
  track_script {
    chk_haproxy
  }
}

I will still need to workout some minor details like the hooks, DKIM transfer etc but I am almost there.

Hope this helps someone else and I will come back to post updates if I hit any more issues.

 

[cPanelLauren]

@sideways

This is great! Thank you very much for including this thorough of a walkthrough for users, it's much appreciated.