Postfix: Difference between revisions

From Wildsong
Jump to navigationJump to search
← Older edit
VisualWikitext
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,549 edits
mNo edit summary
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,549 edits
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
Postfix is my preferred SMTP server. Dovecot is my preferred IMAP server.
Postfix is my preferred SMTP server. Dovecot is my preferred IMAP server.


2023-02-19 I copied content here from the [[Leaving Google]] page. I am rewriting . updating the old content here.  
2023-11-10 I have been running my own mail server on Tektonic for months now. I decided to dump the w6gkd.radio domain, I have not used it. It's just extra complexity for me. I am stuck with [email protected] forevermore I guess.
 
2023-02-19 I copied content here from the [[Leaving Google]] page.  


== Quick tips ==
== Quick tips ==
Line 9: Line 11:
mailq
mailq


== Initial set up ==
== Monitoring and graphing ==
I want to know how my server is doing, so I am going to try Prometheus.
 
https://gitlab.com/anarcat/grafana-dashboards
 
https://grafana.com/grafana/dashboards/10013-postfix/
 
== SMTP server: Postfix ==


=== Install Postfix ===
=== Install Postfix ===
Starting with a VPS at Tektonic that runs a basic Debian image. Out goes Exim4, in with Postfix. I tried putting Postfix in a Docker and failed. Maybe later.
My VPS at Tektonic runs a basic Debian image. Out goes Exim4, in with Postfix. I tried putting Postfix in a Docker and failed. Maybe later. Perhaps just not worth the effort? Still not using email in Docker.
 
Bellman will get only the postfix package since all it does is forward mail to Porkbun.
  apt remove exim4-base exim4-config exim4-daemon-light  
  apt remove exim4-base exim4-config exim4-daemon-light  
  apt install postfix postgrey clamav spamassassin
  apt install postfix<code>libsasl2-modules</code>
You have to configure in /etc/postfix especially main.cf before starting it.
 
<code>apt install postgrey clamav spamassassin</code>
You have to configure in /etc/postfix especially main.cf before starting it. Skip ahead to the config section for a smarthost like Bellman.


=== Set up DNS ===
=== Set up DNS ===
You need to make sure that the SPF, DKIM, DMARC and MX records are set up correctly in DNS (Cloudflare). Read [https://support.google.com/a/answer/9948472?hl=en&visit_id=638105396125080766-3550268350&p=show_original&rd=1 this Google doc] to learn more.
Set up the reverse (PTR) record by sending a message to support at Tektonic. You need to make sure that the SPF, DKIM, DMARC and MX records are set up correctly in DNS (Cloudflare). Read [https://support.google.com/a/answer/9948472?hl=en&visit_id=638105396125080766-3550268350&p=show_original&rd=1 this Google doc] to learn more.
======MX: Set DNS records ======
======MX: Set DNS records ======
The MX record(s) tell a mail server where to direct mail, for example, I want wildsong.biz mail to be handled by my VPS so I set it to tell mail senders that mail for wildsong.biz should be sent to w6gkd.w6gkd.radio. Note there is also a reverse entry that has to be set up by the service provider for the VPS (Tektonic.net in my case) so that the sender can check to make sure the IP address points to the same place as the DNS name.
The MX record(s) tell a mail server where to direct mail, for example, I want wildsong.biz mail to be handled by my VPS so I set it to tell mail senders that mail for wildsong.biz should be sent to the current server.  
 
For Google mail there were about 6 MX records for wildsong.biz and they pointed at various Google servers. Now there is just one and it points at w6gkd.wildsong.biz.


Currently there are about 6 MX records for wildsong.biz and they all point at various Google servers. When I am done there will be just one and it will point at w6gkd.w6gkd.radio.
======SPF======
======SPF======
Should be a TXT record for the domain (e.g. wildsong.biz) set to v=spf1 ip4:108.161.129.155 ip6:fe80::216:3eff:fea2:8358 -all
 
Should be a TXT record for the domain (e.g. wildsong.biz) set to  
 
v=spf1 ip4:108.161.129.155 include:w6gkd.wildsong.biz include:_spf.google.com -all


Google also made me add another TXT similar to "google-site-verification=-Y...."
Google also made me add another TXT similar to "google-site-verification=-Y...."
2024-4-20 Changed to using Porkbun so now it's
v=spf1 mx include:_spf.porkbun.com ~all
======DMARC======
======DMARC======
Should be a TXT record for "_dmarc" similar to this


v=DMARC1; p=quarantine; rua=mailto:[email protected]
Refer to https://mxtoolbox.com/dmarc/details/what-is-a-dmarc-record
It should be a TXT record for "_dmarc" similar to this (set MYEMAILADDRESS)
 
v=DMARC1; p=quarantine; rua=mailto:MYEMAILADDRESS;ruf=mailto:MYEMAILADDRESS;fo=0:1:s
 
rua = address that will receive aggregate reports
ruf = address that will receive forensic reports
fo = forensic reporting 0:1:s means three different reporting levels
======DKIM======
======DKIM======
How does DKIM work? https://mailtrap.io/blog/dkim/
How does DKIM work? https://mailtrap.io/blog/dkim/


OpenDKIM runs as a service on my VPS. '''It listens on port 12301.'''There is a config file, /etc/opendkim.conf.
OpenDKIM runs as a service on my VPS. '''It listens on port 12301.'''There is a config file, /etc/opendkim.conf.


You can check /var/log/mail.log for messages to see it's working. I sent a message, and Google called back to make sure I am legit.<pre>
You can check /var/log/mail.log for messages to see it's working. I sent a message, and Google called back to make sure I am legit.
 
<pre>
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: mail-qk1-f201.google.com [209.85.222.201] not internal
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: mail-qk1-f201.google.com [209.85.222.201] not internal
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: not authenticated
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: not authenticated
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: DKIM verification successful
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: DKIM verification successful
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: s=20210112 d=google.com a=rsa-sha256 SSL
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: s=20210112 d=google.com a=rsa-sha256 SSL
</pre>
My DKIM dns record looked like this for Tektonic
Type: TXT
Name: mail._domainkey
Content: v=DKIM1; h=sha256; k=rsa;  p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwbhuRbhD7P8TNStPUA6cpcTsdiB/jNLZaQJ8nfX1cmpIyGq5mWzggfZKhqRtNv5P7rmAqcQxztSapfiFLyPqN6VQS1HetO+dlPVdpjznqxjwQyfumxPmMFWmR68Y+KJ0rpvh0zR2MXVU3LYfXAjIG9IpB9M+YVdyCksa2f/p8bVho5t7Z51XyvaC8s4kHPBd4QZjcBNaXzxmRq3DfxiGRbYu0YRhzBRbh+fIqs16FLnxKzkqmbuTXFPkeex88LorJ1rER6JrnqNXlpgdgJ6a6aG1A4b6L79UtlCi7yFrb8nCmLIntx2a3rFhP80k18I6ekHiGBqVdZmoL4BJE8+XRQIDAQAB
For Porkbun I have Name: default._domainkey.wildsong.biz. and Content of
{| class="wikitable"
|
|v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQClsMBoql1hQKTf0IrU7UMUrf1SUUxPCkaFSuAc0qtH3vqpvIvBsHU2FS+e9BOK9iVkDTptgQYuByqvqeBF/NZln7KnWBUSDciLRftUVfRp88qWXz6e0yQXV/OK+kyiFCfjEQeupuEaFJdU6zoFWoih7tdSeGzxc5zLvU6d73ZwsQIDAQAB
|}


</pre>
====== Install and configure OpenDKIM ======
====== Install and configure OpenDKIM ======
  apt install opendkim opendkim-tools
  apt install opendkim opendkim-tools
Help with Postfix: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy
Help with Postfix: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy


I have to make keys for each domain supported. Currently that's hupi.org, map46.com, wildsong.biz, and w6gkd.radio
I have to make keys for each domain supported. Currently that's '''hupi.org, wildsong.biz'''


Keys go in /etc/opendkim/keys/ and there needs to be an entry for each domain in /etc/opendkim/KeyTable and SigningTable
Keys go in /etc/opendkim/keys/ and there needs to be an entry for each domain in /etc/opendkim/KeyTable and SigningTable
Line 62: Line 109:


Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".
Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".
===Testing Postfix===
To send mail on the host, I want the address to have the domain not the hostname,
date | mail bwilson
should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]


This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf
I installed postfixadmin for web access and administration.  
  address {
 
    email-domain w6gkd.radio;
dbconfig-common: writing config to /etc/dbconfig-common/postfixadmin.conf
  }
   
#Can I send from w6gkd.radio?
Creating config file /etc/dbconfig-common/postfixadmin.conf with new version
#Can I send to [/cdn-cgi/l/email-protection [email protected]]?
   
# Are the letsencrypt keys working?
Creating config file /etc/postfixadmin/dbconfig.inc.php with new version
/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh
granting access to database postfixadmin for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
verifying access for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
 
The user that owns the top level mail folders is '''vmailbox'''. There are UID's assigned own the individual folders, starting at 20001. See the 'users' table below.
Actual mail folders are here, organized by virtual host domain name:
 
/var/mail/vhosts/''domainname''
 
=== Postfix config files ===
 
This is a config for Bellman as a smarthost with Porkbun, also called a "satellite system" during the apt install step. If I actually went back to running a full SMTP host I'd need more here.
 
This config also supports virtual mailboxes via postgresql, I don't do that on Bellman.
 
The critical thing is setting a username and password to connect to porkbun on port 587.


See /etc/letsencrypt/live to see what is set up
==== main.cf ====
===Additional Postfix configuration: Filters===
2023-02 I will be working on these soon but I did not let them impede leaving Google.
* Postgrey -- https://postgrey.schweikert.ch/
* Spamassassin -- https://spamassassin.apache.org/
*ClamAV -- Antivirus / malware -- https://www.clamav.net/
===IMAP - Dovecot===
I made a valiant effort at installing Dovecot into a Docker container. I failed. It's installed in the host.


I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS.
Smart host through Porkbun
== Testing dovecot ==
<pre>
# See /usr/share/postfix/main.cf.dist for a commented, more complete version


Dovecot is the IMAP server so it has to work alongside Postfix.
http://wiki2.dovecot.org/TestInstallation


== Postfix configuration ==
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname


On my personal servers, most email is simply forwarded over to mailboxes handled by Google for wildsong.biz.
smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
Only special purpose or hosted domain email stays on the server.
biff = no


The hosted domain mailboxes will be configured in mysql. Initially I set them up from
# appending .domain is the MUA's job.
command line just using mysql!. I like working with mysql and phpmysqladmin because
append_dot_mydomain = no
they are so straightforward.


Initially I loosely followed instructions found here:
# Uncomment the next line to generate "delayed mail" warnings
http://wiki2.dovecot.org/HowTo/DovecotPostgresql
delay_warning_time = 4h


On my Debian based server, I installed everything from packages, did not have to compile anything.
readme_directory = no


sudo apt-get install dovecot-imapd dovecot-postfix postfix-mysql libsasl2-2 libsasl2-modules squirrelmail
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# This command will confirm mysql support is available (and lots of other things)
# fresh installs.
postconf -m
compatibility_level = 3.6
# This command will confirm dovecot is supported
postconf -a
# This command will confirm dovecot has mysql support built
dovecot --build-options


I installed postfixadmin for web access and administration.


dbconfig-common: writing config to /etc/dbconfig-common/postfixadmin.conf
Creating config file /etc/dbconfig-common/postfixadmin.conf with new version
Creating config file /etc/postfixadmin/dbconfig.inc.php with new version
granting access to database postfixadmin for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
verifying access for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.


# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key


Squirrelmail allows users use email via web.
smtp_tls_CApath=/etc/ssl/certs
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = bellman.wildsong.biz
mydomain = wildsong.biz
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = localhost.local localhost
relayhost = [smtp.porkbun.com]:587
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = loopback-only
inet_protocols = all


# Porkbun part
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt


The user that owns the top level mail folders is '''vmailbox'''.  
# Rewrite EVERYTHING to appear to come from brian@wildsong.biz
There are UID's assigned own the individual folders, starting at 20001. See the 'users' table below.
sender_canonical_classes = envelope_sender, header_sender
Actual mail folders are here, organized by virtual host domain name:
sender_canonical_maps = regexp:/etc/postfix/sender_canonical_maps
smtp_header_checks = regexp:/etc/postfix/header_check
</pre>


/var/mail/vhosts/''domainname''
This requires three additional files, sasl_passwd, sender_canonical_maps and header_check


== Postfix config files ==
sasl_passwd
<pre>
[smtp.porkbun.com]:587  SECRET_USERNAME_HERE:SECRET_PASSWORD_HERE
</pre>


This is a config that uses Google aka gmail as a smarthost for outbound mail.
sender_canonical_maps
This config also supports virtual mailboxes via postgresql.
<pre>
/.+/    brian@wildsong.biz
</pre>


=== main.cf ===
header_check
<pre>
/From:.*/ REPLACE From: [email protected]
</pre>


Full email support for a custom domain
<pre>
<pre>
smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
Line 244: Line 317:
</pre>
</pre>


=== aliases.cf ===
 
<code>echo > /etc/postfix/sasl_passwd "[smtp.porkbun.com]:587 apikey:yourSendGridApiKey"</code>
 
<code>chmod 600 /etc/postfix/sasl_passwd</code>
 
<code>postmap /etc/postfix/sasl_passwd</code>
 
systemctl restart postfix
 
==== aliases.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 251: Line 333:
  query=SELECT forw_addr FROM aliases WHERE alias='%s'
  query=SELECT forw_addr FROM aliases WHERE alias='%s'


=== transport.cf ===
==== transport.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 258: Line 340:
  query=SELECT transport FROM transport WHERE domain='%s'
  query=SELECT transport FROM transport WHERE domain='%s'


=== mailbox.cf ===
==== mailbox.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 265: Line 347:
  query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'
  query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'


=== uid.cf ===
==== uid.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 272: Line 354:
  query=SELECT uid FROM users WHERE userid='%s'
  query=SELECT uid FROM users WHERE userid='%s'


=== gid.cf ===
==== gid.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 279: Line 361:
  query=SELECT gid FROM users WHERE userid='%s'
  query=SELECT gid FROM users WHERE userid='%s'


=== virtual.cf ===
==== virtual.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 286: Line 368:
  query=SELECT userid FROM postfix_virtual WHERE address='%s'
  query=SELECT userid FROM postfix_virtual WHERE address='%s'


=== virtual-domains.cf ===
==== virtual-domains.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 292: Line 374:
  hosts=localhost
  hosts=localhost
  query=SELECT domain FROM virtual_domains WHERE domain='%s'
  query=SELECT domain FROM virtual_domains WHERE domain='%s'
===Testing Postfix===
To send mail on the host, I want the address to have the domain not the hostname,
date | mail bwilson
should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]


== PostgreSQL ==
This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf
address {
    email-domain w6gkd.radio;
}
#Can I send from w6gkd.radio?
#Can I send to [/cdn-cgi/l/email-protection [email protected]]?
# Are the letsencrypt keys working?
/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh
 
See /etc/letsencrypt/live to see what is set up
==Additional Postfix configuration: Filters==
2023-09 Time to crank up the spam filters!
What's this mean? More config needed.


For a while I was looking into use PostgreSQL as the back end. I ended up going with MySQL,
config: registryboundaries: no tlds defined, need to run sa-update
but left these notes here just in case.
Timeout::_run: check: no loaded plugin implements 'check_main': cannot scan!
Check that the necessary '.pre' files are in the config directory.
At a minimum, v320.pre loads the Check plugin which is required.


=== Database for postfix ===
2023-08-15 Turned off Postgrey; it works but makes new account sign up and password recovery things painful. I might try again later.


In addition to adding the tables don't forget to set up pg_hba.conf entry if needed
2023-08-01 I enabled a DNSBL block list because it was so easy. See https://docs.iredmail.org/enable.dnsbl.html


$ createdb -U postgres mail
2023-07-31 Postgrey operational-- https://postgrey.schweikert.ch/ -- works by delaying connections.
  $ psql -U postgres mail
   
=== Spam filters ===


=== Tables ===
# Run amavisd, it will launch spamassassin (no separate daemon needed)
# Integrate it with Postfix
# It runs the other filters.


The first two (aliases and transport) are generic, the rest are specific to virtual hosts support.
This page looks reasonable: https://help.ubuntu.com/community/PostfixAmavisNew


query = SELECT forw_addr FROM aliases WHERE alias='%s';
The Debian packages for Amavis and SpamAssassin were missing many config files so I gave up and installed from sources.
 
Amavisd-new was installed from a tarball.
 
Spamassassin was installed via CPAN, I downloaded the tarball and followed the instructions in INSTALL. I had to "apt-get remove spamassassin pyzor razor" first. This takes a long time as it builds the dependency list and you sit there like a dope clicking YES from time to time. CPAN is actually amazing. I had forgotten it exists.


<pre>
<pre>
CREATE TABLE aliases (
perl -MCPAN -e shell  # as root
  alias VARCHAR(255) NOT NULL UNIQUE,
o conf prerequisites_policy ask
  forw_addr VARCHAR(255) NOT NULL DEFAULT('root'),
install YAML
  comment TEXT,
install Mail::SpamAssassin
  PRIMARY KEY (alias)
quit
);
</pre>
-- Standard aliases are still in the /etc/aliases file
 
INSERT INTO aliases (alias, forw_addr, comment) VALUES ('root', 'bwilson', 'Person who should get root email');
When it asked I set the address to be [email protected]
INSERT INTO aliases (alias, forw_addr, comment) VALUES ('bwilson', 'Brian.Wilson@Alseageo.Net', '');
 
# Now a group in our google set up
This installs the versions of perl modules that it wants.
#INSERT INTO aliases (alias, forw_addr, comment) VALUES ('early-warning', 'bwilson', 'Part of Crow monitoring system');
 
Spamassassin -- https://spamassassin.apache.org/


CREATE TABLE transport (
ClamAV -- Antivirus / malware -- https://www.clamav.net/
  domain VARCHAR(128) NOT NULL,
  transport VARCHAR(128) NOT NULL,
  PRIMARY KEY (domain)
);
INSERT INTO transport (domain, transport) values('gmail.com', 'smtp:[smtp.gmail.com]');


-- Create a list of all the domains for which we are the final destination
=== Postgrey ===
CREATE TABLE virtual_domains (
I installed it from "apt". To enable it I created postfix/main.cf.POSTGREY and main.cf.NOPOSTGREY. I changed master.cf to remove the smtpd_recipient_restrictions and moved them to the main.cf files. I used files in /usr/share/postgrey to create the files in /etc/postgrey to give it whitelists for common servers.
  domain VARCHAR(255) NOT NULL,
  PRIMARY KEY (domain)
);
INSERT INTO virtual_domains (domain) VALUES ('dispatch.incidentview.com');


-- This is completely separate from /etc/passwd and you use different id ranges for each supported domain.
In postgrey its possible to whitelist senders as well as recipients. All that needs doing in order to whitelist a host is to add its fully qualified domain name or its ip address to the /etc/postfix/postgrey_whitelist_clients.local file. eg:
CREATE TABLE users (
  192.168.1.10
  userid VARCHAR(128) NOT NULL, -- note this is text
  mydesktop.office.mydomain.com
  password VARCHAR(128),
  realname VARCHAR(128),
  uid INTEGER NOT NULL,
  gid INTEGER NOT NULL,
  domain VARCHAR(255) REFERENCES virtual_domains(domain),
  home VARCHAR(128),  -- maildir folder where mail messages will be written
  mail VARCHAR(255),  -- alias for resending?
  PRIMARY KEY (userid)
);
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('atomiczombie'), 20001, 20000, 'dispatch.incidentview.com', 'bwilson/');
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('landoflakes'), 20002, 20000, 'dispatch.incidentview.com', 'lebanon/');
INSERT INTO users (userid, password, uid, gid, domain, home) values('[email protected].com', md5('bigriver'), 20003, 20000, 'dispatch.incidentview.com', 'corvallis/');
INSERT INTO users (userid, password, uid, gid, domain, home) values('spokanefd9@dispatch.incidentview.com', md5('clocktower'), 20004, 20000, 'dispatch.incidentview.com', 'spokanefd9/');


-- Maps a full address to another alias, I use this for administrative accounts like postmaster
== IMAP server: Dovecot ==
CREATE TABLE virtual (
I made an effort at installing Dovecot into a Docker container. I gave up. It's installed in the host.  
  address VARCHAR(255) NOT NULL, -- full address including domain, like 'postmaster@mydomain.com'
  userid VARCHAR(255) NOT NULL,  -- this is text, not a foreign key so it can be in aliases database
  PRIMARY KEY (address)
);
INSERT INTO virtual (address, userid) VALUES('[email protected].com', 'postmaster');


-- Views that postfix uses, replaces /etc/postfix/vmailbox,
I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS. Then I could use a real database for the backend too? But then I'd need to use LMTP and each delivery of a message would require a connection over the Internet so, eh.
-- which maps a virtual user to a folder in /var/spool/mail/
CREATE VIEW postfix_mailboxes as
  SELECT userid, domain||'/'||home as mailbox from users
  UNION all
  SELECT domain AS userid, 'dummy' AS mailbox FROM transport;
--
-- View that replaces /etc/postfix/virtual,
--  which maps a virtual email address to a virtual user name or local host in /etc/aliases
CREATE VIEW postfix_virtual AS
  SELECT userid, userid AS address FROM users
  UNION all
  SELECT userid, address FROM virtual;
</pre>


=== Roles ===
=== Install Dovecot ===
Currently I have installed Dovecot from apt. I am only using files (well, postfix maps) for auth and mail delivery. No fancy SQL mailboxes right now, it's only me here, that would be overkill.


Set some real passwords when you run these!
apt install dovecot-core dovecot-imapd dovecot-sieve


-- postfix and dovecot only need read access (mail messages are not stored in postgresql)
Configure /etc/dovecot/conf.d files to support sieve.
CREATE USER mailreader PASSWORD 'secret';
GRANT SELECT ON aliases, transport, users, virtual, postfix_mailboxes, postfix_virtual, virtual_domains TO mailreader;
-- web ui will need write access
CREATE USER mailwriter password 'secret';
GRANT SELECT,INSERT,UPDATE,DELETE ON aliases, transport, users, virtual, postfix_mailboxes, postfix_virtual, virtual_domains  TO mailwriter;


=== Test Dovecot ===
Dovecot is the IMAP server so it has to work alongside Postfix.
http://wiki2.dovecot.org/TestInstallation
[[Category: System Administration]]
[[Category: System Administration]]


== Resources ==
== Resources ==
[https://acm.percipio.com/books/b8d57580-f219-11e6-b0e2-0242c0a80804 The Book of Postfix] at Percipio
[https://acm.percipio.com/books/b8d57580-f219-11e6-b0e2-0242c0a80804 The Book of Postfix] at Percipio

Latest revision as of 19:38, 24 November 2024

Postfix is my preferred SMTP server. Dovecot is my preferred IMAP server.

2023-11-10 I have been running my own mail server on Tektonic for months now. I decided to dump the w6gkd.radio domain, I have not used it. It's just extra complexity for me. I am stuck with [email protected] forevermore I guess.

2023-02-19 I copied content here from the Leaving Google page.

Quick tips

Checking the postfix queue

mailq

Monitoring and graphing

I want to know how my server is doing, so I am going to try Prometheus.

https://gitlab.com/anarcat/grafana-dashboards

https://grafana.com/grafana/dashboards/10013-postfix/

SMTP server: Postfix

Install Postfix

My VPS at Tektonic runs a basic Debian image. Out goes Exim4, in with Postfix. I tried putting Postfix in a Docker and failed. Maybe later. Perhaps just not worth the effort? Still not using email in Docker.

Bellman will get only the postfix package since all it does is forward mail to Porkbun. 

apt remove exim4-base exim4-config exim4-daemon-light 
apt install postfixlibsasl2-modules

apt install postgrey clamav spamassassin

You have to configure in /etc/postfix especially main.cf before starting it. Skip ahead to the config section for a smarthost like Bellman.

Set up DNS

Set up the reverse (PTR) record by sending a message to support at Tektonic. You need to make sure that the SPF, DKIM, DMARC and MX records are set up correctly in DNS (Cloudflare). Read this Google doc to learn more.

MX: Set DNS records

The MX record(s) tell a mail server where to direct mail, for example, I want wildsong.biz mail to be handled by my VPS so I set it to tell mail senders that mail for wildsong.biz should be sent to the current server.

For Google mail there were about 6 MX records for wildsong.biz and they pointed at various Google servers. Now there is just one and it points at w6gkd.wildsong.biz.

SPF

Should be a TXT record for the domain (e.g. wildsong.biz) set to 

v=spf1 ip4:108.161.129.155 include:w6gkd.wildsong.biz include:_spf.google.com -all

Google also made me add another TXT similar to "google-site-verification=-Y...."

2024-4-20 Changed to using Porkbun so now it's 

v=spf1 mx include:_spf.porkbun.com ~all
DMARC

Refer to https://mxtoolbox.com/dmarc/details/what-is-a-dmarc-record It should be a TXT record for "_dmarc" similar to this (set MYEMAILADDRESS) 

v=DMARC1; p=quarantine; rua=mailto:MYEMAILADDRESS;ruf=mailto:MYEMAILADDRESS;fo=0:1:s

rua = address that will receive aggregate reports
ruf = address that will receive forensic reports
fo = forensic reporting 0:1:s means three different reporting levels
DKIM

How does DKIM work? https://mailtrap.io/blog/dkim/

OpenDKIM runs as a service on my VPS. It listens on port 12301.There is a config file, /etc/opendkim.conf.

You can check /var/log/mail.log for messages to see it's working. I sent a message, and Google called back to make sure I am legit. 

Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: mail-qk1-f201.google.com [209.85.222.201] not internal
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: not authenticated
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: DKIM verification successful
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: s=20210112 d=google.com a=rsa-sha256 SSL

My DKIM dns record looked like this for Tektonic

Type: TXT

Name: mail._domainkey

Content: v=DKIM1; h=sha256; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwbhuRbhD7P8TNStPUA6cpcTsdiB/jNLZaQJ8nfX1cmpIyGq5mWzggfZKhqRtNv5P7rmAqcQxztSapfiFLyPqN6VQS1HetO+dlPVdpjznqxjwQyfumxPmMFWmR68Y+KJ0rpvh0zR2MXVU3LYfXAjIG9IpB9M+YVdyCksa2f/p8bVho5t7Z51XyvaC8s4kHPBd4QZjcBNaXzxmRq3DfxiGRbYu0YRhzBRbh+fIqs16FLnxKzkqmbuTXFPkeex88LorJ1rER6JrnqNXlpgdgJ6a6aG1A4b6L79UtlCi7yFrb8nCmLIntx2a3rFhP80k18I6ekHiGBqVdZmoL4BJE8+XRQIDAQAB

For Porkbun I have Name: default._domainkey.wildsong.biz. and Content of

v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQClsMBoql1hQKTf0IrU7UMUrf1SUUxPCkaFSuAc0qtH3vqpvIvBsHU2FS+e9BOK9iVkDTptgQYuByqvqeBF/NZln7KnWBUSDciLRftUVfRp88qWXz6e0yQXV/OK+kyiFCfjEQeupuEaFJdU6zoFWoih7tdSeGzxc5zLvU6d73ZwsQIDAQAB
Install and configure OpenDKIM
apt install opendkim opendkim-tools

Help with Postfix: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy

I have to make keys for each domain supported. Currently that's hupi.org, wildsong.biz

Keys go in /etc/opendkim/keys/ and there needs to be an entry for each domain in /etc/opendkim/KeyTable and SigningTable

Generate keys for each domain,

cd /etc/opendkim/keys/
mkdir wildsong.biz
cd wildsong.biz
opendkim-genkey -s mail -d wildsong.biz
chown opendkim:opendkim mail.private

The file "mail.txt" contains the text to put in the DNS DKIM record for mail._domainkey

Remember to restart opendkim and postfix.

Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".

I installed postfixadmin for web access and administration. 

dbconfig-common: writing config to /etc/dbconfig-common/postfixadmin.conf

Creating config file /etc/dbconfig-common/postfixadmin.conf with new version 

Creating config file /etc/postfixadmin/dbconfig.inc.php with new version
granting access to database postfixadmin for [/cdn-cgi/l/email-protection [email protected]]: success.
verifying access for [/cdn-cgi/l/email-protection [email protected]]: success.

The user that owns the top level mail folders is vmailbox. There are UID's assigned own the individual folders, starting at 20001. See the 'users' table below. Actual mail folders are here, organized by virtual host domain name: 

/var/mail/vhosts/domainname

Postfix config files

This is a config for Bellman as a smarthost with Porkbun, also called a "satellite system" during the apt install step. If I actually went back to running a full SMTP host I'd need more here.

This config also supports virtual mailboxes via postgresql, I don't do that on Bellman.

The critical thing is setting a username and password to connect to porkbun on port 587.

main.cf

Smart host through Porkbun 

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6



# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = bellman.wildsong.biz
mydomain = wildsong.biz
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = localhost.local localhost
relayhost = [smtp.porkbun.com]:587
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = loopback-only
inet_protocols = all

# Porkbun part
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

# Rewrite EVERYTHING to appear to come from [email protected]
sender_canonical_classes = envelope_sender, header_sender
sender_canonical_maps = regexp:/etc/postfix/sender_canonical_maps
smtp_header_checks = regexp:/etc/postfix/header_check

This requires three additional files, sasl_passwd, sender_canonical_maps and header_check

sasl_passwd 

[smtp.porkbun.com]:587  SECRET_USERNAME_HERE:SECRET_PASSWORD_HERE

sender_canonical_maps 

/.+/    [email protected]

header_check 

/From:.*/ REPLACE From: [email protected]

Full email support for a custom domain 

smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
biff = no

# Uncomment the next line to generate "delayed mail" warnings
delay_warning_time = 4h

queue_directory = /var/spool/postfix

myorigin = alseageo.net

# ---------------------------------------------------------

relayhost = [smtp.gmail.com]:587

disable_dns_lookups = yes

# authentication via SASL
# incoming connections
smtpd_sasl_auth_enable = no
smtpd_sasl_local_domain = $myhostname

# outgoing connections
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

# tls 
smtp_tls_loglevel = 1

#smtp_enforce_tls = yes
smtp_tls_per_site = hash:/etc/postfix/tls_per_site

smtp_tls_enforce_peername = no

smtp_tls_CAfile = /etc/postfix/certs/cakey.pem
smtp_tls_cert_file=/etc/postfix/certs/alseageo.pem
smtp_tls_key_file=/etc/postfix/certs/alseageo.key
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_use_tls = yes

smtpd_tls_CAfile = /etc/postfix/certs/cacert.pem
smtpd_tls_cert_file=/etc/postfix/certs/alseageo.pem
smtpd_tls_key_file=/etc/postfix/certs/alseageo.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes

#-------------------------------------------                                    
# Virtual mail support                                                          
#                                                                               
virtual_mailbox_base = /var/mail/vhosts
virtual_minimum_uid = 20000
#                                                                               
# Old file version                                                              
#transport_maps = hash:/etc/postfix/transport
#virtual_uid_maps = static:20000
#virtual_gid_maps = static:20000
#virtual_mailbox_domains = dispatch.incidentview.com
#virtual_mailbox_maps = hash:/etc/postfix/vmailbox
#virtual_alias_maps = hash:/etc/postfix/virtual
#                                                                               
# New SQL version                                                               
transport_maps = pgsql:/etc/postfix/transport.cf
virtual_uid_maps = pgsql:/etc/postfix/uid.cf
virtual_gid_maps = pgsql:/etc/postfix/gid.cf
virtual_mailbox_domains = pgsql:dispatch.incidentview.com
virtual_mailbox_maps = pgsql:/etc/postfix/mailbox.cf
virtual_maps = pgsql:/etc/postfix/virtual.cf


# ---------------------------------------------------------

command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix

mail_owner = postfix

myhostname = kilchis.alseageo.com
mydomain = alseageo.com

inet_interfaces = all

# This is to allow the /etc/aliases file to have an effect.    
mydestination =
        kilchis.alseageo.com kilchis
        roaring.alseageo.com roaring
        minam.alseageo.com minam
        white.alseageo.com white
        dev.alseageo.com dev
        fall.alseageo.com fall
        localhost localhost.localdomain kilchis.localdomain

mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

mail_spool_directory=/var/spool/mail

mynetworks = 127.0.0.0/8 10.1.10.0/24 10.8.0.0/24
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases


echo > /etc/postfix/sasl_passwd "[smtp.porkbun.com]:587 apikey:yourSendGridApiKey"

chmod 600 /etc/postfix/sasl_passwd

postmap /etc/postfix/sasl_passwd

systemctl restart postfix

aliases.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT forw_addr FROM aliases WHERE alias='%s'

transport.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT transport FROM transport WHERE domain='%s'

mailbox.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'

uid.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT uid FROM users WHERE userid='%s'

gid.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT gid FROM users WHERE userid='%s'

virtual.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT userid FROM postfix_virtual WHERE address='%s'

virtual-domains.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT domain FROM virtual_domains WHERE domain='%s'

Testing Postfix

To send mail on the host, I want the address to have the domain not the hostname, 

date | mail bwilson

should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]

This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf 

address {
   email-domain w6gkd.radio;
}
  1. Can I send from w6gkd.radio?
  2. Can I send to [/cdn-cgi/l/email-protection [email protected]]?
  3. Are the letsencrypt keys working?

/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh

See /etc/letsencrypt/live to see what is set up

Additional Postfix configuration: Filters

2023-09 Time to crank up the spam filters! What's this mean? More config needed. 

config: registryboundaries: no tlds defined, need to run sa-update
Timeout::_run: check: no loaded plugin implements 'check_main': cannot scan!
Check that the necessary '.pre' files are in the config directory.
At a minimum, v320.pre loads the Check plugin which is required.

2023-08-15 Turned off Postgrey; it works but makes new account sign up and password recovery things painful. I might try again later.

2023-08-01 I enabled a DNSBL block list because it was so easy. See https://docs.iredmail.org/enable.dnsbl.html

2023-07-31 Postgrey operational-- https://postgrey.schweikert.ch/ -- works by delaying connections.

Spam filters

  1. Run amavisd, it will launch spamassassin (no separate daemon needed)
  2. Integrate it with Postfix
  3. It runs the other filters.

This page looks reasonable: https://help.ubuntu.com/community/PostfixAmavisNew

The Debian packages for Amavis and SpamAssassin were missing many config files so I gave up and installed from sources.

Amavisd-new was installed from a tarball.

Spamassassin was installed via CPAN, I downloaded the tarball and followed the instructions in INSTALL. I had to "apt-get remove spamassassin pyzor razor" first. This takes a long time as it builds the dependency list and you sit there like a dope clicking YES from time to time. CPAN is actually amazing. I had forgotten it exists. 

perl -MCPAN -e shell  # as root
o conf prerequisites_policy ask
install YAML
install Mail::SpamAssassin
quit

When it asked I set the address to be [email protected]

This installs the versions of perl modules that it wants.

Spamassassin -- https://spamassassin.apache.org/

ClamAV -- Antivirus / malware -- https://www.clamav.net/

Postgrey

I installed it from "apt". To enable it I created postfix/main.cf.POSTGREY and main.cf.NOPOSTGREY. I changed master.cf to remove the smtpd_recipient_restrictions and moved them to the main.cf files. I used files in /usr/share/postgrey to create the files in /etc/postgrey to give it whitelists for common servers.

In postgrey its possible to whitelist senders as well as recipients. All that needs doing in order to whitelist a host is to add its fully qualified domain name or its ip address to the /etc/postfix/postgrey_whitelist_clients.local file. eg: 

192.168.1.10
mydesktop.office.mydomain.com

IMAP server: Dovecot

I made an effort at installing Dovecot into a Docker container. I gave up. It's installed in the host.

I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS. Then I could use a real database for the backend too? But then I'd need to use LMTP and each delivery of a message would require a connection over the Internet so, eh.

Install Dovecot

Currently I have installed Dovecot from apt. I am only using files (well, postfix maps) for auth and mail delivery. No fancy SQL mailboxes right now, it's only me here, that would be overkill.

apt install dovecot-core dovecot-imapd dovecot-sieve

Configure /etc/dovecot/conf.d files to support sieve.

Test Dovecot

Dovecot is the IMAP server so it has to work alongside Postfix. http://wiki2.dovecot.org/TestInstallation

Resources

The Book of Postfix at Percipio

Retrieved from "https://wiki.wildsong.biz//index.php?title=Postfix&oldid=14484"