Postfix: Difference between revisions

From Wildsong
Jump to navigationJump to search
← Older editNewer edit →
VisualWikitext
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,540 edits
mNo edit summary
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,540 edits
mNo edit summary
Line 9: Line 9:
mailq
mailq


== Initial set up ==
== SMTP server: Postfix ==


=== Install Postfix ===
=== Install Postfix ===
Line 62: Line 62:


Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".
Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".
===Testing Postfix===
To send mail on the host, I want the address to have the domain not the hostname,
date | mail bwilson
should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]
This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf
address {
    email-domain w6gkd.radio;
}
#Can I send from w6gkd.radio?
#Can I send to [/cdn-cgi/l/email-protection [email protected]]?
# Are the letsencrypt keys working?
/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh
See /etc/letsencrypt/live to see what is set up
===Additional Postfix configuration: Filters===
2023-02 I will be working on these soon but I did not let them impede leaving Google.
* Postgrey -- https://postgrey.schweikert.ch/
* Spamassassin -- https://spamassassin.apache.org/
*ClamAV -- Antivirus / malware -- https://www.clamav.net/
===IMAP - Dovecot===
I made a valiant effort at installing Dovecot into a Docker container. I failed. It's installed in the host.
I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS.
== Testing dovecot ==
Dovecot is the IMAP server so it has to work alongside Postfix.
http://wiki2.dovecot.org/TestInstallation
== Postfix configuration ==
On my personal servers, most email is simply forwarded over to mailboxes handled by Google for wildsong.biz.
Only special purpose or hosted domain email stays on the server.
The hosted domain mailboxes will be configured in mysql. Initially I set them up from
command line just using mysql!. I like working with mysql and phpmysqladmin because
they are so straightforward.
Initially I loosely followed instructions found here:
http://wiki2.dovecot.org/HowTo/DovecotPostgresql
On my Debian based server, I installed everything from packages, did not have to compile anything.
sudo apt-get install dovecot-imapd dovecot-postfix postfix-mysql libsasl2-2 libsasl2-modules squirrelmail
# This command will confirm mysql support is available (and lots of other things)
postconf -m
# This command will confirm dovecot is supported
postconf -a
# This command will confirm dovecot has mysql support built
dovecot --build-options


I installed postfixadmin for web access and administration.  
I installed postfixadmin for web access and administration.  
Line 122: Line 72:
  granting access to database postfixadmin for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
  granting access to database postfixadmin for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
  verifying access for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
  verifying access for [/cdn-cgi/l/email-protection <nowiki>[email protected]</nowiki>]: success.
Squirrelmail allows users use email via web.




Line 133: Line 80:
  /var/mail/vhosts/''domainname''
  /var/mail/vhosts/''domainname''


== Postfix config files ==
=== Postfix config files ===
I need to update this. I currently run my own server, not a forwarder.


This is a config that uses Google aka gmail as a smarthost for outbound mail.
This is a config that uses Google aka gmail as a smarthost for outbound mail.
This config also supports virtual mailboxes via postgresql.
This config also supports virtual mailboxes via postgresql.


=== main.cf ===
==== main.cf ====
 
<pre>
<pre>
smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
Line 244: Line 191:
</pre>
</pre>


=== aliases.cf ===
==== aliases.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 251: Line 198:
  query=SELECT forw_addr FROM aliases WHERE alias='%s'
  query=SELECT forw_addr FROM aliases WHERE alias='%s'


=== transport.cf ===
==== transport.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 258: Line 205:
  query=SELECT transport FROM transport WHERE domain='%s'
  query=SELECT transport FROM transport WHERE domain='%s'


=== mailbox.cf ===
==== mailbox.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 265: Line 212:
  query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'
  query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'


=== uid.cf ===
==== uid.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 272: Line 219:
  query=SELECT uid FROM users WHERE userid='%s'
  query=SELECT uid FROM users WHERE userid='%s'


=== gid.cf ===
==== gid.cf ====
  user=mailreader
  user=mailreader
  password=<secret>
  password=<secret>
Line 279: Line 226:
  query=SELECT gid FROM users WHERE userid='%s'
  query=SELECT gid FROM users WHERE userid='%s'


=== virtual.cf ===
==== virtual.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 286: Line 233:
  query=SELECT userid FROM postfix_virtual WHERE address='%s'
  query=SELECT userid FROM postfix_virtual WHERE address='%s'


=== virtual-domains.cf ===
==== virtual-domains.cf ====
  user=mailreader
  user=mailreader
  password=secret
  password=secret
Line 292: Line 239:
  hosts=localhost
  hosts=localhost
  query=SELECT domain FROM virtual_domains WHERE domain='%s'
  query=SELECT domain FROM virtual_domains WHERE domain='%s'
===Testing Postfix===
To send mail on the host, I want the address to have the domain not the hostname,
date | mail bwilson
should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]


== PostgreSQL ==
This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf
address {
    email-domain w6gkd.radio;
}
#Can I send from w6gkd.radio?
#Can I send to [/cdn-cgi/l/email-protection [email protected]]?
# Are the letsencrypt keys working?
/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh


For a while I was looking into use PostgreSQL as the back end. I ended up going with MySQL,
See /etc/letsencrypt/live to see what is set up
but left these notes here just in case.
===Additional Postfix configuration: Filters===
2023-02 I will be working on these soon but I did not let them impede leaving Google.
* Postgrey -- https://postgrey.schweikert.ch/
* Spamassassin -- https://spamassassin.apache.org/
*ClamAV -- Antivirus / malware -- https://www.clamav.net/


=== Database for postfix ===
== IMAP server: Dovecot ==
I made a valiant effort at installing Dovecot into a Docker container. I failed. It's installed in the host.


In addition to adding the tables don't forget to set up pg_hba.conf entry if needed
I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS.


$ createdb -U postgres mail
=== Install Dovecot ===
$ psql -U postgres mail
Currently I have installed Dovecot from apt. Specifically "dovecot-core" and "dovecot-imapd". I am only using files (well, postfix maps) for auth and mail delivery. No fancy SQL mailboxes right now, it's only me here, that would be massive overkill.


=== Tables ===
I am migrating it to a Docker container soon.
 
The first two (aliases and transport) are generic, the rest are specific to virtual hosts support.
 
query = SELECT forw_addr FROM aliases WHERE alias='%s';
 
<pre>
CREATE TABLE aliases (
  alias VARCHAR(255) NOT NULL UNIQUE,
  forw_addr VARCHAR(255) NOT NULL DEFAULT('root'),
  comment TEXT,
  PRIMARY KEY (alias)
);
-- Standard aliases are still in the /etc/aliases file
INSERT INTO aliases (alias, forw_addr, comment) VALUES ('root', 'bwilson', 'Person who should get root email');
INSERT INTO aliases (alias, forw_addr, comment) VALUES ('bwilson', '[email protected]', '');
# Now a group in our google set up
#INSERT INTO aliases (alias, forw_addr, comment) VALUES ('early-warning', 'bwilson', 'Part of Crow monitoring system');
 
CREATE TABLE transport (
  domain VARCHAR(128) NOT NULL,
  transport VARCHAR(128) NOT NULL,
  PRIMARY KEY (domain)
);
INSERT INTO transport (domain, transport) values('gmail.com', 'smtp:[smtp.gmail.com]');
 
-- Create a list of all the domains for which we are the final destination
CREATE TABLE virtual_domains (
  domain VARCHAR(255) NOT NULL,
  PRIMARY KEY (domain)
);
INSERT INTO virtual_domains (domain) VALUES ('dispatch.incidentview.com');
 
-- This is completely separate from /etc/passwd and you use different id ranges for each supported domain.
CREATE TABLE users (
  userid VARCHAR(128) NOT NULL, -- note this is text
  password VARCHAR(128),
  realname VARCHAR(128),
  uid INTEGER NOT NULL,
  gid INTEGER NOT NULL,
  domain VARCHAR(255) REFERENCES virtual_domains(domain),
  home VARCHAR(128),  -- maildir folder where mail messages will be written
  mail VARCHAR(255),  -- alias for resending?
  PRIMARY KEY (userid)
);
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('atomiczombie'), 20001, 20000, 'dispatch.incidentview.com', 'bwilson/');
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('landoflakes'), 20002, 20000, 'dispatch.incidentview.com', 'lebanon/');
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('bigriver'), 20003, 20000, 'dispatch.incidentview.com', 'corvallis/');
INSERT INTO users (userid, password, uid, gid, domain, home)  values('[email protected]', md5('clocktower'), 20004, 20000, 'dispatch.incidentview.com', 'spokanefd9/');
 
-- Maps a full address to another alias, I use this for administrative accounts like postmaster
CREATE TABLE virtual (
  address VARCHAR(255) NOT NULL, -- full address including domain, like '[email protected]'
  userid VARCHAR(255) NOT NULL,  -- this is text, not a foreign key so it can be in aliases database
  PRIMARY KEY (address)
);
INSERT INTO virtual (address, userid) VALUES('[email protected]', 'postmaster');
 
-- Views that postfix uses, replaces /etc/postfix/vmailbox,
-- which maps a virtual user to a folder in /var/spool/mail/
CREATE VIEW postfix_mailboxes as
  SELECT userid, domain||'/'||home as mailbox from users
  UNION all
  SELECT domain AS userid, 'dummy' AS mailbox FROM transport;
--
-- View that replaces /etc/postfix/virtual,
--  which maps a virtual email address to a virtual user name or local host in /etc/aliases
CREATE VIEW postfix_virtual AS
  SELECT userid, userid AS address FROM users
  UNION all
  SELECT userid, address FROM virtual;
</pre>
 
=== Roles ===
 
Set some real passwords when you run these!
 
-- postfix and dovecot only need read access (mail messages are not stored in postgresql)
CREATE USER mailreader PASSWORD 'secret';
GRANT SELECT ON aliases, transport, users, virtual, postfix_mailboxes, postfix_virtual, virtual_domains TO mailreader;
-- web ui will need write access
CREATE USER mailwriter password 'secret';
GRANT SELECT,INSERT,UPDATE,DELETE ON aliases, transport, users, virtual, postfix_mailboxes, postfix_virtual, virtual_domains  TO mailwriter;


=== Test Dovecot ===
Dovecot is the IMAP server so it has to work alongside Postfix.
http://wiki2.dovecot.org/TestInstallation
[[Category: System Administration]]
[[Category: System Administration]]


== Resources ==
== Resources ==
[https://acm.percipio.com/books/b8d57580-f219-11e6-b0e2-0242c0a80804 The Book of Postfix] at Percipio
[https://acm.percipio.com/books/b8d57580-f219-11e6-b0e2-0242c0a80804 The Book of Postfix] at Percipio

Revision as of 22:30, 19 February 2023

Postfix is my preferred SMTP server. Dovecot is my preferred IMAP server.

2023-02-19 I copied content here from the Leaving Google page. I am rewriting . updating the old content here.

Quick tips

Checking the postfix queue

mailq

SMTP server: Postfix

Install Postfix

Starting with a VPS at Tektonic that runs a basic Debian image. Out goes Exim4, in with Postfix. I tried putting Postfix in a Docker and failed. Maybe later. 

apt remove exim4-base exim4-config exim4-daemon-light 
apt install postfix postgrey clamav spamassassin

You have to configure in /etc/postfix especially main.cf before starting it.

Set up DNS

You need to make sure that the SPF, DKIM, DMARC and MX records are set up correctly in DNS (Cloudflare). Read this Google doc to learn more.

MX: Set DNS records

The MX record(s) tell a mail server where to direct mail, for example, I want wildsong.biz mail to be handled by my VPS so I set it to tell mail senders that mail for wildsong.biz should be sent to w6gkd.w6gkd.radio. Note there is also a reverse entry that has to be set up by the service provider for the VPS (Tektonic.net in my case) so that the sender can check to make sure the IP address points to the same place as the DNS name.

Currently there are about 6 MX records for wildsong.biz and they all point at various Google servers. When I am done there will be just one and it will point at w6gkd.w6gkd.radio.

SPF

Should be a TXT record for the domain (e.g. wildsong.biz) set to v=spf1 ip4:108.161.129.155 ip6:fe80::216:3eff:fea2:8358 -all

Google also made me add another TXT similar to "google-site-verification=-Y...."

DMARC

Should be a TXT record for "_dmarc" similar to this

v=DMARC1; p=quarantine; rua=mailto:[email protected]

DKIM

How does DKIM work? https://mailtrap.io/blog/dkim/

OpenDKIM runs as a service on my VPS. It listens on port 12301.There is a config file, /etc/opendkim.conf.

You can check /var/log/mail.log for messages to see it's working. I sent a message, and Google called back to make sure I am legit.

Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: mail-qk1-f201.google.com [209.85.222.201] not internal
Jan 29 03:31:51 w6gkd opendkim[729097]: A8F6485D: not authenticated
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: DKIM verification successful
Jan 29 03:31:52 w6gkd opendkim[729097]: A8F6485D: s=20210112 d=google.com a=rsa-sha256 SSL
Install and configure OpenDKIM
apt install opendkim opendkim-tools

Help with Postfix: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy

I have to make keys for each domain supported. Currently that's hupi.org, map46.com, wildsong.biz, and w6gkd.radio

Keys go in /etc/opendkim/keys/ and there needs to be an entry for each domain in /etc/opendkim/KeyTable and SigningTable

Generate keys for each domain,

cd /etc/opendkim/keys/
mkdir wildsong.biz
cd wildsong.biz
opendkim-genkey -s mail -d wildsong.biz
chown opendkim:opendkim mail.private

The file "mail.txt" contains the text to put in the DNS DKIM record for mail._domainkey

Remember to restart opendkim and postfix.

Test it by sending mail to [/cdn-cgi/l/email-protection [email protected]] and also by sending mail to a gmail.com address and checking the "original message".

I installed postfixadmin for web access and administration. 

dbconfig-common: writing config to /etc/dbconfig-common/postfixadmin.conf

Creating config file /etc/dbconfig-common/postfixadmin.conf with new version 

Creating config file /etc/postfixadmin/dbconfig.inc.php with new version
granting access to database postfixadmin for [/cdn-cgi/l/email-protection [email protected]]: success.
verifying access for [/cdn-cgi/l/email-protection [email protected]]: success.


The user that owns the top level mail folders is vmailbox. There are UID's assigned own the individual folders, starting at 20001. See the 'users' table below. Actual mail folders are here, organized by virtual host domain name: 

/var/mail/vhosts/domainname

Postfix config files

I need to update this. I currently run my own server, not a forwarder.

This is a config that uses Google aka gmail as a smarthost for outbound mail. This config also supports virtual mailboxes via postgresql.

main.cf

smtpd_banner = $myhostname ESMTP $mail_name (AlseaGeOS)
biff = no

# Uncomment the next line to generate "delayed mail" warnings
delay_warning_time = 4h

queue_directory = /var/spool/postfix

myorigin = alseageo.net

# ---------------------------------------------------------

relayhost = [smtp.gmail.com]:587

disable_dns_lookups = yes

# authentication via SASL
# incoming connections
smtpd_sasl_auth_enable = no
smtpd_sasl_local_domain = $myhostname

# outgoing connections
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

# tls 
smtp_tls_loglevel = 1

#smtp_enforce_tls = yes
smtp_tls_per_site = hash:/etc/postfix/tls_per_site

smtp_tls_enforce_peername = no

smtp_tls_CAfile = /etc/postfix/certs/cakey.pem
smtp_tls_cert_file=/etc/postfix/certs/alseageo.pem
smtp_tls_key_file=/etc/postfix/certs/alseageo.key
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_use_tls = yes

smtpd_tls_CAfile = /etc/postfix/certs/cacert.pem
smtpd_tls_cert_file=/etc/postfix/certs/alseageo.pem
smtpd_tls_key_file=/etc/postfix/certs/alseageo.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes

#-------------------------------------------                                    
# Virtual mail support                                                          
#                                                                               
virtual_mailbox_base = /var/mail/vhosts
virtual_minimum_uid = 20000
#                                                                               
# Old file version                                                              
#transport_maps = hash:/etc/postfix/transport
#virtual_uid_maps = static:20000
#virtual_gid_maps = static:20000
#virtual_mailbox_domains = dispatch.incidentview.com
#virtual_mailbox_maps = hash:/etc/postfix/vmailbox
#virtual_alias_maps = hash:/etc/postfix/virtual
#                                                                               
# New SQL version                                                               
transport_maps = pgsql:/etc/postfix/transport.cf
virtual_uid_maps = pgsql:/etc/postfix/uid.cf
virtual_gid_maps = pgsql:/etc/postfix/gid.cf
virtual_mailbox_domains = pgsql:dispatch.incidentview.com
virtual_mailbox_maps = pgsql:/etc/postfix/mailbox.cf
virtual_maps = pgsql:/etc/postfix/virtual.cf


# ---------------------------------------------------------

command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix

mail_owner = postfix

myhostname = kilchis.alseageo.com
mydomain = alseageo.com

inet_interfaces = all

# This is to allow the /etc/aliases file to have an effect.    
mydestination =
        kilchis.alseageo.com kilchis
        roaring.alseageo.com roaring
        minam.alseageo.com minam
        white.alseageo.com white
        dev.alseageo.com dev
        fall.alseageo.com fall
        localhost localhost.localdomain kilchis.localdomain

mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

mail_spool_directory=/var/spool/mail

mynetworks = 127.0.0.0/8 10.1.10.0/24 10.8.0.0/24
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

aliases.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT forw_addr FROM aliases WHERE alias='%s'

transport.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT transport FROM transport WHERE domain='%s'

mailbox.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT mailbox FROM postfix_mailboxes WHERE userid='%s'

uid.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT uid FROM users WHERE userid='%s'

gid.cf

user=mailreader
password=<secret>
dbname=mail
hosts=localhost
query=SELECT gid FROM users WHERE userid='%s'

virtual.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT userid FROM postfix_virtual WHERE address='%s'

virtual-domains.cf

user=mailreader
password=secret
dbname=mail
hosts=localhost
query=SELECT domain FROM virtual_domains WHERE domain='%s'

Testing Postfix

To send mail on the host, I want the address to have the domain not the hostname, 

date | mail bwilson

should go to [/cdn-cgi/l/email-protection [email protected]] not [/cdn-cgi/l/email-protection [email protected]]

This is controlled by "mail" NOT postfix. So put this in /etc/mailutils.conf 

address {
   email-domain w6gkd.radio;
}
  1. Can I send from w6gkd.radio?
  2. Can I send to [/cdn-cgi/l/email-protection [email protected]]?
  3. Are the letsencrypt keys working?

/etc/cron.weekly runs /usr/local/sbin/renew_certs.sh

See /etc/letsencrypt/live to see what is set up

Additional Postfix configuration: Filters

2023-02 I will be working on these soon but I did not let them impede leaving Google.

IMAP server: Dovecot

I made a valiant effort at installing Dovecot into a Docker container. I failed. It's installed in the host.

I wonder if it might be better to install it at home and just keep the SMTP agent on the VPS.

Install Dovecot

Currently I have installed Dovecot from apt. Specifically "dovecot-core" and "dovecot-imapd". I am only using files (well, postfix maps) for auth and mail delivery. No fancy SQL mailboxes right now, it's only me here, that would be massive overkill.

I am migrating it to a Docker container soon.

Test Dovecot

Dovecot is the IMAP server so it has to work alongside Postfix. http://wiki2.dovecot.org/TestInstallation

Resources

The Book of Postfix at Percipio

Retrieved from "https://wiki.wildsong.biz//index.php?title=Postfix&oldid=13686"