Postfix will be used to receive mails, it will use Rspamd as effective spam filter and check over LMTP with dovecot if mail adresses exists and if yes, deliver them to dovecot, as long as rspamd doesn't reject the mail.
Dovecot will authenticate user against an Ldap server and only accepts mail for users of the group mail. The Ldap data actually come from an keycloak installation, but that is irrelevant here .
Solr is used to eficiently search mails, you mainly need that for webmail clients.

Let's get Dovecot working.

aptitude install dovecot-core dovecot-imapd dovecot-ldap dovecot-lmtpd dovecot-managesieved dovecot-sieve dovecot-solr bsd-mailx

In most config files you can just uncomment the necessary settings.

Edit file /etc/dovecot/conf.d/10-auth.conf

!include auth-ldap.conf.ext
#!include auth-system.conf.ext
#To strip the domain name from the username before authentication, and make it lowercase
auth_username_format = %Ln

Edit file /etc/dovecot/dovecot-ldap.conf.ext uncomment and set:

hosts = 127.0.0.1
auth_bind = yes
base = ou=People,dc=example,dc=com
pass_attrs = uid=user 
# only Members of group mail may receive mails and login to see them.
pass_filter = (&(objectClass=inetOrgPerson)(uid=%u)(memberof=cn=mail,ou=Group,dc=example,dc=com))

Restart dovecot and test authentication:

servicectl restart dovecot
doveadm auth test exampleuser

Edit file /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/mail
mail_privileged_group = mail
#if we use mail, as owner for the mail directories, we need to
#change the frst_valid_uid here, to te uid of mail. otherwise create a new user. e.g. vmail
first_valid_uid = 8

# some performance improvement
maildir_very_dirty_syncs = yes

edit file /etc/dovecot/conf.d/10-ssl.conf (assuming you've already setup letsencrypt or some other certificates)

ssl = required
ssl_cert = </etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/example.com/privkey.pem

Edit file /etc/dovecot/conf.d/10-master.conf

# use lmtp for mail delivery from postfix
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp { 
    user = postfix    
    group = postfix
    mode = 0600
}

service auth {
...
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }

....
}

edit file /etc/dovecot/conf.d/15-mailboxes.conf
and add to every mailbox:

 
auto = subscribe

edit file /etc/dovecot/conf.d/20-lmtp.conf

protocol lmtp {
  # Space separated list of plugins to load (default is global mail_plugins).
  postmaster_address = postmaster@example.com   # required
  # add plugins here which should be supported by lmtp. we use sieve for sorting spam to spam folder
  mail_plugins = sieve $mail_plugins
}

Add solr support for searching in mails:

aptitude install solr-tomcat

to make tomcat only listen on localhost edit file /etc/tomcat8/server.xml and add address attribute to connector directive:

<Connector address="127.0.0.1" port="8080" ..../>

you need to copy dovecot solr schema:

mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.dist
ln -s /usr/share/dovecot/solr-schema.xml /etc/solr/conf/schema.xml

Then restart tomcat:

systemctl restart tomcat8

edit file /etc/dovecot/conf.d/90-plugin.conf

plugin {
  ...
  fts = solr
  fts_solr = url=http://127.0.0.1:8080/solr/
  ...
}

edit /etc/dovecot/conf.d/10-mail.conf

mail_plugins = fts fts_solr

Create file /etc/cron.d/solr-optimize

# dovecot-solr commits & optimization
# http://wiki2.dovecot.org/Plugins/FTS/Solr

# Commit should be run pretty often, e.g. every two minutes
*/2 * * * * root /usr/bin/curl -s http://localhost:8080/solr/update?commit=true &>/dev/null

# Optimize should be run somewhat rarely, e.g. once a day
23 3 * * *  root /usr/bin/curl -s http://localhost:8080/solr/update?optimize=true &>/dev/null

use fail2ban with dovecot to ban ips which try several times to authenticate unsuccessful, add to
/etc/fail2ban/jail.local

[dovecot]
enabled = true

Install postfix

aptitude install postfix

Add alias for root

echo root: user@example.com >> /etc/aliases
postalias /etc/aliases

edit /etc/postfix/main.cf

#comment out
#mydestination = ...

delay_warning_time = 4h

smtpd_tls_cert_file=/etc/letsencrypt/live/example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/example.com/privkey.pem

#comment out
#smtpd_use_tls=yes
smtp_tls_security_level=may
smtpd_tls_security_level=may
smtpd_tls_auth_only = yes

# sasl authentification against dovecot
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination

# reject_unverified_recipient uses lmtp to verify if receiver exists.
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unverified_recipient, reject_unauth_destination

#allow bigger messages
message_size_limit = 73400320

virtual_mailbox_domaines = example.com
# Transport to dovecot
virtual_transport = lmtp:unix:private/dovecot-lmtp

#map with aliases
virtual_alias_maps = hash:/etc/postfix/virtual-alias-map
# If you have some docker containers or similar stuff, we need to add the
# 172.17.0.0/16 subnet to mynetwork, so they are able to send mail.
# If you don't use that subnet, you can leave that directive on it's default setting.
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 172.17.0.0/16

Our virtual alias map file. It maps some important accounts, propably some admin user wants to receive those mails.
/etc/postfix/virtual-alias-map

postmaster@example.com adminuser@example.com
www-data@example.com adminuser@example.com
root@example.com adminuser@example.com

Postfix will spam syslog with every single connection it made. So we tell it not to log to syslog, you get the same information in mail.info anyway….
Edit file /etc/rsyslog.d

change line

*.*;auth,authpriv.none		-/var/log/syslog

to

*.*;auth,authpriv.none,mail.none		-/var/log/syslog

The documentation about the functionality and configuration of rspamd is a bit sparse.
The one here is working, but no garantees, that it is perfect….

Install rspamd and redis.

CODENAME=`lsb_release -c -s`
wget -O- https://rspamd.com/apt-stable/gpg.key | apt-key add -
echo "deb http://rspamd.com/apt-stable/ $CODENAME main" > /etc/apt/sources.list.d/rspamd.list
echo "deb-src http://rspamd.com/apt-stable/ $CODENAME main" >> /etc/apt/sources.list.d/rspamd.list
aptitude update
aptitude install rspamd redis-server

edit /etc/redis/redis.conf (according to rspamd.com)

maxmemory 500mb
maxmemory-policy volatile-lru

Config files in /etc/rspamd/local.d override defaults settings.

Create /etc/rspamd/local.d/worker-normal.inc

bind_socket = "localhost:11333";

Create a passwordhash for the webinterface.

rspamadm pw

Then enter that hash in the file /etc/rspamd/local.d/worker-controller.inc

#password for read access
password = "$2$17qeh8cdsqxgufkz9or9ecm6uquj6duk$tbniammzqfxdigogkm1abdoa78pmfzag4u5xqkgswabpp8zxrkzb"
#password for write access. you need to set both. I know, it's stupid ;-)
enable_password = "$2$17qeh8cdsqxgufkz9or9ecm6uquj6duk$tbniammzqfxdigogkm1abdoa78pmfzag4u5xqkgswabpp8zxrkzb"

/etc/rspamd/local.d/worker-proxy.inc (milter for postfix)

bind_socket = "localhost:11332";
milter = yes;
timeout = 120s;
upstream "local" {
    default = yes;
    self_scan = yes;
}

We want that rspamd adds some headers to the mails. For details see the documentation
Create file /etc/rspamd/local.d/milter_headers.conf

use = ["x-spam-status", "x-spam-level", "authentication-results"];
skip_local = false;
authenticated_headers = ["authentication-results"];

rspamd is incredibly verbose. let's decrease logging:
Create /etc/rspamd/local.d/logging.inc

level = "warning";

activate redis support
Create file /etc/rspamd/local.d/redis.conf

servers = "127.0.0.1";

For Bayes filter redis support needs to be activated explicitly
Create /etc/rspamd/local.d/classifier-bayes.conf

backend = "redis";
autolearn = true;

To be able to access the rspamd webinterface we need to add a config file to apache.
create /etc/apache2/conf-available/rspamd.conf

RewriteEngine on
RewriteRule ^/rspamd$ /rspamd/ [R]
<Location /rspamd>
        ProxyPass http://localhost:11334
        ProxyPassReverse http://localhost:11334
        ProxyPreserveHost On
</Location>

Then enable it:

a2enconf rspamd

Make postfix use rspamd, therefore add following lines to
/etc/postfix/main.cf

#TODO: It would be more effective using unix sockets.
#smtpd_milters = unix:/var/lib/rspamd/milter.sock
smtpd_milters = inet:localhost:11332
non_smtpd_milters = inet:localhost:11332
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
# skip mail without checks if something goes wrong
milter_default_action = accept

We want Dovecot to automatically move new mails to spam folder when they were recognized as spam mail. Additionally, when the user moves mail to the spam folder or away from spam folder, rspamd should learn them as spam or ham.

Create file /etc/dovecot/sieve/spam.sieve

require "fileinto";

if header :contains "X-Spam-Flag" "YES" {
    fileinto "Junk";
}

if header :is "X-Spam" "Yes" {
    fileinto "Junk";
}

Create file /etc/dovecot/sieve/learn-spam.sieve

require ["vnd.dovecot.pipe", "copy", "imapsieve"];

pipe :copy "rspamc" ["learn_spam"];

Create file /etc/dovecot/sieve/learn-ham.sieve

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.mailbox" "*" {
  set "mailbox" "${1}";
}

if string "${mailbox}" "Trash" {
  stop;
}

pipe :copy "rspamc" ["learn_ham"];

Since dovecot does not have write permission for /etc/dovecot/sieve we need to compile the sieve scripts by hand

sievec learn-ham.sieve
sievec learn-spam.sieve
sievec spam.sieve

For the learning scripts to work, they need to be added to the dovecot configuration:
edit /etc/dovecot/conf.d/90-plugin.conf

plugin{
...
  #to make spam stuff work
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_before = /etc/dovecot/sieve/spam.sieve
  sieve_global_extensions = +vnd.dovecot.pipe
  sieve_pipe_bin_dir = /usr/bin
  # From elsewhere to Spam folder
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_before = file:/etc/dovecot/sieve/learn-spam.sieve

    # From Spam folder to elsewhere
  imapsieve_mailbox2_name = *
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_before = file:/etc/dovecot/sieve/learn-ham.sieve...
}

Edit file /conf.d/20-imap.conf

protocol imap {
 mail_plugins = $mail_plugins imap_sieve
}

file conf.d/20-lmtp.conf{
  mail_plugins = $mail_plugins sieve
}

To let Rspamd dkim sign outgoing mails, following is needed:

mkdir /etc/rspamd/dkim
rspamadm dkim_keygen -b 2048 -s 2018 -k 2018.key > 2018.txt
chown _rspamd:_rspamd -R /etc/rspamd/dkim
chmod 440 /etc/rspamd/dkim/*

Look at 2018.txt to see how your dns entry should look like. and then make that DNS-entry.

edit file dkim_signing.conf

path = "/etc/rspamd/dkim/$selector.key";
selector = "2018";

# I need that, since in my case, usernames are without the domain.
# Otherwise he won't do any dkim signing.
allow_username_mismatch = true;

If you provide mailinglists, mail forwarding or similar stuff, you also want to use arc. It has the same config as dkim. So we'll just generate a symlink for it:

ln -s /etc/rspamd/local.d/dkim_signing.conf /etc/rspamd/local.d/arc.conf

setup clamav for use with rspamd

aptitude install clamav-daemon

choose all the defaults during setup (or think a bit yourself )

add user _rspamd to group clamav

adduser _rspamd clamav

/etc/rspamd/local.d/antivirus.conf

clamav {

# if we want to detect fishing mail and stuff we should deactivate that
# also if you want to detect a mail with the eicar signature
# X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
attachments_only = false;
symbol = "CLAM_VIRUS";
type = "clamav";
#log_clean = false;
servers = "/var/run/clamav/clamd.ctl";

# patterns {

  # symbol_name = "pattern";
 # JUST_EICAR = "^Eicar-Test-Signature$";

# }
}

setup unbound for dns caching and forwarding

aptitude install unbound

make localhost main dns server in /etc/resolv.conf

nameserver 127.0.0.1

apache reverse proxy for web interface

more fail2ban

[apache-auth]
enabled = true

[apache-badbots]
enabled = true

[apache-noscript]
enabled = true

[apache-overflows]
enabled = true

[roundcube-auth]
enabled = true

[postfix]
enabled = true

[postfix-sasl]
enabled = true

see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881449 to make it work

[postfix-rbl]
enabled = true

[sshd-ddos]
enabled = true