Get started with Fetchmail, Procmail and Dovecot

Sysadmin

Having already shown you how to run your own web server using Apache, we'll now turn our attention to the most important application of networking: email. Running your own mail server may seem like overkill, but there are a number of good reasons for doing so. And if you consider yourself well-versed in the lore of sysadmin, this is definitely a topic you need to be comfortable with. Read on!

  • For a start, you can access your mail from more than one computer (for example, a desktop and a laptop, or one machine at work and one at home) but keep both in sync.
  • You can download mail from several email accounts (your ISP, Google Mail and so on) putting them all in one place.
  • You can download mail for all family members and sort it into separate mailboxes.
  • Mail downloads become much faster, because the slow part of pulling the information from your ISP has already been done in the background.
  • Last, but by no means least, you can run spam and virus filtering software to sort out the rubbish from the good stuff before you even fire up your mail program.

There are several aspects to mail serving: receiving mail from outside, delivering it to local mailboxes, serving mail from the local mailboxes to clients, providing web access to those mails and receiving and forwarding outgoing mail from the clients.

There are two ways of getting the mail on to your system; you can run a full-blown mail server such as Postfix and set up the MX records to point to your IP address, or you can pull it from an external mailbox using Fetchmail. We've already covered how to build your own email server with Postfix, so here we're going to show you how the other option, Fetchmail, works.

Get the mail

Fetchmail connects to one or more mailboxes, downloads the mail from them and delivers it to local mailboxes. The first step is to use your package manager to make sure that Fetchmail and Procmail are installed. Fetchmail reads its configuration from either ~/.fetchmailrc when run as a user or /etc/fetchmailrc if run as a system process from init. You can create a .fetchmailrc file by running fetchmailconf and filling in your details. Move this to /etc/fetchmailrc if you plan on running Fetchmail as an init service. Or you can create a config file by entering

set daemon 300
poll mail.myisp.com with proto POP3
  user 'myispuser' there with password 'mypass' is 'myuser' here options keep
mda '/usr/bin/procmail -d %T'

The first line asks Fetchmail to check your mailboxes every 300 seconds (five minutes), the last line tells it how to deliver the mail. By default, Fetchmail tries to connect to a local SMTP server, but we won't be setting one of those up in this tutorial, so we'll use Procmail to deliver the mail for us instead.

The middle two lines are really one line split for ease of reading. These tell Fetchmail to poll the POP3 mailbox for myispuser at mail.myisp.com and deliver the mail to myuser on the local machine. The options keep part instructs Fetchmail to leave the mail on the server, which you should use until you're sure things are working, and then you can remove it. You may have any number of poll lines, pulling mail in from various mail servers or for different users.

If you want to pull mail from your Google Mail account on to your local server, you must enable POP3 access in Google's settings.

If you want to pull mail from your Google Mail account on to your local server, you must enable POP3 access in Google's settings.

If you want to collect mail from a Google Mail account, you'll need to enable POP3 access in the Forwarding and POP/IMAP section of the Gmail settings. Fetchmail is fussy about the order in which options appear in the config file; the global options, such as Set Daemon, must appear first, then the server poll options and finally the delivery settings must follow all the server settings.

Because the configuration file contains passwords, it must be readable only by the user running Fetchmail, or the program aborts with an error. If using it as a service, run

chown root: /etc/fetchmailrc
chmod 600 /etc/fetchmailrc

...to avoid any potential issues.

Deliver it to the users

Now Fetchmail is set to collect mail from your accounts and pass it to Procmail for delivery, but before you start it up, we need to make sure Procmail knows what to do with the mail. Once again, this can use either per-user configurations in ~/.procmailrc or global ones in /etc/procmailrc. If both exist, the user configuration is read after the global file, so you can override the global settings for each user. A suitable /etc/procmailrc would contain the following:

MAILDIR=/var/spool/mail
DEFAULT=$MAILDIR/$LOGNAME/
LOGFILE=/var/log/procmail
VERBOSE=on

The first two lines are important, because they tell Procmail where to store the mail. DEFAULT must end in a trailing / to inform Procmail to use maildir storage, which we'll need for the IMAP server shortly. The other two lines are useful during testing, but you can remove the VERBOSE setting once everything is working as it should. Create the directories for each user with this command:

mkdir -p /var/spool/mail/myuser
chown myuser:mail /var/spool/mail/myuser
chmod 770 /var/spool/mail/myuser

Test your setup by running

fetchmail --daemon 0 -v -f /etc/fetchmailrc

...which runs Fetchmail in a terminal and shows you everything it's doing. After this, you should have a file in /var/spool/mail/user/new for each mail downloaded. Press Ctrl+C to stop this process and then set Fetchmail to run as a startup service in your distro's services manager.

Let them read it

There are two main ways of retrieving your mail from a server, whether that server is at an ISP, sitting under your desk or even on the same computer. POP3 connects to the server and downloads all mail since the last time you connected, storing that mail on the local system. That's the way we used it in the days of dialup, and it's how we're using Fetchmail here - except we're transferring email from your ISP to your new server.

Unfortunately, POP3 has several disadvantages when used with mail clients. The main one is that each client has to work with a separate copy of your email and can only track information about those mails it has downloaded. Even if you set all your programs to leave your mail on the server after they've download it, the mail program on your laptop will have no idea which of those emails you have already read, or replied to, on your desktop computer.

The other choice is IMAP - a newer protocol that keeps the mail on the server and reads it from the clients, although most clients keep a cached copy of anything you read, to save downloading it again each time you want to read it. All flags, such as read/unread, important and so on, are stored on the server and are visible to whichever computer you use to read the mail.

Similarly, if you delete a spam mail on one computer, it's gone when you use another. Sent emails are also stored on the server, so all the computers you use to read your mail are fully in sync. The one drawback of IMAP is that it uses the network in real time, so a slow connection to the mail server can cause a slowdown, which is another good reason to run your own local server at Ethernet speeds.

There are several IMAP servers available and, unlike the web servers we looked at previously, there's no clearly dominant choice. We'll use Dovecot (www.dovecot.org) here, because it works well, it's secure and it's straightforward to set up. If you want alternatives, try out Cyrus (http://asg.web.cmu.edu/cyrus/imapd) and Courier (www.courier-mta.org). Install your chosen option in the usual way through your package manager.

The default configuration for Dovecot needs little alteration for a standard IMAP setup. If your distro didn't install a configuration file (usually /etc/dovecot/dovecot.conf), copy dovecot-example.conf to this file instead. Now open it in your favourite text editor, as root, find the section for Mailbox Locations and Namespaces and add this line:

mail_location = /var/spool/mail/%u

This tells Dovecot where each user's mail is stored and should match the DEFAULT setting for Procmail. Then find the line that reads

#protocols imap imaps

...and remove the leading # to enable IMAP. If you want to access the server using POP3 too, add pop3 pop3s to the line. These variants use SSL, and are needed if you want to access the server from outside of your LAN. If you intend to only use Dovecot on your local network and your router isn't set up to forward IMAP or POP3 connections from the outside to your server you can do without IMAPS and POP3S.

However, if you plan to allow connections from the outside world, say from your laptop, you really should make use of the SSL options. Without them, your login and password are transmitted as plain text, which is readable by anyone with access to any of the routers or wireless data streams between you and your server. More options for enabling an SSL connection are covered in Keep It Secure, below. Other options you should set are:

log_path = /var/log/dovecot
mail_privileged_group = mail
protocol pop3 {
 pop3_uidl_format = %v.%u
}

These are already in the config file, but commented out and often with no value. The first sets logging (syslog is the default), the second is the group used to create new files, which is why we made the mail directories group writable and owned by the mail group. The pop3 section is only needed if you use POP3 and it controls the format of the UIDL records - the unique IDs assigned to mails, so that mail clients can ensure they don't download the same mails over and over again when they are left on the server.

Don't worry about this if you're using IMAP, which is often the better alternative. There's one more option you may need to set when testing:

disable_plaintext_auth = no

This allows plaintext logins over a non-secure connection, which you may need to use if you're testing a mail client on another computer on your network. The default is to disallow plaintext logins unless you're either using SSL/TLS or are connecting from the same computer, which is considered inherently secure.

Now you can go into your distro's services manager and start Dovecot, or restart it if it was already running. While you're there, make sure it's set to start when you boot. Fire up your favourite mail client, then set up an IMAP account to connect to your server and look for emails. If you don't see any, check the log and config files for errors and try again.

Make sure your firewall allows IMAPS and POP3S connections, plus the non-secure versions for use over a local, wired network (stick to IMAPS for wireless).

Make sure your firewall allows IMAPS and POP3S connections, plus the non-secure versions for use over a local, wired network (stick to IMAPS for wireless).

Firewalls and routers

Fetching mail from your ISP's mail server is initiated locally, but connecting to your mail server from another computer requires permission from your firewall. The ports you may have to open are:

 IMAP 143
 POP3 110
 IMAPS 993
 POP3S 995

Open whichever of these ports you're going to use in the firewall on your server. If you want to connect from outside, you also need to set your internet router to forward TCP connections on these ports to the computer running the server. In this case, you should set up SSL (as described in Keep It Secure, right) and only forward port 995 (and 993 if you want to use POP3). That way you can test without SSL on your local network, but force the internet to use more secure connections.

Keep it secure

If you want to access your server from outside of your network, you'll need a secure connection. This means that the data you're sending is encrypted, and certificates are used to verify that the server you're connecting to is the one you meant to connect to, which prevents any potentially harmful man-in-the-middle exploits.

Before you can set Dovecot (or any other server) to provide secure connections, you'll need a certificate. For a commercial project, you should buy one from a recognised certificate authority, because this provides a level of trust. If you're running a personal server, a self-signed certificate is fine. Dovecot includes a script to create and sign the certificate at /usr/share/doc/packages/dovecot/mkcert.sh.

The first step is to edit the dovecot-openssl.cnf file in this directory and change the settings to suit. The CN item is the most important one here, because it must contain the host name of your server. If it doesn't, or if this doesn't match the address used by the mail client to connect, all connections will be rejected. This also forms the externally visible name of your server if you intend to connect from outside.

Save the file and run mkcert.sh, which will create two files in /etc/ssl/. With that done, edit dovecot.conf and add (or uncomment) the lines

ssl_disable = no
ssl_cert_file = /etc/ssl/certs/dovecot.pem
ssl_key_file = /etc/ssl/private/dovecot.pem

The paths to the certificate files may vary, but some distros put them in /etc/ssl/dovecot.

The first time your mail client connects to a server with a self-signed certificate, it will ask for confirmation. It's crucial that you connect over your local network when you make this first request and confirm the sever's location, because this will ensure you're connecting to the correct server.

Delivering to more than one user

You now have a fully functional mail server that can download mail from one or more mailboxes and make it available for reading with any IMAP mail client, anywhere in the world (well, anywhere with an internet connection anyway). At the moment it drops the mail from each external mailbox into a single user's inbox, but there's much more you can do with Procmail. First, let's sort mail for different users. If you can accept mail for anyuser@yourmaildomain.com, you may want to sort these incoming mails for different users, so add these lines to /etc/procmailrc

:0:
* ^Delivered-To: user@yourmaildomain.com
!user

This is the simplest of Procmail recipes; check the man pages if you want to see a few more elaborate examples. The first line (beginning with a :) starts the recipe, and the second line (starting with an *) is matched against the mail. The recipe looks for a Delivered-To header that matches one particular user. There can be any number of these match lines, and all must be valid for the recipe to proceed.

The last action line tells Procmail what to do with the mail. The leading ! means that this is a mail address to deliver to, while the lack of an @ indicates this is a local user. Note that Procmail stops at the first matching recipe that delivers the mail. If none is found, the mail is delivered to the default address. You can use standard egrep regular expressions as match rules, so you could match mail sent to john, john.smith and jsmith with

* ^Delivered-To: (john|john\.smith|jsmith)@yourmaildomain.com

Sorting your own mail

Once Procmail has decided which user to deliver an email to, it looks for a .procmailrc file in their home directory. This can be used to sort mailing list posts into their own folders, instead of setting up clients on each computer to do so. For example

:0:
* List-Id: geeks-anonymous
$MAILDIR/$LOGNAME/.INBOX.Geeks\ Anonymous/

Note the use of a backslash to escape the space in the list name, and the trailing forward slash to make sure it's delivered into a directory. Add some of these to your ~/.procmailrc directory and your mailing list posts are sorted on arrival. You can even add a holiday response message to ~/.procmailrc by including these lines

:0 Whc: $HOME/.vacation.lock
* $^(To: .*$LOGNAME|CC: .*$LOGNAME)
* !^FROM_DAEMON
* !^List-
* !^(Mailing-List|Approved-By|BestServHost|Resent-(Message-ID|Sender)):
* !^Sender: (.*-errors@|owner-)
* !^X-[^:]*-List:
* !^X-(Authentication-Warning|Loop|Sent-To|(Listprocessor|Mailman)-Version):
* !$^From +$LOGNAME(@| |$)
| /usr/bin/formail -rD 8192 $HOME/.vacation.cache
:0 ehc
| (/usr/bin/formail -rI"Precedence: junk" \
 -A"X-Loop: $LOGNAME@example.com" ; \
 cat $HOME/.vacation.msg ) | $SENDMAIL -t

The first recipe excludes list and system emails, and records every other sender in a file. The second, which only runs if the first succeeds, sends the email. The idea of the cache file is that you only send a holiday response to the first mail from each sender. The Procmail man pages will help you translate the rest of these recipes.

As recipes are run in order, put the most used ones first. For me, these are the spam ones, followed by scripts to deal with the busier mailing lists, reducing the time Procmail has to spend checking each mail. Make sure your holiday response message appears last, because people hate receiving vacation notices on mailing lists.

What else can you do?

We've used Procmail to deliver the mail, but you can also pass it to another program that processes the mail and then gives it to Procmail. This is most often used for spam and virus filtering, where the mail can be passed to SpamAssassin, which then passes it to ClamAV and that finally gives it back to Procmail.

Since the previous two programs will mark any suspect mails, you can set up Procmail recipes to filter these mails to a quarantine area, or even /dev/null.

Glossary

  • MTA Mail Transport Agent (or Mail Transfer Agent) - The program that transfers mail from one computer to another, using SMTP. Examples are Postfix, Sendmail and Exim.
  • MDA Mail Delivery Agent - Once an email has arrived on a machine, this is the program that delivers it to the user's mailbox. Many MTAs can handle this, but it's more common to pass the message to a dedicated MDA like Procmail or Maildrop.
  • MUA Mail User Agent - any client program used to download, store or read mail. KDE users will be most familiar with KMail, while terminal junkies use Mutt.
  • Daemon A program that runs in the background, waiting for connections. These are usually servers and often have a name ending in d, such as sshd or ftpd.
  • mbox A way of storing mailbox messages. The mbox format stores all messages in a single file, making it more efficient in terms of disk space. It's also susceptible to corruption, and one error can make several messages unreadable.
  • maildir An alternative to mbox. Each message is stored as a separate file within a mail directory. As well as providing faster access and better security, it also allows for a folder hierarchy. This format is required to use IMAP effectively.
  • MX record A DNS server record that tells an MTA which IP address should be used to receive mail for a particular domain name.
First published in Linux Format

First published in Linux Format magazine

You should follow us on Identi.ca or Twitter


Your comments

terminal users are not junkies

"terminal junkies use Mutt" ? why do you call us junkies? i use terminal every day not because I can't afford a new computer or trying to conserve some resources or any other silliness. Terminal usage is a must-have! skill if you do system administration or any kind of professional software development.

Bad info

Those cialis and Viagras you've been getting are poo.

how to forward email to an email address

how to make procmail forward the emails to an email address other than a local user

I tried change the local user name with a email address.
!xxx@xxx.com

It seems to need more configuration of postfix to make it work. Could anyone help me with this issue?

But the dry skin remains

But the dry skin continues

With a neon lighting, You definitely will illuminate almost all the shed. It is also necessary that the light is not installed to be by your back as you are sitting or standing. If your project area is thrown with some shadow, You will find it hard to see what you are working on,

Watch the film and read the book for a better grasp of Leslie Burke's individuality. Take notes on anything from her fashion to the way she uses vernacular. Leslie has a very positive lifestyle and seems to always have a smile on her face, With a spark in her eye.

Shopping for ideal birthday gift doesn have to be stressful. vital of all, you have to have a general idea of what the recipient likes. Show you care with a personalized gift and help make your partner birthday the best one yet! The better you know the recipient the more unique birthday gift idea you can write down,

Итальянская керамогранитная плитка

Спасибо, очень интересная заметка.

винный цвет

Автор продолжай в том же духе

This is my first forum

All the best from Russia. I'm new on this website and want simple tell hi.

air max 1

projects\a\c34.txt

Взлом почты на заказ.

Это сервис, предназначенный для взлома и поиска
утерянных паролей от почтовых ящиков.
Получить то, что для многих пользователей почтовых
серверов является секретом, который закрыт от посторонних глаз,
для нас не составляет особого труда.
Имея большой опыт, мы можем с лёгкостью
завладеть самой ценной информацией.

Для чего люди заказывают взлом?
В наше время информация является мощным средством
для достижения разнообразных целей. Поэтому,
в некоторых случаях email сегодня может стоить
дороже сейфа, в котором спрятаны драгоценности.
Ведь именно в электронном почтовом ящике может находиться
наиболее сокровенное, начиная от переписки любого
близкого человека, в том числе и ребёнка,
за которым всегда нужен глаз да глаз,
заканчивая ценными секретами конкурентов по бизнесу.
Сервис создан именно для людей.
В этом случае, не рискуя потерять драгоценное время
можно сразу же сейчас заказать взлом любой почты,
который наши специалисты выполнят в самые короткие сроки!
Гарантии, которые этот сервис предоставляет каждому
своему клиенту, обеспечивают нам одну из наиболее
высоких репутаций среди подобного вида сервисов.

Почта: hackerpromail@gmail.com
Скайп: hackerpromail

What do you think of the

What do you think of the White House touting Voter ID cards?

zerrin egelileri seven biri

zerrin egelilerden hoslanan var mı?

erhthz

herhferth

Public pictures

Porn from social networks
http://twerking.erolove.in/?facebook.angela
hermaphrodite sex amateur bogyaraj sex twink boys muscle xhamster homemade strip teen

Obviously far highest there rate, beyond sense brand Brand and

[url=http://www.rifitex.com/images/botter.asp]louis vuitton outlet[/url] La prpondrance des conomies nettes vrificateurs a rvolutionn l'achat au monde. The funny thing about the fashion world is today something is a trend that everyone has to have or they will die without it, and tomorrow it is something different. Today Hermes has been globally renowned for its exclusive leather items for instance shoes and bags along with perfumes.
[url=http://www.logodesignerdirectory.com/images/header.cfm]Cheap Jerseys Free Shipping[/url] the manager was Cheap Louis Vuitton Taiga Leather Laguito Bags M31098 Brown louis vuitton handbags on sale uk louis vuitton diaper bag baby Louis Vuitton Zippy Wallet M93026 louis vuitton for men belts Louis Vuitton Multiple Wallet N60895 louis vuitton coin purse amazon Louis Vuitton Robusto 2 Compartment M31042 Louis Vuitton Sunset Boulevard M91679 louis vuitton france bags Louis Vuitton Cabas Beaubourg M53013 louis vuitton accessories outlet Louis Vuitton Kalahari PM M97016 louis vuitton outlet nj Louis Vuitton Raspail PM M40608 louis vuitton jewelry case replica. He could say that he can stop whenever he desires to quit. Leaving one particular piece to himself, he devoted the other twelve items to the twelve Olympians in Olympus.
[url=http://www.gosporttravel.com/nyhetsbver.cfm]Louis Vuitton Handbags On Sale outlet[/url] Since handbags from Hermes are exclusive creations, there are some people who are taking advantage of the huge demand for these designer bags the Hermes replica has a great impact and so these Hermes replica bags have been able to reach every corner of the worldalso because the authentic Hermes bags are at a very high price only a few people can afford them.. My group is an idiot to enable types and designs, And in the past I in areas a replica of some 33 1/3 education audio novels, Papers inside your retailers independent quantity supplier normally individuals were available on the market close to throughout lps. Mentioning this is important as many other fashion brands don put nearly as much effort into the design of their timepiece (with the French houses being an exception)..
[url=http://www.goevent.se/omgevent.cfm]Louis Vuitton Handväskor på Sale utlopp[/url] Many in the Bass Fishing lures also range from the fat, some fishing lures are ultra lgt and they are really angled in order that it can trigger a spin. Eventually Hey Ya became a hit.. That exactly what Hermes stands for the chic understated elegance with the latest of trendy fashion.
[url=http://www.stilography.com/img/common/monlv.html]Louis Vuitton Handbags[/url] The one-of-a-kind design takes the Isabel Marant sneaker wedge to another level with what the company called the "Eden treatment." The press release says "A hybrid of land and sea skins are used to create this handcrafted creation. Michele Norsa, CEO of Salvatore Ferragamo, the legendary designer Italian shoe empire, was there. Real people are very wise which will our god will probably formidable device.
[url=http://www.green-hotel.it/images/loction.asp]Louis Vuitton Handbags On Sale outlet[/url] For anybody who is this individual that is able to acquire purses and handbags like Hermes Birkin, Hermes Kelly felix, Hermes Lindy, do not fret, you need to them how to start working. For this collection, timed to Thursday's opening of a Sprouse retrospective at New York's Deitch Gallery and the release of a Sprouse Rizzoli book on Feb. Quite simply, females associated with fields like the Celine archives.
[url=http://www.microcredito.lazio.it/footer.asp]Sconto Louis Vuitton[/url] I was completely attracted by a shine bag. Your only second I found myself possibly that you experienced definitely let down was initially to look at purchased a couple of ASICS. The first impression that one makes is the previous impression and for working women, especially those who work in the showbiz, their out look is the main.
[url=http://www.gruppolh.it/umbraco/log.aspx]sito web low cost Louis Vuitton ufficiale[/url] Malaisia. Ghana. It is definitely a knockout look.

[url=http://www.strategis.co.uk/footer.cfm]Louis Vuitton Handbags On Sale outlet[/url] louis vuitton outlet online I believe you need to give more prominence to the three nations that have actually won it - USA, Australia and NZ, and the two nations that have successfully defended it - USA and NZ. Another pink Hermes bag was sold with regard to $140, 000! These bags are generally so outstanding concerning be beyond comparison with their matchless workmanship. The burden of what, I do not say so, we guessed. [url=http://www.goevent.se/omgevent.cfm]Louis Vuitton utlopp Online Sale[/url] Louis-Ferdinand Destouches met Cillie Pam in Paris, at the Caf de la Paix, in September 1932. Alemanha. And then you have your friends, and you have your family, and you have your own doctors, and you count on them. [url=http://theaquamarine.net/marlv.html]Louis Vuitton Handbags[/url] So why not a Rue Margaret Thatcher, some French politicians are asking. Ricky ran out in an instant, By working that have surplus loochi, Superior ranks, Clients conversing, Acquire-Succeed in for each. Limited edition and destined to be a favorite of Louis Vuitton and camouflage fanatics alike.. [url=http://www.hotelaquiete.it/images/logfile.asp]raccoglitore di uscita Cheap Louis Vuitton[/url] ShoppingWhen looking for the best a newly purchased running shoes, bring in the proper tools. fell. "And for Eddie (his brother), that was another hero of our lives as well. [url=http://www.hotelatlantic.com/footer.html]portafogli e portamonete Cheap Louis Vuitton[/url] Michael PhelpsFILE - In this Aug. It is 100% UV protection and made by metal frame with hand-polished acetate arms. So that is something to think about - your money. [url=http://www.4site.it/webalizer/main.asp]Louis Vuitton Sale[/url] Moreover it also boasts of the latest craftman ID as 'H' with numbered lock and keys. Blgica. Rangers which of you happen to be to assault Point du Hoc properly trained judging by ascending cliffs and a guy made practice beach landings.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

CAPTCHA
We can't accept links (unless you obfuscate them). You also need to negotiate the following CAPTCHA...

Username:   Password:
Create Account | About TuxRadar