You are here:

ActiveXperts.com > Support > ActiveEmail > SMTP Authentication
Activexperts Support Pages
ActiveXperts Support Pages

SMTP Authentication

Introduction

SMTP Authentication is a scheme which was introduced in 1999 by J. Meyers of Netscape Communications and finally released as RFC 2554 ("SMTP Service Extension for Authentication"). It is partly based on the SMTP Service Extensions as defined in RFC 1869. Most modern SMTP implementations support SMTP Authentication, whereas Qmail 1.03 does not (without a patch). On the other hand, a lot of Mail User Agents (MUAs) - which include a SMTP Client - make SMTP Authentication available (e.g. Outlook, Eudora, Netscape, Mozilla, The Bat! ....).

SMTP Authentication is advertised by the SMTP Authentication server, requires a client to authenticate, while finally both parties have to mutually accept and support the chosen authentication procedure. Originally invented as a Host-to-Host protocol, with SMTP Authentication, a User has to identify itself and after successful authentication, reception/transmission of his/her emails is granted.

RFC 2554 does not explicitly state, what advantages/benefits a user has being SMTP authenticated, except that optionally a "security layer" for subsequent protocol interactions may be chosen. However, in common sense, an authenticated user is allowed for email transmission not only to the target system (the SMTP server) but rather anywhere. In Qmail terminology, this is equivalent to a 'relayclient'.

SMTP Authentication takes some ideas of the Simple Authentication and Security Layer (SASL) and does not fit well into the SMTP scheme, as will be outlined in this document.

Request For Comments

  • RFC 1869 defines a protocol extension (ESMTP) for the SMTP dialog, in order to indicate extended capabilities by the SMTP Server and/or to transmit additional SMTP protocol information by the SMTP client. SMTP servers, supporting ESMTP, have to use the keyword 'EHLO' in the SMTP greeting. The SMTP client may only use those extensions the server offers. By construction, the server may send the offered extensions as ESMTP verb anywhere in the SMTP dialog or as part of the 'MAIL FROM: ' or 'RCPT TO: ' command. A typical use is 'MAIL FROM: <foobar@example.com> SIZE=1512'. In this sample, 'SIZE' is the ESMTP keyword, '1512' is the ESMTP value and the whole term 'SIZE=1512' is the ESMTP parameter (RFC 1870 " SMTP Service Extension for Message Size Declaration"). RFC 1869 employes two different schemes to promote the ESMTP value:
    • As ESMTP verb, it uses "SIZE xxxxx",
    • whereas in the 'MAIL FROM: <foobar@example.com> SIZE=1512' command, the ESMTP keyword and it's value are joined by a "=" equal sign.
  • In this scope, RFC 2554 describes SMTP Authentication with the particular ESMTP keyword 'AUTH'. In the text passages and samples of RFC 2554, the ESMTP Auth values 'CRAM-MD5', 'DIGEST-MD5', and 'PLAIN' are mentioned (which correspond to particular authentication methods or mechanisms) but no reference to any of those is provided.
  • The attempt, to find the meaning of the above mentioned ESMTP AUTH mechanisms in RFC 2222 "Simple Authentication and Security Layer (SASL)" fails. In this RFC (also published by John Myers), only the overall SASL mechanism is outlined and how to register a new "SASL mechanism name". However, the SASL mechanisms 'KERBEROS_V4', 'GSSAPI', and 'SKEY' are defined.
  • In order to succeed, one has to dig out RFC 1731 "IMAP4 Authentication Mechanisms" and RFC 2195 "IMAP/POP Authorize Extension for Simple Challenge/Response". Here, some practical samples for authentication are given based upon the POP3 and IMAP4 protocol. Those RFC are originated as well by John Myers. RFC 2060 "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1" (John Myers, sic) tells about the IMAP4 'LOGIN' command. Too bad; this has nothing in common with the ESMTP 'AUTH LOGIN' method.
  • The way the actual ESMTP Auth values are en-/decoded, corresponds to the BASE64 scheme, which was first described in RFC 1113 "Privacy Enhancement for Internet Electronic Mail: Part I -- Message Encipherment and Authentication Procedures"; though not explicitly declared as BASE64 here (but later called it that). RFC 2045 "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies" gives an identical outline of the BASE64 "alphabet" in section 6.8.
  • If in addition the Challenge/Response authentication mechanism is used, one has to become familiar with the so-called HMAC procedure from RFC 2104 "HMAC: Keyed-Hashing for Message Authentication" and in addition according to RFC 1321 with "The MD5 Message-Digest Algorithm" as an en-/decryption scheme.
  • Until recently, there was no common understanding, how to propagate the SMTP Authentication information in the email's header. With RFC 3848 however, there exist at least a minimal scheme, including a particular keyword ESMTPA in the last included "Received:" header line in case of an authenticated SMTP session.

It seems to be clear by know, that SMTP Authentication depends upon a patchwork of mechanisms/methods/procedures scattered over a wide range of RFC. Now, we have to go on and discuss the SMTP Authentication framework and will realize, that things are even more complicated.

Authentication Framework

Server Announcement

We take a sample from RFC 2554. "S:" denotes the SMTP Server and "C:" the SMTP Client.

S: 220 smtp.example.com ESMTP server ready
C: EHLO jgm.example.com
S: 250-smtp.example.com
S: 250 AUTH CRAM-MD5 DIGEST-MD5
C: AUTH FOOBAR
S: 504 Unrecognized authentication type.
C: AUTH CRAM-MD5
S: 334
PENCeUxFREJoU0NnbmhNWitOMjNGNndAZWx3b29kLmlubm9zb2Z0LmNvbT4=
C: ZnJlZCA5ZTk1YWVlMDljNDBhZjJiODRhMGMyYjNiYmFlNzg2ZQ==
S: 235 Authentication successful.

Here, RFC 2554 uses multiple values for the keyword AUTH as ESMTP command, which is allow by RFC 1869, however broke the parsing of several ESMTP client implementations. One work around is, to add artificially a "=" (equal sign) between the AUTH keyword and the value, eg. AUTH=LOGIN.

There are three authentication mechanisms widely used for SMTP Authentication. In the documentation coming with the qmail-smtp-auth-patch by Krzysztof Dabrowski, an overview of MUAs and their AUTH mechanisms is provided:

  • AUTH_LOGIN
  • PLAIN
  • CRAM-MD5

AUTH LOGIN

The most common 'AUTH LOGIN' mechanism looks like this:

S: 220 esmtp.example.com ESMTP
C: ehlo client.example.com
S: 250-esmtp.example.com
S: 250-PIPELINING
S: 250-8BITMIME
S: 250-SIZE 255555555
S: 250 AUTH LOGIN PLAIN CRAM-MD5
C: auth login
S: 334 VXNlcm5hbWU6
C: avlsdkfj
S: 334 UGFzc3dvcmQ6
C: lkajsdfvlj
S: 535 authentication failed

From all the ESMTP Authentication mechanisms the offered, the client selects 'auth login'. The ESMTP server issues then a '334 VXNlcm5hbWU6' where 'VXNlcm5hbWU6' is a BASE64 encoded string 'Username:'. The client provides the BASE64 encoded user name and the sever responses with the request for the 'Password:' ('334 UGFzc3dvcmQ6'). In the sample above, random input is given and the server finally rejects the authentication request.

AUTH PLAIN

According to IANA's documentation, the PLAIN Authentication is defined in RFC 2245 "Anonymous SASL Mechanism". However, a more useful explanation of the PLAIN Authentication can be found in RFC 2595 "Using TLS with IMAP, POP3 and ACAP" (chapter 6): "The mechanism consists of a single message from the client to the server. The client sends the authorization identity (identity to login as), followed by a US-ASCII NULL character, followed by the authentication identity (identity whose password will be used), followed by a US-ASCII NULL character, followed by the clear-text password. The client may leave the authorization identity empty to indicate that it is the same as the authentication identity."

In other words, the correct form of the AUTH PLAIN value is 'authid\0userid\0passwd' where '\0' is the null byte.

Some ESMTP AUTH PLAIN implementations don't follow that procedure completely. We see that in the trace using Netscape's 4.8 MUA connecting to a modified Qmail 1.03 to do PLAIN authentication:

C: ehlo client.example.com
S: 220-esmtp.example.com
C: AUTH PLAIN dGVzdAB0ZXN0AHRlc3RwYXNz
S: 235 ok, go ahead (#2.0.0)
C: RCPT TO:<....>

In this sample, the user name was 'test' and the password 'testpass'. Here, the Netscape client immediately blasts the authentication information to the server (including the artificial authorization identity 'test') without waiting for the server to announce his SMTP Auth capabilites.

AUTH CRAM-MD5

While for AUTH PLAIN and LOGIN clear user names and password are transmitted, things go significantly more secure with the CRAM-MD5 authentication mechanism. As already mentioned in it's name, CRAM-MD5 combines a Challenge/Response mechanism to exchange information and a (cryptographic) Message Digest 5 algorithm to encrypt important information.

I use an example based on a posting of Markus Stumpf (SpaceNet) to the Qmail mailing list. A typical ESMTP AUTH CRAM-MD5 dialog starts like this:

S: 220 popmail.space.net ESMTP
C: ehlo client.example.com
S: 250-popmail.space.net
S: 250-PIPELINING
S: 250-8BITMIME
S: 250-SIZE 0
S: 250 AUTH CRAM-MD5
C: auth cram-md5
S: 334 PDI0NjA5LjEwNDc5MTQwNDZAcG9wbWFpbC5TcGFjZS5OZXQ+

Unlike AUTH LOGIN, the server's response is now a one-time BASE64 encoded 'challenge'. The challenge 'PDI0NjA5LjEwNDc5MTQwNDZAcG9wbWFpbC5TcGFjZS5OZXQ+' translates to '<24609.1047914046@popmail.Space.Net>'. The leading and trailing brackets ('<', '>') are mandatory, as well the portion of the challenge which provides the hostname after the '@'. '24609.1047914046' is a random string, typically build from the 'pid' and the current time stamp to make that challenge unique.

While the user name is transmitted in clear text (but of course BASE64 encoded), the servers's challenge is used by the client to generate a 'digest' from the challenge and the password (which is commonly called 'secret' or 'shared secret' in this context) employing the following MD5 hashing algorithm:

digest = MD5(('secret' XOR opad), MD5(('secret' XOR ipad), challenge))

If both the ESMTP server and the client 'share' the same challenge and secret, the user may now be authenticated successfully by means of the transmitted and BASE 64 encoded 'user name' and 'digest'.