Latest Tweets

It’s so dark, that it is not even implemented (yet) Part III

The DIME standard and its tool

After fixing the signet and genrec tools, I had a valid signet for my test domain “lud.org” and the DIME management record correctly set up in my dnssec authoritative server (see Part I and Part II of this series). I tried it using dig:

dig +dnssec _dx.lud.org. @localhost txt|grep TXT
;_dx.lud.org. IN TXT
_dx.lud.org. 120 IN TXT “ver=1 pok=E7gyvx3E6ksBVkg9CD5XBoXX18txj45iFSqtn9NLqjA dx=dmtp.lud.org tls=c2jM4G+EFZROQYNOyvwVSiQhgL5QW3UJN3CaIipR/Z4hjoSZoO72UlGXdKsAl1T1RQh+/h9rETD1+vaPbkIGCg”
_dx.lud.org. 120 IN RRSIG TXT 5 3 120 20170630110908 20170531110908 10287 lud.org. f8+HE5fw6cSsEwI1CznT2CUoJsIE57Bb/PsoLM4eNlkaIMoWtsLrh8sL EJ2GG2UtoJihLLEXLn+cmEFP7HT9971qd309et48oZCvwBfki0MG0HLy 9rEoG0XgrWODjBU5BKQcSC/dqOogiqTul55TjnCTGBNydYCklolcCQzK Wprsoa2qiBcrW8GFOMDKeXDgx6W7nZiaiYQs5n4aCAHtbXDODz/c89qi FI/5DAFvw/weVwAeRjqNBed4AsCp3UVSu+M4arqItrMagqb53G9OORH/ g4+olkIxNKw0Wqvcez6yHiZyETj0zChiM3zOwk3nrkAw+jrp6Yvztzon Xf/qjQ==

So next step would be to use the dime tool in dev/tools/dime in order to test some addresses. Of course, it would not be that easy!

dime

I built this tool and executed it in order to verify a particular user signet using the Full fingerprint of “lud.org” organizational signet. This came out:

../../dime -f ‘UlVB9t7+GcttbVzhDgdHqyZWFWRZQOfFXW8HQIoZX+Rk6JkuuizQzqmyRriQ1pU667FnhSzODVT9tPugwQjvUg’ “magma@lud.org”
Querying DIME management record for: lud.org
Establishing connection to DX server…
Error: could not connect to DX server.
[0]: src/providers/dime/signet-resolver/dmtp.c:178 [_sgnt_resolv_dmtp_connect()]: 4 (an unspecified error has occurred), errno = 0, aux = “could not establish DMTP connection to host: DIME management record DNSSEC signature was invalid”

So I re-ran the command using the “-v -v -v” flags to add some verbosity:

../../dime -f ‘UlVB9t7+GcttbVzhDgdHqyZWFWRZQOfFXW8HQIoZX+Rk6JkuuizQzqmyRriQ1pU667FnhSzODVT9tPugwQjvUg’ “magma@lud.org” -v -v -v -n

— Started parsing DIME management record…
— — VERSION: — [1]
— — PUBLIC KEY: — [E7gyvx3E6ksBVkg9CD5XBoXX18txj45iFSqtn9NLqjA]
— — DX: — [192.168.56.100]

….
— DIME management record for: — hashed —
—— version : 1
—— pok : 13b832bf1dc4ea4b0156483d083e570685d7d7cb718f8e62152aad9fd34baa30 [1]
—— tlssig : [not present]
—— policy : experimental
—— syndicates: [not present]
—— dx : 192.168.56.100 [1]
—— expiry : [not present]
—— subdomain : strict
****** This record was retrieved with an INVALID DNSSEC signature.

According to the previous output, the DIME management record was successfully read but its signature (RRSIG) was invalid.

Again, going through the sources I ended up here:

int _load_dnskey_file(const char *filename) {
 
// The ttl is unlimited and we don't want to save this entry to the cache.
if (!(dk = _add_dnskey_entry_rsa(dname, flags, algorithm, pubkey, keytag, rdata, rdlen, 0, 0, 1))) 
    fclose(fp);
    RET_ERROR_INT(ERR_UNSPEC, "unable to import DNS root key entry");
}
 
// Any key from a local file is automatically validated.
dk-->validated = 1;

In my case, the “root-anchor.key” file located in /root/.dime/ already had the “.” and “lud.org” DNSKEYS. According to the previous code, any DNSKEY in that file would be automatically validated. Clearly, this was not the case. The memcached daemon was working fine, so I though that maybe the issue should reside in the way the object was constructed and then added to the cache. So I modified the  _add_dnskey_entry_rsa function so that, inside it, the key was already validated before adding the object to the cache:

dnskey->pubkey = pubkey;
dnskey->keytag = keytag;
dnskey->do_cache = do_cache;
 
//TCG:
dnskey->validated = 1;

Once this change was made, I re-built the dime tool and tried again:

++++++ This record WAS retrieved with a valid DNSSEC signature.
Establishing connection to DX server…
— Returning cached DIME record.
– Attempting DMTP connection to DIME record-supplied DX server #1 at dmtp.lud.org:26 …
—- Initialized openssl library.
—- Initialized SSL context with cipher list: ECDHE-RSA-AES256-GCM-SHA384
— Established TCP connection (IPV4) to dmtp.lud.org:26.
– Attempting validation in x509 certificate chain: localhost.localdomain (level 0); verified = no / 18
– Attempting validation in x509 certificate chain: localhost.localdomain (level 0); verified = yes / 18
— Successfully established TLS connection to dmtp.lud.org:26.

So far so good; but after passing the signature validation of the RRSIG field, the tool blocked right after establishing a valid TLS connection to the DMTP Magma server. Again, reading the sources I discovered where the new issue resided:

if (!(banner = _sgnt_resolv_read_dmtp_line(session, NULL, &bcode, 0))) {
        RET_ERROR_INT(ERR_UNSPEC, "unable to read DMTP banner");
}
while (nleft && (!(lbreak = strstr((char *)session->_inbuf, "\r\n")))) {
 ....
}

Dime was expecting the server’s banner; inside _sgnt_resolv_read_dmtp_line the freezing happened because the tool was expecting “\r\n” as the end-of-line, but the server was returning only “\n” (this would be clearly demonstrated by looking inside the commands.c file in the srv/servers/dmtp directory later on; keep reading) So the loop never ended, and the whole dime tool blocked. I altered the line like this:

while (nleft && (!(lbreak = strstr((char *)session->_inbuf, "\n")))) {

This time it worked:

– Continuing verification of self-signed DX TLS certificate …
– DX TLS certificate matched DIME record signature.
– DX TLS certificate verification succeeded automatically (TLS cert match + dnssec).
– DX certificate successfully verified.
– Attempting to verify fingerprint (UlVB9t7+GcttbVzhDgdHqyZWFWRZQOfFXW8HQIoZX+Rk6JkuuizQzqmyRriQ1pU667FnhSzODVT9tPugwQjvUg) for signet: magma@lud.org
Error: signet verification failed.

The connection to the DMTP server was fine this time, but the signet could not be verified. That was odd, because I made sure to have that user signet already installed in the database. So I tried the connection to the DMTP server myself using the openssl connect command:

openssl s_client -crlf -connect 192.168.56.100:26
CONNECTED(00000003)


220 lud.org DSMTP Magma

Then, I tried the EHLO command and the VRFY command to no avail:

EHLO HOST=dmtp.lud.org
250 lud.org
VRFY DOMAIN=lud.org FINGERPRINT=”UlVB9t7+GcttbVzhDgdHqyZWFWRZQOfFXW8HQIoZX+Rk6JkuuizQzqmyRriQ1pU667FnhSzODVT9tPugwQjvUg”
250 VRFY COMMAND COMPLETE

I tried some other commands, according to the specs document reference to no avail either. The only ones working were: RST, NOOP, and QUIT. So I went, once again, through the sources and I found simple function placeholders for the DMTP commands:

**
 * @brief       Specify the destination domain for a message in response to an DMTP RCPT command.
 * @param       con             the DMTP client connection issuing the command.
 * @return      This function returns no value.
 */
void dmtp_rcpt(connection_t *con) {
        con_write_bl(con, "250 RCPT COMMAND COMPLETE\n", 26);
        return;
}
 
**
 * @brief       Specify the origin domain for a message in response to an DMTP MAIL command.
 * @param       con             the DMTP client connection issuing the command.
 * @return      This function returns no value.
 */
void dmtp_mail(connection_t *con) {
        // Spit back the all clear.
        con_write_bl(con, "250 MAIL COMMAND COMPLETE\n", 26);
        return;
}
 
/**
 * @brief       Process an DMTP MAIL command.
 * @param       con             the DMTP client connection issuing the command.
 * @return      This function returns no value.
 */
void dmtp_data(connection_t *con) {
        con_write_bl(con, "451 DATA FAILED - INTERNAL SERVER ERROR - PLEASE TRY AGAIN LATER\n", 65);
        dmtp_requeue(con);
        return;
}

So, basically, the DMTP server is not functional at all yet. This surprised me a lot, because there’s a talk at Defcon in 2014 where Ladar shown his MAGMA server capabilities (there’s even a video showing the direct communication and sending of DMTP commands via Telnet). If you are as mystified as I am, go watch it: https://www.youtube.com/watch?v=TWzvXaxR6us (minute 40:00).

So maybe they have been tuning things a bit, or they have decided to start afresh, or whatever. I opened a new issue on @github but they have deleted it. So, well, as far as I’m concerned, DIME and MAGMA are far from being something to test and deploy. Hopefully, this great idea would come to fruition, eventually. In the meantime, I suggest to keep an eye on this project.

It’s so dark, that it is not even implemented (yet) Part II

The DIME management record

According to the specs document, the DIME standard is based on “the DIME management record”. This record, set as a new sort of TXT record in a DNS domain server, will allow other DMTP-capable servers to communicate securely and privately with no information leakage. Its the cornerstone of the standard, so to speak. This record, of course, has only one mandatory field (POK), and a bunch of optional ones. The Public Organisational Key (POK for short) can be easily obtained by means of using the signet tool, as described in my previous POST. However, and according to the specs document, an additional TLS field must be provided in order to secure the channel. When using dnssec, it is possible to have a self-signed server certificate, but it is mandatory to provide the TLS field in the DIME management record; otherwise there is not going to be any valid TLS communication using the DMTP (DSMTP in this case) protocol.

Inside the dev/tools directory there is a tool called “genrec”. This is the tool you are supposed to use in order to generate this record for your DNS authoritative server. Well, as expected it did not work.

genrec

I had my new signet and my MAGMA server set up accordingly; next step I guessed was to generate my DIME management record. So I built the genrec tool and executed it:

../../genrec -k lud.org.keys -c tls.localhost.localdomain.pem
Error: could not read ed25519 POK from keyfile.

Of course I should have generated a new TLS file (this file has the certificate and the private key), because my test domain was “lud.org”. However, this had nothing to do with the issue at hand. Again, I delve into the source code and I found this:

if (!(pemdata = _read_pem_data(filename, "ED25519 PRIVATE KEY", 1))) {
    RET_ERROR_PTR(ERR_UNSPEC, "unable to read ed25519 private key data from PEM file");
}

So, genrec was looking for a file with the tags “—–BEGIN ED25519 PRIVATE KEY—–” and “—–END ED25519 PRIVATE KEY—–“. Guess what? The private keys file that was generated using the signet tool had these ones instead: “—–BEGIN ORGANIZATIONAL SIGNET—–” and “—–END ORGANIZATIONAL SIGNET—–“. This is a clear example of forgotten left-overs!

So because I was a bit impatient, I made a copy of my lud.org.key file and then I changed the tags accordingly. Again, genrec complaint about not being able to read the key. So more code-delving! A bit later, I ended up here:

if (!(keys_bin = keys_file_serialize(filename, &keys_len))) {
    RET_ERROR_PTR(ERR_UNSPEC, "could not retrieve keys binary string");
}
key = keys_signkey_from_binary(keys_bin, keys_len);

The previous code snippet was reading a total of 73 bytes! Obviously, a valid key is 32-byte long (after being base64-decoded, of course). So I had a quick look at the way the signet tool was generating and reading the keys. Loaded with this useful information, I set about to modify the function _load_ed25519_privkey, the one performing the loading of the private keys file for genrec (the commented-out lines are mine):

ED25519_KEY * _load_ed25519_privkey(char const *filename) {
        ED25519_KEY *result;
        //unsigned char *keydata;
        //char *pemdata;
        //size_t klen;
 
        if (!filename) {
                RET_ERROR_PTR(ERR_BAD_PARAM, NULL);
        }
 
        /*if (!(pemdata = _read_pem_data(filename, "ED25519 PRIVATE KEY", 1))) {
                RET_ERROR_PTR(ERR_UNSPEC, "unable to read ed25519 private key data from PEM file");
        }*/
 
        // TCG: proceed as the signet code; first read from the key file:
        if (!(result = dime_keys_signkey_fetch(filename))) {
                RET_ERROR_PTR(ERR_UNSPEC, "unable to read ed25519 private key data from PEM file");
        }
        //TCG:
/*      keydata = _b64decode(pemdata, strlen(pemdata), &klen);*/
//      _secure_wipe(pemdata, strlen(pemdata));
//      free(pemdata);
 
//      printf("Keylength: %zu\n", klen);
 
        /* Okay, something is really wrong here. The klen is 73 bytes!! */
 
//      if (!keydata /**/|| (klen != ED25519_KEY_SIZE)/**/) {
        if (!result) {
 
/*              if (keydata) {
                        _secure_wipe(keydata, klen);
                        free(keydata);
                }*/
 
                RET_ERROR_PTR(ERR_UNSPEC, "bad ED25519 key data was read from file");
        }
 
/*      if (!(result = malloc(sizeof(ED25519_KEY)))) {
                PUSH_ERROR_SYSCALL("malloc");
                _secure_wipe(keydata, klen);
                free(keydata);
                RET_ERROR_PTR(ERR_NOMEM, "unable to allocate space for ED25519 key");
        }
 
        memset(result, 0, sizeof(ED25519_KEY));
        memcpy(result->private_key, keydata, sizeof(result->private_key));
        _secure_wipe(keydata, klen);
        free(keydata);
        ed25519_publickey_donna(result->private_key, result->public_key);*/
 
        return result;
}

After this change, I re-built the genrec tool and tried again:

../../genrec -k lud.org.key -c tls.localhost.localdomain.pem
ver=1 pok=E7gyvx3E6ksBVkg9CD5XBoXX18txj45iFSqtn9NLqjA tls=c2jM4G+EFZROQYNOyvwVSiQhgL5QW3UJN3CaIipR/Z4hjoSZoO72UlGXdKsAl1T1RQh+/h9rETD1+vaPbkIGCg pol=experimental sub=strict

This time it did work; I made sure the POK value was the same by calling signet: they matched up. So I added the DIME management record to my zone file and re-signed the entire zone:

_dx.lud.org. 120 IN TXT “ver=1 pok=E7gyvx3E6ksBVkg9CD5XBoXX18txj45iFSqtn9NLqjA dx=dmtp.lud.org tls=c2jM4G+EFZROQYNOyvwVSiQhgL5QW3UJN3CaIipR/Z4hjoSZoO72UlGXdKsAl1T1RQh+/h9rETD1+vaPbkIGCg”

dnssec-signzone -l dlv.isc.org -o lud.org -k Klud.org.+005+55903.key db.lud.org Klud.org.+005+10287.key

So far so good. But there’s still more to come! See you guys in Part III!

It’s so dark, that it is not even implemented (yet) Part I

DIME and MAGMA

DIME has been publicized a lot since last January. I’ve been reading the specs document (from 2015) and I have cloned the git repository in order to compile and test MAGMA, their open source server. This server, at least in theory, implements SMTP, IMAP, HTTP, MOLTEN, DMTP and DMAP. But of course, according to github:

The magma server daemon, is an encrypted email system with support for SMTP, POP, IMAP, HTTP and MOLTEN,. Additional support for DMTP and DMAP is currently in active development.

Anyway, I thought it would be interesting to test it because, although I suspected that maybe some parts would not be completely functional, the best part of the MAGMA server would. Otherwise, they would not be saying this (from their website: https://lavabit.com/explain-lavabit.html):

What is Magma?

Magma is Lavabit’s open source, commercial-grade, and full-featured server ready for use with the Dark Internet Mail Environment. Magma is now ready for commercial implementation and will fundamentally change the way business transmits encrypted data. Whether you are an individual, SME, or corporate enterprise wanting your own DIME compatible server, the Lavabit technical team can assist with your implementation and development needs.

But no; unluckily for us all, MAGMA seems far from being “full-featured server ready for use with the Dark Internet Mail Environment”. This is the first of a series about my impressions on MAGMA and the DIME standard. Keep reading!

First thing that do not seem to work: its tools.

Signet

After being able to compile and run MAGMA, I wanted to delve into the concept of “signets” and the “DIME management record”. I read the specs document, dated from 2015, and after that I tried the “signet” tool under dev/tools. This tool can generate new signets,  sign the SSR files (sort of a CSR file) and dump some information about signets. Of course, it worked out-of-the-box, at least apparently. This is me generating a new organizational signet for my MAGMA test environment:

../../signet -g lud.org
Organization name: LUD.ORG
Organization address: ADDRESS
Organization country: COUNTRY
Organization postal code: POSTAL_CODE
Organization phone number: PHONE_NUMBER

After running the previous command, I had two files: the signet itself and the private keys file:

ls -l lud*
-rw——- 1 root root 176 mai 26 23:33 lud.org.key
-rw-r–r– 1 root root 531 mai 26 23:34 lud.org.signet

Now some basic stuff, like getting some information from this new signet:

../../signet -d lud.org.signet
— version: 1, size = 333, signet type = organizational
— 1 Primary-Organizational-Key -> QO3pMGOqP51Wz/Q8HRgLP4U0eIhYC+29kT0fDaSyX36B

Obviously, you don’t want to have the dime.localhost.localdomain.signet in your MAGMA testing environment. You want to test your own signets. So replacing these two server configuration lines with the ones pointing to my new signet and its private keys file counterpart only seemed natural at the time:

magma.dime.key = sandbox/etc/lud.org.key
magma.dime.signet = sandbox/etc/lud.org.signet

Then, I re-started the MAGMA server and the first issue arose:

Unable to parse the PRIME organizational signing key.
Unable to initialize the privacy respecting internet mail environment. Exiting.
magma.init != shutdown {23 != 39}
Magma shutdown complete.

Of course, I was puzzled. This change seemed trivial at the time. So I looked for issues in the code, using the “Unable to parse the PRIME organizational signing key” as the starting point. Soon I realized that the main problem was that the MAGMA developers do not seem to follow their own specs document so close. With the signet tool,  the POK for this new organizational signet was 44-byte long, base64-encoded:

QO3pMGOqP51Wz/Q8HRgLP4U0eIhYC+29kT0fDaSyX36B

This is so in the specs document. On the other hand, the already provided dime.localhost.localdomain.signet POK is 43-byte long:

../../signet -d dime.localhost.localdomain.signet
— version: 1, size = 139, signet type = organizational
Primary-Organizational-Key -> h44kA4AXy2gFnlF+osucG2t4SOBeu8CA3WWxo6RLfIA

WTF? Following the code path, I ended up right here:

 case SIGNKEY_DEFAULT_FORMAT:
                serial_size = ED25519_KEY_SIZE + 1;
 
                if (!(serial_key = malloc(serial_size))) {
                        PUSH_ERROR_SYSCALL("malloc");
                        RET_ERROR_PTR(ERR_NOMEM, "could not allocate memory for serialized signing key");
                }
 
                serial_key[0] = format;
                memcpy(serial_key + 1, key->public_key, ED25519_KEY_SIZE);
                break;

So the first byte of the key, that is, serial_ley[0] is 0x40 (SIGNKEY_DEFAULT_FORMAT). This is what makes the base64-encoded key 44-byte long. The magma server routines were expecting a 43-byte base64-encoded string instead! This was not what the specs document says at all! So I made this slight alteration:

//              serial_key[0] = format;
//              memcpy(serial_key + 1, key->public_key, ED25519_KEY_SIZE);
                memcpy(serial_key, key->public_key, ED25519_KEY_SIZE);

After that, I re-built the signet tool:

make signet
Building src/providers/dime/signet/signet.c
Constructing signet

And I generated a new signet, this time I made sure its POK was 43-byte long (base64-encoded):

../../signet -d lud.org.signet
— version: 1, size = 287, signet type = organizational
— 1 Primary-Organizational-Key -> E7gyvx3E6ksBVkg9CD5XBoXX18txj45iFSqtn9NLqjA

This time, the MAGMA server ran with no issues at all:

00:21:05 – src/engine/context/process.c – process_start() – 343] = Magma initialization complete.

Signing SSR files

Of course, this small alteration broke the signing of SSR files:

../../signet -s “test@lud.org” -r test-lud.org.ssr -k lud.org.key
The signet is not a valid SSR.

After going through the code once again, I ended up here:

       //switch (serial_key[0]) {
 
        //case SIGNKEY_DEFAULT_FORMAT:
 
                // This not applies here anymore!
/*              if (key_size != ED25519_KEY_SIZE + 1) {
                        PUSH_ERROR(ERR_UNSPEC, "invalid signet signing key size or format");
                        key = NULL;
                        break;
                }*/
 
                //key = _deserialize_ed25519_pubkey(serial_key + 1);
                key = _deserialize_ed25519_pubkey(serial_key);
                //break;
        /*default:
                key = NULL;
                PUSH_ERROR_FMT(ERR_UNSPEC, "unsupported format specifier for signing key: %u", serial_key[0]);
                break;
 
        }*/
 
        if (!key) {
                RET_ERROR_PTR(ERR_UNSPEC, "could not convert signet serial format key to object");
        }
 
        return key;
}

The lines commented out are mine, of course. With this yet another small alteration, and after re-bulding the signet tool, now the singing of SSR files are working and the MAGMA server runs and everything looks great …. right?

../../signet -s “test@lud.org” -r test-lud.org.ssr -k lud.org.key
User name: Test MAGMA
User address: ADDRESS
User country: COUNTRY
User postal code: POSTAL_CODE
User phone number: PHONE_NUMBER

ls -lt test*
-rw-r–r– 1 root root 609 mai 27 00:50 test-lud.org.signet
-rw-r–r– 1 root root 156 mai 27 00:24 test-lud.org.key
-rw-r–r– 1 root root 271 mai 27 00:24 test-lud.org.ssr

Not quite. More on MAGMA soon! In the meantime, if you happen to be a MAGMA developer, I emailed support@lavabit.com and I have opened a new issue on github about this, so please let me know any improvements about these issues if possible, that would be great!