Running a SMTP email server (e.g., exim) in Gramine #1013
Replies: 7 comments 40 replies
-
|
I've never used |
Beta Was this translation helpful? Give feedback.
-
|
At first, @boryspoplawski, thanks for your answer! However, I could not follow your thoughts completely. What do you mean by virtual and system users, especially in the context of Gramine? Besides, what is the Maybe I give it another try, what I think the problem of running exim in Gramine is and how it is affected especially with the user configuration:
So, to make exim run as fluently as possible, we must somehow reproduce the user environment of a normal Linux installation inside Gramine. Normally, Gramine uses the root user. If we apply this, exim gets into a bunch of trouble, which we can partly accommodate by uncommenting some lines of code, but this is not very elegant and did not lead to a fully working environment yet. One interesting problem originates, that the root user does not write every output to the log files; instead, exim starts a new process with the second non-root user to accomplish this (more details here: https://github.com/Exim/exim/blob/master/src/src/log.c#L393). Especially, we aren’t allowed to configure the exim user as root, otherwise, the compilation fails. We can also reconfigure the UID and GID to the currently active user. However, this didn't seem to work out properly when taking the UID and GID of the host system (practically speaking, printing those values when running exim directly and copying them into the Gramine manifest). Exim then claims that it doesn’t know this user on startup, more exactly: So the question is if there is a way to simulate the two users (root and the exim user) within Gramine. (And if that’s not possible: I’m also open to other SMTP mail server suggestions that support TLS and password authentification. ;)) |
Beta Was this translation helpful? Give feedback.
-
It's not in the context of Gramine, but exim, which I assume uses system users when delivering/sending emails, i.e. uses
I'm not familiar with |
Beta Was this translation helpful? Give feedback.
-
|
Ok, thanks, I'll investigate this further. |
Beta Was this translation helpful? Give feedback.
-
@gabriel2029 I would definitely follow the approach of non-root users. This particular message probably stems from the fact that you didn't specify some user-related files in the Gramine manifest, like If you'll find out that e.g.
Ideally, you probably don't want to pass-through your computer's |
Beta Was this translation helpful? Give feedback.
-
There is no |
Beta Was this translation helpful? Give feedback.
-
|
First, huge thanks to all of you, I now managed to successfully send an email. For the library problem I temporarily added some mounts in the manifest: fs.mounts = [
{ path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
{ path = "{{ arch_libdir }}", uri = "file:{{ arch_libdir }}" },
{ path = "/usr{{ arch_libdir }}", uri = "file:/usr{{ arch_libdir }}" },
# TODO: Only hacky workaround (for problematic library management in exim), find better solutions
{ path = "/usr/local/lib/libcrypt.so.1", uri = "file:/lib/x86_64-linux-gnu/libcrypt.so.1" },
{ path = "/usr/local/lib/libresolv.so.2", uri = "file:{{ gramine.runtimedir() }}/libresolv.so.2" },
{ path = "/usr/local/lib/libm.so.6", uri = "file:/lib/x86_64-linux-gnu/libm.so.6" },
{ path = "/usr/local/lib/libdb-5.3.so", uri = "file:/lib/x86_64-linux-gnu/libdb-5.3.so" },
{ path = "/usr/local/lib/libssl.so.1.1", uri = "file:/lib/x86_64-linux-gnu/libssl.so.1.1" },
{ path = "/usr/local/lib/libcrypto.so.1.1", uri = "file:/lib/x86_64-linux-gnu/libcrypto.so.1.1" },
{ path = "/usr/local/lib/libpcre2-8.so.0", uri = "file:/lib/x86_64-linux-gnu/libpcre2-8.so.0" },
{ path = "/usr/local/lib/libc.so.6", uri = "file:{{ gramine.runtimedir() }}/libc.so.6" },
{ path = "/usr/local/lib/libpthread.so.0", uri = "file:{{ gramine.runtimedir() }}/libpthread.so.0" },
{ path = "/usr/local/lib/libdl.so.2", uri = "file:{{ gramine.runtimedir() }}/libdl.so.2" },
# End of hacky workaround
{ path = "/etc", uri = "file:/etc" },
[...]Since this is a very dirty solution, I look at how to configure exim to set a different However, the email ended up in the spam folder. I think this is because the FQDN of the mail server is wrong inside Gramine, more exactly the hello message is (the localhost should be substituted with the actual hostname): Do you have a rough idea why the hostname is |
Beta Was this translation helpful? Give feedback.
-
|
Gramine did not forward |
Beta Was this translation helpful? Give feedback.
-
|
Only as an update: In Exim, there is the option to configure which environment variables to keep via a configuration flag For the |
Beta Was this translation helpful? Give feedback.
-
|
// Update: Gramine 1.3 fixed the problem with the |
Beta Was this translation helpful? Give feedback.
-
|
Would you like to try Postfix instead? Postfix is split between many daemons and it will easy to run only some of them in gramine. (I've never tried it though). |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, I'll have a look at postfix. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, apart from simply sending emails, I also wanted to receive E-Mails with Exim. The rough concept is working, however, I encountered some issues:
Best regards, |
Beta Was this translation helpful? Give feedback.
-
Unfortunately, multi-processing is complicated currently with gramine/libos/src/fs/tmpfs/fs.c Lines 8 to 10 in 72f6524 So only if you have a But this is currently an implementation detail. Ideally, we'll implement copying of even not-currently-opened files (but you can imagine that then we pollute enclave space with possibly-never-used-later files). And really ideally, we'll implement the synchronization of contents of
To me, it's still unclear what Exim does internally: does Exim spawn a new process for each new email? If yes, then you have a multi-process environment that manipulates the same file, which is a mine field in Gramine... Unless the application (Exim) does very specific locking on this file so that all processes wait for each other and don't overwrite each other. |
Beta Was this translation helpful? Give feedback.
-
|
To
To my understanding, Exim uses the |
Beta Was this translation helpful? Give feedback.
-
I see. This is a good use-case. So the "copy-file-contents-to-child" would work fine in your case (i.e., you don't need any synchronization between parent and children). This could be definitely implemented in Gramine. I think it will be even not too hard to do it. However, we didn't allocate any resources on tasks like this. Maybe you'd be willing to submit a PR for Gramine? :)
Something is wrong. This is surprising to me. Maybe you could create a small reproducer for this problem (even e.g. in Python, for easier prototyping)? |
Beta Was this translation helpful? Give feedback.
-
Why not simply use encrypted files? |
Beta Was this translation helpful? Give feedback.
-
I also thought about this, but in my opinion, |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, At first, I had no major problems letting it run inside Gramine by following the Rootless Dovecot Wiki (cmp. https://wiki.dovecot.org/HowTo/Rootless). However, it seems to me that Unix sockets are not shared between multiple processes. To be exact I see in the log (here as an example with the Unix socket file Later, in other child processes: So my impression is that the Unix socket is opened successfully, but cannot be accessed by the child process. For the mounting of the path, I tried different options (e.g., normal file systems, If Unix sockets are indeed not feasible in the way I want to use them, what are possible workarounds, e.g., configurations I could search in Dovecot or using another IMAP server? Apart from that, some system calls are unimplemented, however, they (currently) don't seem to do a large problem: Best regards, |
Beta Was this translation helpful? Give feedback.
-
|
@gabriel2029 that would definitely be a good idea, to check whether this is the only issue |
Beta Was this translation helpful? Give feedback.
-
|
Hello, after I while I had again a look at the project and I adapted a small test program to test the Unix sockets. However, this didn't work inside Gramine and the connection resulted in: Since I have currently no idea, where this error might originate, I share a larger fraction of the code and manifest file with you: Code/* Code cloned and adapted from: https://users.cs.cf.ac.uk/Dave.Marshall/C/node28.html */
void client(void);
int main(void) {
char c;
FILE *fp;
int fromlen;
register int i, s, ns, len;
struct sockaddr_un saun, fsaun;
if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
perror("server: socket");
exit(1);
}
saun.sun_family = AF_UNIX;
strcpy(saun.sun_path, ADDRESS);
unlink(ADDRESS);
len = sizeof(saun.sun_family) + strlen(saun.sun_path);
if (bind(s, (struct sockaddr *) &saun, len) < 0) {
perror("server: bind");
exit(1);
}
if (listen(s, 5) < 0) {
perror("server: listen");
exit(1);
}
int ret = fork();
if (ret < 0) {
perror("fork");
exit(1);
} else if (ret == 0) {
close(s);
client();
exit(0);
}
if ((ns = accept(s, (struct sockaddr *) &fsaun, (socklen_t*) &fromlen)) < 0) {
perror("server: accept");
exit(1);
}
fp = fdopen(ns, "r");
// [...]
exit(0);
}
void client() {
char c;
FILE *fp;
register int i, s, len;
struct sockaddr_un saun;
if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
perror("client: socket");
exit(1);
}
saun.sun_family = AF_UNIX;
strcpy(saun.sun_path, ADDRESS);
len = sizeof(saun.sun_family) + strlen(saun.sun_path);
if (connect(s, (struct sockaddr *) &saun, len) < 0) {
perror("client: connect");
exit(1);
}
fp = fdopen(s, "r");
// [...]
exit(0);
}Manifest fileloader.preload = "file:{{ gramine.libos }}"
loader.entrypoint = "file:{{ gramine.libos }}"
libos.entrypoint = "unix-socket"
loader.env.RA_TLS_CLIENT_INSIDE_SGX = "1"
loader.pal_internal_mem_size = "512M"
loader.log_level = "debug"
# arguments.txt file is created by start script
loader.argv_src_file = "file:arguments.txt"
loader.env.LD_LIBRARY_PATH = "/lib:{{ arch_libdir }}:/usr{{ arch_libdir }}"
# Define user for unix-socket test program
loader.uid = 25567
loader.gid = 1201
fs.mounts = [
{ path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
{ path = "{{ arch_libdir }}", uri = "file:{{ arch_libdir }}" },
{ path = "/usr{{ arch_libdir }}", uri = "file:/usr{{ arch_libdir }}" },
{ path = "/etc", uri = "file:/etc" },
]
sgx.enclave_size = "8192M"
sys.stack.size = "1M"
sgx.remote_attestation = "epid"
sgx.debug = true
sgx.ra_client_spid = "{{ ra_client_spid }}"
sgx.ra_client_linkable = {{ 'true' if ra_client_linkable == '1' else 'false' }}
sgx.trusted_files = [
"file:{{ gramine.libos }}",
"file:unix-socket",
"file:arguments.txt",
"file:/lib/x86_64-linux-gnu/",
"file:{{ gramine.runtimedir() }}/",
]
sgx.allowed_files = [
] |
Beta Was this translation helpful? Give feedback.
-
|
Your program is buggy and does not initialize |
Beta Was this translation helpful? Give feedback.
-
|
Hey Borys, the missing initialization from Therefore I wanted to confirm that the Unix sockets are indeed working (which they are). The problem must therefore be lying somewhere else ... |
Beta Was this translation helpful? Give feedback.
-
|
// Update: I've now figured one problem out with the sockets: Dovecot created the socket for one service at |
Beta Was this translation helpful? Give feedback.
-
|
Hi, the problems with the Unix sockets have to seem resolved now when changing to absolute paths. However, as I currently noticed, Dovecot shares file descriptors between processes via |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately, Gramine doesn't implement this functionality yet... I tried to implement Hopefully this year (2023) we'll have the time/resources to implement @gabriel2029 Do you know what exact FDs your Dovecot workload sends between processes? Are these socket FDs? |
Beta Was this translation helpful? Give feedback.
-
I think that these are socket FDs: there are multiple processes in Dovecot responsible for different tasks, e.g., one for the IMAP login and one other for the other communication. After login has finished, one process sends the file descriptors for communication to another process. As code example see here: https://github.com/dovecot/core/blob/main/src/imap-urlauth/imap-urlauth-client.c#L162 (definition of the client in https://github.com/dovecot/core/blob/main/src/imap-urlauth/imap-urlauth-client.h#L16)
Only for interest: do you still have some code fragments for this implementation (maybe I try to implement it myself as a workaround)? |
Beta Was this translation helpful? Give feedback.
-
|
@gabriel2029 Here you go: gramineproject/graphene#1511 Be aware, Gramine code was changed like 99% from those times... |
Beta Was this translation helpful? Give feedback.
-
|
@dimakuv Thanks for your answer. But I think the shared FDs are TCP sockets, as the following test code i_info("fd_send: handle: %d, file descriptor number: %d", handle, send_fd);
struct stat statbuf;
fstat(send_fd, &statbuf);
if (S_ISSOCK(statbuf.st_mode)) {
struct sockaddr_in sin;
socklen_t len = sizeof(sin);
// TODO: try to get rid of warning regarding strict-aliasing rules
if (getsockname(send_fd, (struct sockaddr *) &sin, &len) == -1) {
i_error("getsockname failed: %m");
return -1;
}
i_info("fd_send: address: %s, port number: %d\n", inet_ntoa(sin.sin_addr), ntohs(sin.sin_port));
} else {
// Not a socket
i_info("fd_send: not a socket");
}outputs me: 10143 is my alternative non-root IMAP connection port. I guess realizing sharing TCP sockets would be more tricky. |
Beta Was this translation helpful? Give feedback.
-
|
@gabriel2029 Yep, not that simple. In general, we would anyway require proper |
Beta Was this translation helpful? Give feedback.
-
Hello everyone,
I wanted to run the SMTP email server exim (https://www.exim.org/) inside Gramine. However, there seem to be some problems:
Here are some error messages that occurred:
getrlimit(RLIMIT_NOFILE) failed: Bad addressexim: getgroups() failed: Bad addressfailed to write to main log: length=65 result=-1 errno=9 (Bad file descriptor)[P1:T1:exim] warning: "struct msghdr" ancillary data is not supportedHow could I fix those problems, at best in a very generic way by, e.g., adjusting the Gramine manifest, and not by changing a lot of the actual code?
Apart from that, if that is not likely to make exim make work easily within Gramine: does anyone has experience in which SMTP server might be easily adapted to be run in Gramine and has the support of TLS as well as password authentication?
Best regards,
Gabriel
(Disclaimer: I already posted this question into the Google forum (https://groups.google.com/g/gramine-users/c/2UUgGqZ054U), but reposted it here as it was recommended there for me.)
Beta Was this translation helpful? Give feedback.
All reactions