On Tue, Nov 05, 2013 at 06:16:37PM -0500, Pavel Simerda wrote:
> Hi,
>
> I started a new network name resolver library project (extending the idea of glibc's nsswitch) that is using c-ares as its backend. I had the chance to design both the external API and the backend API from scratch. While I was first searching for inspiration in the c-ares API, I realized that `ares_fds()` style API is suboptimal (unless the mainloop is indeed based on `select()`).
>
> When designing the APIs, I considered the following well-known options:
>
> * select
> * poll
> * epoll
> * libevent
> * glib main loop
Maybe you could also include libverto. It's relatively new event loop
library that encapsulates other event loop approaches as back ends. It's
being used by MIT Kerberos and GSS-Proxy among others.
>
> All of them has more or less native support for adding and removing file descriptors but for example *epoll* doesn't have support for inspection. My APIs take this into account and so they are based on *addition*, *modification* and *removal* of file descriptors from the multiplex, all of them handled by one function with the following arguments:
>
> * driver: the epoll fd, resolver struct pointer, etc...
> * fd: the file descriptor
> * events: bit array of monitored events, non-zero for *add*/*modify*, zero for *delete*
>
> The main difference from `ares_fds()` is that for external API this has to be implemented using callbacks (or similar mechanism) bound directly to the places where the library needs to start or stop listening to file descriptor events. The supplied *callback* then handles addition/modification/removal of the file descriptors and you don't have to retrieve the whole file descriptor set before each `select()`, `poll()`, `epoll_wait()`, `g_main_context_iteration()` nor transform it into a data structure suitable for the respective main loop waiting method.
>
> An *epoll* example can be found in the source code of the library:
>
> 11 void
> 12 watch_fd(netresolve_t resolver, int fd, int events, void *user_data)
> 13 {
> 14 int epoll_fd = *(int *) user_data;
> 15
> 16 struct epoll_event event = { .events = events, .data = { .fd = fd} };
> 17
> 18 if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, &event) == -1 && errno != ENOENT) {
> 19 perror("epoll_ctl");
> 20 abort();
> 21 }
> 22 if (events && epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &event) == -1) {
> 23 perror("epoll_ctl");
> 24 abort();
> 25 }
> 26 }
>
> https://sourceware.org/git/?p=netresolve.git;a=blob;f=tests/test-async.c;h=fc146cd41a45ca744e382ab0258319c1a35d1881;hb=HEAD#l11
>
> On the other hand, the added complexity of `ares_fds()` approach can be easily seen in the DNS backend:
>
> 17 void
> 18 register_fds(Data *data)
> 19 {
> 20 netresolve_backend_t resolver = data->resolver;
> 21 fd_set rfds;
> 22 fd_set wfds;
> 23 int nfds, fd;
> 24
> 25 FD_ZERO(&rfds);
> 26 FD_ZERO(&wfds);
> 27 nfds = ares_fds(data->channel, &rfds, &wfds);
> 28
> 29 for (fd = 0; fd < nfds || fd < data->nfds; fd++) {
> 30 if (!FD_ISSET(fd, &rfds) && FD_ISSET(fd, &data->rfds)) {
> 31 FD_CLR(fd, &data->rfds);
> 32 netresolve_backend_watch_fd(resolver, fd, 0);
> 33 } else if (FD_ISSET(fd, &rfds) && !FD_ISSET(fd, &data->rfds)) {
> 34 FD_SET(fd, &data->rfds);
> 35 netresolve_backend_watch_fd(resolver, fd, POLLIN);
> 36 }
> 37 if (!FD_ISSET(fd, &wfds) && FD_ISSET(fd, &data->wfds)) {
> 38 FD_CLR(fd, &data->wfds);
> 39 netresolve_backend_watch_fd(resolver, fd, 0);
> 40 } else if (FD_ISSET(fd, &wfds) && !FD_ISSET(fd, &data->wfds)) {
> 41 FD_SET(fd, &data->wfds);
> 42 netresolve_backend_watch_fd(resolver, fd, POLLOUT);
> 43 }
> 44 }
> 45
> 46 data->nfds = nfds;
> 47
> 48 if (!nfds)
> 49 netresolve_backend_finished(resolver);
> 50 }
>
> https://sourceware.org/git/?p=netresolve.git;a=blob;f=backends/dns.c;h=7a006f497d0b24652285f9e3d5d55a463481c6ae;hb=HEAD#l17
>
> Also `ares_process_fd()` seems to assume `select()`.
>
> 110 void
> 111 dispatch(netresolve_backend_t resolver, int fd, int events)
> 112 {
> 113 Data *data = netresolve_backend_get_data(resolver);
> 114
> 115 int rfd = events & POLLIN ? fd : ARES_SOCKET_BAD;
> 116 int wfd = events & POLLOUT ? fd : ARES_SOCKET_BAD;
> 117
> 124 ares_process_fd(data->channel, rfd, wfd);
> 125 register_fds(data);
> 126 }
>
> https://sourceware.org/git/?p=netresolve.git;a=blob;f=backends/dns.c;h=7a006f497d0b24652285f9e3d5d55a463481c6ae;hb=HEAD#l110
>
> Would you consider an additional API as an alternative to `ares_fds()`, `ares_timeout()` and `ares_process_fd()` consisting of functions similar to the following ones useful?
>
> * void ares_set_fd_callback(ares_channel channel, int (*callback)(int fd, int events, void *user_data), void *user_data)
> * void ares_set_timeout_callbacks(ares_channel channel, int (*create)(int seconds, void *user_data), int (*remove)(int timeout_id), void *user_data)
> * int ares_dispatch_fd(ares_channel channel, int fd, int events)
From purely API perspective I think this makes sense..
>
> Apart from possible improvements of *c-ares*, I will be happy for any feedback for the API I'm using in the library. It's in very early stage of development and needs more eyes than just the two of mine.
>
> Cheers,
>
> Pavel
Received on 2013-11-06