BSD.lv tools for BCHS: kcgi, sqlbox, ort

kcgi – minimal CGI and FastCGI library for C/C++

kcgi is an open source CGI and FastCGI library for C/C++ web applications. It is minimal, secure, and auditable.

To start, install the library. Then read the deployment and usage guides. Use the GitHub tracker for questions or comments, or find contact information there for direct contact.

The following echoes Hello, World! as an HTTP response to a CGI request. 
#include <sys/types.h> /* size_t, ssize_t */
#include <stdarg.h> /* va_list */
#include <stddef.h> /* NULL */
#include <stdint.h> /* int64_t */
#include <kcgi.h>

int main(void) {
    struct kreq r;
    const char *page = "index";
    if (khttp_parse(&r, NULL, 0, &page, 1, 0) != KCGI_OK)
        return 1;
    khttp_head(&r, kresps[KRESP_STATUS], 
        "%s", khttps[KHTTP_200]);
    khttp_head(&r, kresps[KRESP_CONTENT_TYPE], 
        "%s", kmimetypes[KMIME_TEXT_PLAIN]);
    khttp_body(&r);
    khttp_puts(&r, "Hello, world!");
    khttp_free(&r);
    return 0;
}
#include <sys/types.h> /* size_t, ssize_t */
#include <stdarg.h> /* va_list */
#include <stddef.h> /* NULL */
#include <stdint.h> /* int64_t */
#include <kcgi.h>

int main(void) {
    struct kreq r;
    const char *page = "index";
  
    /*
     * Parse the HTTP environment.
     * We only know a single page, "index", which is also
     * the default page if none is supplied.
     * (We don't validate any input fields.)
     */
  
    if (khttp_parse(&r, NULL, 0, &page, 1, 0) != KCGI_OK)
        return 1;
  
    /* 
     * Ordinarily, here I'd switch on the method (OPTIONS, etc.,
     * defined in the method variable) then switch on which
     * page was requested (page variable).
     * But for brevity's sake, just output a response: HTTP 200.
     */
  
    khttp_head(&r, kresps[KRESP_STATUS], 
        "%s", khttps[KHTTP_200]);
  
    /* 
     * Show content-type unilaterally as text/plain.
     * This would usually be set from r.mime.
     */
  
    khttp_head(&r, kresps[KRESP_CONTENT_TYPE], 
        "%s", kmimetypes[KMIME_TEXT_PLAIN]);
  
    /* No more HTTP headers: start the HTTP document body. */
  
    khttp_body(&r);
    
    /*
     * We can put any content below here: JSON, HTML, etc.
     * Usually we'd switch on our MIME type.
     * However, we're just going to put the literal string as noted…
     */
  
    khttp_puts(&r, "Hello, world!");
  
    /* Flush the document and free resources. */
  
    khttp_free(&r);
    return 0;
}

For fuller examples, see sample.c (C and CGI), samplepp.cc (C++ and CGI), sample-fcgi.c (C and FastCGI), or or jump to the Documentation section.

kcgi supports many features: auto-compression, handling of all HTTP input operations (query strings, cookies, page bodies, multipart) with validation, authentication, configurable output caching, request debugging, and so on. Its strongest differentiating feature is using sandboxing and process separation for handling the untrusted input path.

current release

(release archive)

installation

First, check if kcgi isn't already packaged for your system, such as for OpenBSD, FreeBSD, Arch Linux, and so on. (If it is, make sure it's up to date!) If so, install using that system.

If not, you'll need a modern UNIX system. To date, kcgi has been built and run on GNU/Linux machines (musl and glibc), BSD (OpenBSD, NetBSD, FreeBSD), Solaris, OmniOS, and Mac OS X (only Mojave and newer!) on i386, amd64, powerpc, arm64, and sparc64. It has been deployed under Apache, nginx, and OpenBSD's httpd(8) (the latter two natively over FastCGI and via the slowcgi wrapper). The only hard dependency is BSD make (bmake on Linux). If you're running the regression tests (see Testing), you'll need libcurl.

Download kcgi.tgz and verify the archive with kcgi.tgz.sha512. Configure with ./configure, compile with make (or bmake on Linux systems). Finally, install the software using make install. Optionally override default paths with a configure.local file (see the configure script for details) prior to configuration.

If kcgi doesn't compile, please send me the config.log file and the output of the failed compilation. Along with all of your operating system information of course.

To run bleeding-edge code between releases, the CVS repository is mirrored on GitHub. Installation instructions tracking the repository version may be found on that page.

deployment

To compile kcgi applications, use the package configuration. Linking is similarly normative. 

% cc `pkg-config --cflags kcgi` -c yourprog.c
% cc yourprog.o `pkg-config --libs kcgi`

Well-deployed web servers, such as the default OpenBSD server, by default are deployed within a chroot(2). If this is the case, you'll need to statically link your binary. 

% cc -static yourprog.o `pkg-config --static --libs kcgi`

FastCGI applications may either be started directly by the web server (which is popular with Apache) or externally given a socket and kfcgi(8) (this method is normative for OpenBSD's httpd(8) and suggested for the security precautions taken by the wrapper).

documentation

The kcgi manpages, starting with kcgi(3), are the canonical source of documentation. The following is a list of all manpages:

If it's easier to start by example, you can use kcgi-framework as an initial boilerplate to start your project. The following are introductory materials to the system.

In addition to these resources, the following conference sessions have referenced kcgi.

And the following relate to extending standards:

implementation details

The bulk of kcgi's CGI handling lies in khttp_parse(3), which fully parses the HTTP request. Application developers must invoke this function before all others. For FastCGI, this function is split between khttp_fcgi_init(3), which initialises context; and khttp_fcgi_parse(3), which receives new parsed requests. In either case, requests must be freed by khttp_free(3).

All functions isolate the parsing and validation of untrusted network data within a sandboxed child process. Sandboxes limit the environment available to a process, so exploitable errors in the parsing process (or validation with third-party libraries) cannot touch the system environment. This parsed data is returned to the parent process over a socket. In the following, the HTTP parser and input validator manage a single HTTP request, while connection delegator accepts new HTTP requests and passes them along.

Implementation Details Implementation Details

This method of sandboxing the untrusted parsing process follows OpenSSH, and requires special handling for each operating system:

seccomp(2) (Linux)
This requires a seccomp-enabled Linux kernel and a recognised hardware architecture. It is supplemented by setrlimit(2) limiting.
pledge(2) (OpenBSD)
This will only work on OpenBSD >5.8.
sandbox_init(3) (Apple OSX)
This uses the sandboxing profile for pure computation as provided in Mac OS X Leopard and later. This is supplemented by resource limiting via setrlimit(2).
capsicum(4) (FreeBSD)
Uses the capabilities facility on FreeBSD 10 and later. This is supplemented by resource limiting with setrlimit(2).

Since validation occurs within the sandbox, special care must be taken that validation routines don't access the environment (e.g., by opening files, network connections, etc.), as the child might be abruptly killed by the sandbox facility. (Not all sandboxes do this.) If required, this kind of validation can take place after the parse validation sequence.

The connection delegator is similar, but has different sandboxing rules, as it must manage an open socket connection and respond to new requests.

testing

kcgi is shipped with a fully automated testing framework executed with make regress. To test your own applications, use the kcgiregress(3) library. This framework acts as a mini-webserver, listening on a local port, translating an HTTP document into a minimal CGI request, and passing the request to a kcgi CGI client. For internal tests, test requests are constructed with libcurl. The binding local port is fixed: if you plan on running the regression suite, you may need to tweak its access port.

Another testing framework exists for use with the American fuzzy lop. To use this, you'll need to compile the make afl target with your compiler of choice, e.g., make clean, then make afl CC=afl-gcc. Then run the afl-fuzz tool on the afl-multipart, afl-plain, and afl-urlencoded binaries using the test cases (and dictionaries, for the first) provided.

performance

Security comes at a price—but not a stiff price. By design, kcgi incurs overhead in three ways: first, spawning a child to process the untrusted network data; second, enacting the sandbox framework; and third, passing parsed pairs back to the parent context. In the case of running CGI scripts, kcgi performance is bound to the operating system's ability to spawn and reap processes. For FastCGI, the bottleneck becomes the transfer of data.