kcgi – minimal CGI and FastCGI library for C/C++

Version 0.13.4
  • Version 0.13.4 2024-09-15
    Adding CORS request and response headers: Access-Control-Allow-Credentials, Access-Control-Allow-Headers, Access-Control-Allow-Methods, Access-Control-Allow-Origin, Access-Control-Expose-Headers, Access-Control-Max-Age, Origin, Access-Control-Request-Headers, and Access-Control-Request-Method. While here, also add a brief tutorial on handling CORS pre-flight requests.
    Fix a big in handling POST forms with parameters passed in with the application/x-www-form-urlencoded content type. These are now correctly processed.
  • Version 0.13.3 2023-12-02
    Fix shared library on Mac OS X.
  • Version 0.13.2 2023-11-25
    Un-break seccomp for i386.
    Start installing shared libraries alongside the static archives. Also restrict symbol visibility to the contents of the public header files.
    Decouple writer libraries (kcgihtml(3) and such) from reaching into private functions of libkcgi(3). This meant moving some writer functions (e.g., kcgi_writer_write(3) into the public header file. This also allows for creating external writer routines.
  • Version 0.13.1 2023-11-19
    Re-enable seccomp by default after introducing GitHub actions to test on alternate architectures (armv7, aarch64, etc.).
    Add khttp_fcgi_getfd(3), which allows for asynchronous applications (those with a descriptor polling loop) to incorporate kcgi.
  • Version 0.13.0 2021-09-23
    Disable Linux's seccomp by default. I'm just not able to maintain this facility in a reliable way. For future work, it looks like landlock might be a better solution. Maintainers can enable seccomp by editing the Makefile to re-add support. (This will also add debugging.) I'll continue to add patches for those running with seccomp, but don't have the bandwidth to test them.
    Allow khttpbasic_validate(3) to validate both bearer and basic requests. This follows from conventional usage of bearer tokens.
  • Version 0.12.5 2021-05-08
    Add support for bearer authorisation tokens. These are described in RFC 6750. Fix khttp_fcgi_test(3) to work properly when in variable-pool mode.
  • Version 0.12.4 2021-03-02
    Add some attributes and elements for HTML5.2, which is the standard supported by kcgihtml(3).
  • Version 0.12.3 2021-01-20
    Bug fix: fix some bad pointer arithmetic on a regression test on Linux. While here, fix how make regress would spam some systems with erroneous warning messages.
  • Version 0.12.2 2020-07-24
    Bug fix: some errors encountered writing connection file-descriptors to child processors in FastCGI were accidentally ignored. This came because the function returned success, failure, or hangup, while the caller was testing for failure/success. No such error has ever been reported in the wild—it would probably signify much larger issues that would soon be caught anyway—but the fix is still valid.
    Significant simplification of the back-end in removing several crufty interfaces. Back-end logging now uses the front-end logging functions for consistency and to avoid having separate loggers. The same applies to the debugging parameter passed into khttp_parse(3).
    Bug fix for FreeBSD with Capsicum when logging to a file opened with kutil_openlog(3). Same goes with the Linux and Darwin sandboxes.
  • Version 0.12.1 2020-05-11
    Deprecate kutil_date2epoch, kutil_date_check, kutil_datetime2epoch, and kutil_datetime_check in favour of khttp_datetime2epoch(3) and khttp_date2epoch(3). These variants use portable implementations of gmtime(3) and timegm(3) that are not encumbered by per-system constraints such as FreeBSD not accepting years prior to 1900 and of course the 32-bit problem.
    Add khttp_epoch2datetime(3) to symmetrise khttp_datetime2epoch(3).
    Further deprecate kutil_epoch2str, kutil_epoch2utcstr, kutil_epoch2tmvals, and KUTIL_EPOCH2TM as khttp_epoch2str, khttp_epoch2ustr, khttp_epoch2tms, and KHTTP_EPOCH2TM, respectively. The new forms, besides having consistent naming, specifically account for corner cases like negative dates, years with more than four digits, etc. These no longer use the system strtime(3) due to inconsistencies between implementations (e.g., Oracle Solaris not printing >4 digit years) and the 32-bit problem.
  • Version 0.12.0 2020-04-10
    Add the khttp_urlpart(3) family to replace the now-deprecated kutil_urlpart functions. These are for the most part a re-name of the old functions that remove the unused struct kreq argument. There are some small behaviour changes from the original in corner case usage: they have an empty suffix (not just NULL) inhibit printing the dot-suffix, allow a NULL page, and have an empty or NULL page also inhibit the suffix. This way, these functions only produce valid URLs, and also allow for some previously-disallowed (but valid) forms such as /?foo=bar.
    The previous functions have been retained with the original behaviour with one exception: before, an empty suffix would still print the period separator. Now, this is suppressed.
    The khttp_vurlpart and khttp_vurlpartx forms, which accept a variable-length type, are also now exposed for use.
    Add the khttp_urlabs(3) function, which is similar to the earlier kutil_urlabs but significantly more robust and accepts query string arguments. The earlier kutil_urlabs is retained, but deprecated.
    Add khttp_urlencode(3) and khttp_urldecode(3), replacing the legacy kutil_urlencode and kutil_urldecode. They're identical except in how NULL values are handled, in the first case returning them as empty strings instead of NULL, in the second regarding them as errors. For the encoder, this allows all URL formatting tools to pass NULL values as query string values without errors. The earlier functions have been retained with the original behaviour with one exception, in that a NULL destination argument for kutil_urldecode triggers a KCGI_FORM return.
    Have kxml_puts(3) and kxml_write(3) handle NULL pointers as content.
    Verify and fix that all scope-opening functions in kcgixml(3) and kcgihtml(3) properly close variable arguments contexts on error.
    Instead of aborting when the maximum number of scopes in kcgijson(3), kcgihtml(3), or kcgixml(3) has been reached, return KCGI_ENOMEM.
    Check that the element index passed to kxml_pushnull(3) is valid.
    Behaviour change: previously, several kcgixml(3) functions would return KCGI_FORM, such as when popping from an empty stack. To prevent other KCGI_FORM errors from being masked, use KCGI_WRITER to handle these situations.
    Split apart kcgixml(3) into one manpage per function.
  • Version 0.11.0 2020-03-27
    Bumping minor number due to some API and behaviour changes.
    API change: previously, khtml_ncr(3) accepted a uint16_t for its entity value. However, these values can legitimately be 32 bits. It has been changed to uint32_t.
    API change: the struct khtmlreq pointer passed into khtml_elemat(3) is now const.
    Behaviour change: previously, khtml_closeto(3) would return KCGI_FORM if given a stack position greater than the current stack. This is inconsistent with other functions, so such values are now simply ignored. It also masks other problems that cause KCGI_FORM to return. Furthermore, if this function was invoked at the current depth, it would close all scopes instead of none. This has also been fixed.
    Behaviour change: previously, many kcgijson(3) functions would return KCGI_FORM if used out-of-context, for example, trying to open a named object in an array context. To prevent other KCGI_FORM errors from being masked, introduce a new error code KCGI_WRITER to handle these situations.
    Passing a NULL pointer value to the string writing functions of khttp_puts(3), khttp_write(3), kcgihtml(3) or kcgijson(3) would cause undefined behaviour. Now these are noops.
    Add khtml_printf(3). Split apart kcgihtml(3) into one manpage per function. Split apart kcgijson(3) into one manpage per function.
  • Version 0.10.18 2020-03-22
    Add khttp_printf(3) and kcgi_buf_printf(3). Split out manual pages for khttp_putc(3), khttp_puts(3), khttp_printf(3), and khttp_write(3); and also kcgi_buf_putc(3), kcgi_buf_puts(3), kcgi_buf_printf(3), and kcgi_buf_write(3). Add many more regression tests for the behaviour of these functions.
  • Version 0.10.16 2020-03-17
    System now works out-of-the-box on OpenBSD, FreeBSD, NetBSD, Linux, SunOS, OmniOS (IllumOS), Darwin.
  • Version 0.10.15 2020-03-11
    Portability updates to the build system. Bring in the newest oconfigure.
  • Version 0.10.14 2020-02-27
    Start using pkg-config for all kcgi libraries. This makes it much easier to use the system without knowing installation details. All documentation has been upgraded to note the fact.
  • Version 0.10.13 2020-02-17
    Re-write the internal logging function so that it does not use a static buffer, lifting the previous 1024 byte limit. This means that log messages might exhaust memory, but that seems the lesser of two evils.
    Relax kvalid_email(3) so that it accepts three-byte e-mails (e.g., a@b) but tighten it to require not starting or ending with a @.
    Add a new user-visible wrapper function, kvasprintf(3).
    Protect against NULL format strings being passed to the logging functions.
  • Version 0.10.12 2020-01-12
    Fix regression framework. This was failing for FastCGI tests because the framework wasn't properly prefixing the HTTP version as it was for CGI tests. This now allows all tests to run under the new curl.
    Use the proper Capsicum header for FreeBSD.
    Bring in the latest oconfigure and also merge dependent changes for seccomp on Linux. Then enable seccomp protection for aarch64 on Linux.
    Bring in optimisations and corrections in URL decoding by Dapeng Gao, thanks!
  • Version 0.10.11 2019-07-05
    Maintenance release: bring up to date with latest oconfigure, merge pending GitHub pulls.
  • Version 0.10.10 2018-12-27
    A lot of work for functioning on FreeBSD (specifically, breakage with the sandbox) and musl libc. Thanks to href@random.sh for access to a FreeBSD machine for testing and solving this issue!
    Add kutil_urldecode(3) and kutil_urldecode_inplace(3), which are the reverse of the existing encode functions.
    Also, revert to using BSD Makefile instead of GNU. Linux downstream will need to adjust to use bmake for the build sequence.
  • Version 0.10.8 2018-12-18
    Good-bye, systrace(4)…
    Many minor documentation fixes from the GitHub pulls and issues page. Thank you for everybody's submissions!
    Many thanks to Valentin Pistol for verifying that the regressions suite works properly on newer (Majove) builds of MacOS.
    Allow compilation on musl by bringing in latest oconfigure and making some tweaks. This also makes building without zlib to complete without compiler warnings.
    Fix transmitting certain UTF-8 characters via JSON.
  • Version 0.10.7 2018-06-19
    Some excellent fixes from mk-f@ in getting better support for kcaldav: fixing the nc value in digest authentication to be hex and adding support for application/xml to the list of supported MIME types. Thanks!
    Remove -lbsd requirement for Linux regression tests. Fix another issue where fpclassify(3) on Linux requires -lm.
  • Version 0.10.6 2018-04-19
    Quick fix from Ross Richardson regarding URL encoding with the high bit set. Thank you so much! Also push in some small fixes from the GitHub list.
  • Version 0.10.5 2018-04-19
    Add kcgi_strerror(3) for a string representation of the error codes. Split kutil_invalidate(3) into its own manpage. Fix up some corner cases in template handling: trailing delimiters, empty key sequences, and so on. Also allow for escaping delimeters, \@@, allowing for the existence of delimeters as opaque text.
    These fixes were implemented or suggested by Ingo Schwarze (schwarze@) in an audit generously funded by CAPEM Solutions, Inc. Thank you so much!
  • Version 0.10.3 2018-04-10
    The FastCGI release: when running kcgi's FastCGI mode on nginx, processes were being mysteriously killed under high load. This was due to the end-point closing the connection before all data was being read or written. To wit, I now establish a difference (in FastCGI) between the connection closing (which is a recoverable error) and the manager killing the connection or the control socket exiting, which are not recoverable. Since most of this development was on Linux/ARM with nginx, the sandbox for Linux has also been tooled up. A big thanks to Elouan Pignet, who was kind enough to diagnose the problem and provide access to his system for a fix, including several failed attempts. Thanks, Elouan!
    To this end (API change), khttp_fcgi_parse(3) now returns the KCGI_EXIT when the system has exited. The KCGI_HUP is reserved for when the output channel has closed (after parsing) and the current connection is no longer valid. The documentation has been updated for relevant functions.
    While studying these code paths, make sure that a sequence of writes (using khttp_write(3) or any of the writing front-ends) won't fail if khttp_body(3) wasn't able to complete due to the connection closing. Specifically, if the connection closes during khttp_body(3) (returning KCGI_HUP), the system will still expect headers. Earlier, it would assert with subsequent khttp_write(3) if the error were not caught and the In the modified behaviour, it will return KCGI_FORM to indicate that the system is out of state.
    Make khttp_fcgi_parse(3) only require that callers invoke khttp_free(3) if exiting with success. This mirrors khttp_parse(3).
    Merge a set of tutorial fixes from cyball, thanks!
    Allow the kutil_log(3) functions to accept a NULL request. This makes it possible to use these functions for consistent logging without a request.
    Lastly, run through all code snippets, apply style(9), and make sure that the MIME type is properly checked. And add a new tutorial, Best practises for pledge(2) security.
  • Version 0.10.2 2018-03-24
    Merge a set of patches from pull/26. These fix the digest authorisation in auth-int mode, most often used by CalDAV systems. Thanks to Charles Collicutt for the contribution!
    While raising kcaldav from the dead, fix where the XML writer wasn't returning the correct error code and causing strange errors.
    Lastly, fix the Linux sandbox when running on ARM machines, enable building for C++ applications, and add a tutorial and sample file. Pull in most recent oconfigure.
  • Version 0.10.1 2018-03-06
    Many of the improvements and fixes in this release were implemented or suggested by Ingo Schwarze (schwarze@) in an audit generously funded by CAPEM Solutions, Inc. Thank you so much!
    Proper versioning in the header file as documented in kcgi(3).
    Fix kcgijson_string_write(3) and friends to not emit a superfluous quote before each invocation.
    Add kcgi_buf_write(3) and friends to make working with khttp_templatex(3) easier. (I also use it for libcurl(3) in-memory buffers.)
    API change: complete the change-over to writing functions returning values by modifying khttp_template(3) and friends to return a proper error code. While here, significantly improve the manpage.
  • Version 0.10.0 2018-01-18
    A minor bump here due to API changes, along with a plethora of improvements: date handling, simplification of general code, and getting error codes exported in the write sequence. Many of the improvements and fixes in this release were implemented or suggested by Ingo Schwarze (schwarze@) in an audit generously funded by CAPEM Solutions, Inc. Thank you so much!
    Memory leak fix: fix khttp_free(3) to free a forgotten allocation. Probably only affects FastCGI processes at 16 bytes lost per request parsed.
    Feature: add the kcgi_writer_disable(3) function, which allows callers to determine which front-end writers (e.g., kcgixml(3)) are allowed to exist. This locks down the output formatting mechanism.
    API change: the kcgixml(3) library now no longer prints the XML prologue with kxml_open(). This must be manually printed with kxml_prologue().
    API change: the kcgijson(3), kcgihtml(3), and kcgixml(3) now return a consistent error code for all operations. The khtml_text function has been removed (it was deprecated).
    API change: the kutil_urlencode(3) family of functions now all return NULL on memory failure. Earlier, this was inconsistent.
    API change: the khttp_write(3) family of functions now all return enum kcgi_err to indicate a failure condition.
    API change: the khttp_body(3) function now returns enum kcgi_err to indicate a failure condition instead of whether compression was enabled.
    API change: the khttp_body_compress(3) function now returns enum kcgi_err to indicate a failure condition instead of whether compression was enabled. Furthermore, the comp argument simply dictates whether compression should be enabled or not, preventing confusion.
    API change: the khttp_head(3) function now returns enum kcgi_err to indicate a failure condition. Furthermore, this function now dynamically allocates header lengths, removing prior bounds on header length.
    While here, improve khttp_head(3), khttp_body(3), kvalid_string(3), and kmalloc(3) manpages.
    Lastly, the kvalid_date(3) function has been generalised for arbitrary dates, prompting the addition of kutil_date_valid(3) and kutil_datetime_valid(3) functions to validate broken-down dates. And bring up to date with latest oconfigure.
  • Version 0.9.10 2017-11-21
    Many internal small issues (allocation catches, better logging, simplifying logic) fixed as found by Ingo Schwarze (schwarze@) in an extensive audit generously funded by CAPEM Solutions, Inc. None of these change application behaviour except that standalone query parts are let through. For example, localhost/foo?bar=baz&xyzzy now passes xyzzy as a key-pair with a zero-length pair.
    Note that parsing text/plain enctypes is now deprecated, as I'm yet to see this ever used.
  • Version 0.9.9 2017-10-30
    Switch to using oconfigure. Add the kvalid_bit(3) function for bit fields.
  • Version 0.9.8 2017-10-18
    API changes: use the appropriate uint32_t size for the HTTP digest authorisation nonce count. This follows RFC 7616, sec. 3.4. Also add the kutil_err(3) family of functions, which report an error and exit. Split that into kutil_openlog(3) as well. Lastly, commit considerable improvements to the khttp_parse(3) and other manpages, as well as some extra warning messages due to RFC violations during HTTP parse. Most of these were found and patched by Ingo Schwarze (schwarze@) in an extensive audit generously funded by CAPEM Solutions, Inc. Thank you!
  • Version 0.9.7 2017-09-23
    Fix C headers in the documentation to be more minimal and standards-compliant.
    Add Ross Richardson's Using Pages tutorial. Ross developed the new Undeadly Journal!
    Bug fixes… Clarify some function elements in khttp_parse(3). Fixed undefined behaviour caused when khttp_templatex_buf(3) is passed a fallback function and mis-reports the input key length. (If you use templating with a fallback function, you must update.) Fix all instances of zero-length allocations. These are non-portable and might cause erroneous failure on some systems. Fix an error where using digest authentication via khttpdigest_validate(3) might crash with an unknown HTTP method. Most of these were found by Ingo Schwarze (schwarze@) in an extensive audit generously funded by CAPEM Solutions, Inc. Thank you!
  • Version 0.9.6 2017-08-31
    Add the khttp_template_fd(3) functions for passing a file descriptor into the template utility instead of an open file or buffer. (Also clean up that documentation.)
  • Version 0.9.5 2017-05-31
    Clarify documentation for kcgijson(3) and add an uninstall rule to the GNUmakefile for those not using a package-managed version of the library. No code changes.
  • Version 0.9.4 2017-03-12
    Add a feature inspired by Ross Richardson: a fallback function for templates. As described for the khttp_template(3) family of functions, a fallback function is invoked if none of the fixed keys are found. This is handy because you can provide parsed template keys, e.g., foo[bar] and foo[baz], which would be parsed for a type foo that's passed a dynamic value bar or baz.
    Nota bene: this required the khttp_templatex(3) and khttp_templatex_buf(3) functions to have an argument change.
  • Version 0.9.3 2017-01-25
    Numerous spelling errors fixed (no functional change) and added kutil_epoch2utcstr(3) for formatting ISO 8601 dates. Also fixed KUTIL_EPOCH2TM(3) macro to set correct fields.
  • Version 0.9.2 2016-11-27
    Following a report by Ross Richardson (thanks!), fix cookie handling. Before, cookie values (and keys) were being handled as URL-encoded strings. Now they're correctly handled as opaque values; and moreover, they use a stricter check against RFC 6265. While there, update some spelling and typos (thanks Svyatoslav!).
  • Version 0.9.1 2016-10-17
    Get rid of BSD make in favour of GNU make. I generally don't like GNU software, so let me explain: portability and readability. The original Makefile was almost 500 lines; the new one is 350. While this software isn't exactly a moving target, it's still annoying to add the same lines over and over again. If you can think of a better way, please let me know.
    Next, add some logging functions: kutil_log(3) and friends. Why? I find myself re-rolling these same routines over and over again. They're similar (enough) to the NCSA format, sanitise (and bound) output, and time-stamp without tripping pledge(2).
    Lastly, add an index of all functions.
  • Version 0.9.0 2016-10-10
    Regressions, regressions, regressions, regressions. Fix kcgiregress(3) to work in a more general fashion, adding an example usage along the way. Regression tests are (sadly) not used often for web applications, so this tool should be a welcome one! Also add two useful functions: kutil_epoch2str(3) for converting from integral (epoch) UNIX time into an HTTP date and kutil_date2epoch(3) and kutil_datetime2epoch(3) for converting from dates to integral time. Lastly, clarify that khttp_parse(3) requires a call to khttp_free(3) if and only if it returns success.
  • Version 0.8.5 2016-08-19
    Some documentation fixes and clarifications as suggested by Ross Richardson—thanks! Also added a custom validation tutorial.
  • Version 0.8.4 2016-06-14
    Fix several documentation bugs that erroneously noted values can be NULL when they would instead by empty strings. Also fixed some broken links.
  • Version 0.8.3 2016-04-19
    Work around an old but fatal FastCGI problem only found on OpenBSD's httpd. This assumes that HTTP headers are only on the first FastCGI packet, which is not part of the standard. The workaround is to have HTTP headers buffer just like the HTTP body, which will cause multiple headers to lump into (hopefully) one packet. Headers are still flushed when the HTTP body begins, however, although I'm still unsure whether this is a good idea. (It pushes the status code to the web server faster, but incurs an extra trip on the wire.) If you've disabled buffering in khttp_fcgi_init(3), or you have so many headers that the output buffer is flushed before the last header, httpd will intermix your body with headers. Ew. This problem was raised as issues/2 by @cornett—thanks! Also fix issues/3, raised by the same—thanks again!
  • Version 0.8.2 2016-04-06
    Fix the -l flag in kfcgi(8) as dictated in pull/1 on the GitHub. Thanks, cornett!
  • Version 0.8.1 2016-03-15
    Significantly update the FastCGI handling. There were two foci to this effort: first, to make the existing FastCGI system more robust in terms of starting and stopping; the second, to extend FastCGI so that kfcgi(8) can handle variable-sized pools of workers instead of a fixed number. The method of extending FastCGI is described in FastCGI Extensions for Management Control, and is implemented by the -r flag in kfcgi(8).
  • Version 0.7.8 2016-03-02
    Use strtonum(3) whenever possible, inspired by a patch from Jan Schreiber—thanks! Add khttpdigest_validatehash(3), which authenticates an HTTP digest session using a pre-computed hash. (The existing function builds the hash from a password.) This is required by kCalDAV.
  • Version 0.7.7 2016-02-10
    Allow for the developer to set a write buffer size when invoking khttp_parsex(3) or khttp_fcgi_initx(3). The write buffer hooks into khttp_write(3) and, if set to a non-zero size, will cause writes to be buffered. This changes existing behaviour where writes were never buffered. Note that this function is invoked by all writing functions, both within kcgi(3) and its libraries such as kcgijson(3). The buffer is flushed when its size is exceeded or when khttp_free(3) is invoked. If not provided, the default is 8 KiB for CGI and 65 KiB for FastCGI. Also renamed the xmalloc-style internal functions so as not to override weak symbols in any interfacing applications. This was noted by Okan Demirmen—thanks!
  • Version 0.7.6 2016-01-25
    Push read-only repository to GitHub. kcgi has been accepted as a Coverity project. The initial scan revealed a pair of error-path resource links and some false positives. (All of these are visible on the Coverity project site.) It also found one legitimate bug in the newly-installed HTTP basic authentication code. This, obviously, has been fixed. Thanks, Coverity!
  • Version 0.7.5 2016-01-04
    Added ability to easily perform HTTP digest authentication, khttpdigest_validate(3), which is heavily used in kcaldav and a few other systems. This manages all of the aspects of the sequences, from the child process performing an MD5 hash on the full data stream to the end check against a hash. While there, also add khttpbasic_validate(3) for completeness.
    Add an experimental feature for debugging input and output streams. One can set debugging fields with khttp_parsex(3) or khttp_fcgi_initx(3) and have request reads or response writes logged to standard output. Note: this changes the system API, so make sure any systems calling the above functions are upgraded to account for the extra argument! Also merged a thorough documentation patch by Svyatoslav Mishyn—thanks!
  • Version 0.7.4 2015-11-12
    Replace support for tame(2) (which was never enabled in the operating system) with the renamed pledge(2). From a patch by James Turner—thank you!
  • Version 0.7.3 2015-09-30
    Fix for proper FastCGI support on nginx. This work was prompted by Daniel Sinclair, who fixed a read type mismatch (when reading the padding length) that inspired a further fix to serialise the response FastCGI header properly. Lastly, a latent Apache2 FastCGI bug was fixed in incorrectly reporting the return code status.
  • Version 0.7.2 2015-09-04
    If a FastCGI connection closes, don't make failing writes to that connection bring down the application. Also work around a very weird Mac OS X bug wherein a poll will return a timeout even though the poll request is blocking.
  • Version 0.7.1 2015-08-11
    Fix: forgot to install kfcgi(8). Add flag to kfcgi(8) to specify the connection backlog. Add khttp_fcgi_test(3) to see if an application should use the FastCGI or regular CGI functions. A regression noted by James Turner (thanks!): re-add the CGI sample, sample.c, into the distributed source. There is also a FastCGI sample, sample-fcgi.c. There is also a non-kcgi sample, sample-cgi.c, used in creating the performance graphs.
  • Version 0.7.0 2015-08-07
    Implement the FastCGI control process's sandbox for all supported operating systems. (The FastCGI control process is forked from each application process, and is responsible for accepting new FastCGI connections and passing the descriptors to the application for output and worker process for parsing.) This is the last feature for the FastCGI implementation: it now has the same protection as the CGI implementation for all child processes. Add sandbox for OpenBSD's tame(2), although this technically isn't supported yet (in snapshots, anyway) and returns ENOSYS. On OpenBSD machines with both sandboxes, this is tried first. This effort derives from a patch submitted by Reyk Floeter—thanks!
    The FastCGI interface is no longer an experimental feature, but baked into the system.
    In addition to the above, some variables were added to the manpages (which were also cleaned up a bit) and the AFL system was updated with the socket re-write.
  • Version 0.6.4 2015-08-01
    Add some proper security to kfcgi(8): place child FastCGI processes in a file-system jail, drop privileges, and ensure proper ownership of the socket. While here, begin sandbox mechanisms for the FastCGI control process. This only works for Mac OSX for now, but will expand to others. Again, until the next minor version bump, FastCGI support is experimental.
    On that note, finalise the API for FastCGI calls. This involved moving most function arguments previously in khttp_fcgi_parse(3) into khttp_fcgi_init(3). To wit, the khttp_fcgi_parsex() function has been removed: all of the logic has been moved to the initialisation function, making the parse function much simpler. Cement this by adding several new regression tests that exercise the FastCGI functionality. These, of course, required that FastCGI functionality be added to the regression suite. This is documented in kcgiregress(3) (the manpage was renamed from kcgi_regress).
    In adding the regression suite, one critical FastCGI bug was found (and fixed) for forms >1 kB. Fix another critical bug when writing large contiguous blocks, which would be silently truncated. (A regression test has been added for this.) The same would trigger a failure on Linux due to the sandbox: this has also been fixed. Internally, the behaviour of read and write sockets is consistently non-blocking. There has been a report of large compressed output failing on FreeBSD, but this has not yet been verified.
    Lastly, incorporate a patch from Baptiste Daroussin (thanks!) for a tighter Capsicum sandbox for the parser process.
  • Version 0.6.3 2015-07-18
    More FastCGI: move control socket handling into its own process. The control socket is a UNIX socket (or similar) bound by the FastCGI runner, which is either the web server or an application like kfcgi(8), then passed as STDIN_FILENO to the FastCGI application. kcgi will then wait on this socket for incoming connections, which are acted upon with khttp_fcgi_parse(3). In this release, this logic has been moved into its own process instead of being managed by the web application itself during calls to khttp_fcgi_parse(3). While here, I cleaned up and simplified a lot of the sandbox and inter-process socket logic. The control socket is not yet sandboxed: that will come with later releases. Again, the FastCGI implementation is experimental!
    Patches for deployment on FreeBSD contributed by Baptiste Daroussin—thanks! In analysis, found that the argfree function to khttp_parse(3) wasn't being invoked if the arg was itself NULL. (This is clearly bad behaviour—not all functions need that argument!) This has been fixed as well.
  • Version 0.6.2 2015-07-15
    This release includes an initial implementation of FastCGI. While the existing functionality wasn't logically altered, it was shuffled around quite a lot and abstracted to accomodate for the new FastCGI functions. The implementation is documented in kfcgi(8), khttp_fcgi_init(3), and khttp_fcgi_parse(3), and khttp_fcgi_free(3). This functionality is experimental, and needs strong analysis before casual deployment. It is also not quite feature-complete: HTTP compression is not yet functional for FastCGI applications. The kfcgi(8) launcher is also very bare-bones—this will be improved in subsequent releases. Again: this functionality is experimental and under development!
    Also fixed a bug where calling khttp_free(3) after a prior khttp_parse(3) failure would cause a NULL dereference. Added the missing txt and xml suffixes to the suffix table. Fix that the request port number was erroneously disallowed to be >80.
  • Version 0.6.1 2015-07-09
    One-line bug-fix to include new Linux seccomp patch into source archive. Ouch. Noted by James Turner—thanks!
  • Version 0.6 2015-07-07
    Import initial Linux seccomp(2) (via prctl(2)) sandbox! This uses the implementation of OpenSSH, tweaked to work within the kcgi framework. For now, it only allows arm, i386, and x86_64: if you're using another architecture, please let me know your uname -m, as the sandbox (ridiculously) needs to know the system architecture. (Better yet: also send me the relevant AUDIT_ARCH_xxx from /usr/include/linux/audit.h.) While here, allow for compilation on musl. I've also moved the tutorial into a separate file and fleshed it out a little. I'll probably add more tutorials in time.
  • Version 0.5.9 2015-06-17
    Add ability to retrieve HTTP headers from the request. This adds several fields to the structure filled in by khttp_parse(3) that allow for mapped (indexed by known header) and listed header fields. The HTTP headers are reconstituted from the CGI environmental variables according to the RFC. These fields existed in earlier releases, but were not documented and, moreover, did not reconstitute the HTTP header name from the CGI environment variable name. A regression test has been added to test this behaviour. While adding this documentation, clean up the type list in khttp_parse(3).
    Add khttp_body_compress(3), which allows for stipulating auto-compression (the default for khttp_body(3)), compression without emitting the appropriate header (for applications providing the Content-Encoding parameter themselves), and no compression (for applications taking full control of output themselves). While there, make the test for requested compression be sensitive to the RFC 2616 qvalue. Both functions now return whether compression has been enabled. This functionality augments existing behaviour: it does not change it.
    Add HTTP error codes described by RFC 6585.
  • Version 0.5.8 2015-05-11
    Update the included sample file and correct the documentation regarding its compilation. Fix noted by Jan Schreiber — thanks! While doing so, add some documentation bits to the manpages and considerable documentation to the webpage. No functional changes.
  • Version 0.5.7 2015-04-27
    Rename khtml_close to khtml_closeelem in kcgihtml(3), then re-add the close function and an open function to harmonise with kcgijson(3) and kcgixml(3). In the process, allow the closing functions in all libraries to unwind any remaining context, and have the closing functions return whether the request was out of bounds. Prevent some bogus calls to kcgihtml(3) from aborting. Bug-fix for detecting zlib on FreeBSD, found by Baptiste Daroussin. (Thanks!)
  • Version 0.5.6 2015-04-23
    Bug-fix for Capsicum sandbox, found by Baptiste Daroussin. (Thanks!) Also fix passing a zero-length buffer into the template buffer functions.
  • Version 0.5.4 2015-04-12
    Single bug-fix: the nonce count parsed from an HTTP digest was not being correctly recorded.
  • Version 0.5.5 2015-04-12
    Bug-fix when linking to kcgihtml(3). Added the access scheme (HTTP or HTTPS) to struct kreq. Have validation for document body correctly set the ctypepos prior to validation. While there, properly decode the content-type field (i.e., discarding parameters) when looking up the type in the known types.
  • Version 0.5.3 2015-04-10
    Fully process the Authentication header, implementing RFC 2617. This was originally developed in kcaldav, but makes more sense to be run here inside of the untrusted child. Values are stored in the struct khttpauth field documented in khttp_parse(3).
  • Version 0.5.2 2015-04-05
    Allow for opaque HTTP message bodies: if our content type isn't application/x-www-form-urlencoded, multipart/form-data, or text/plain (during a POST only), then accept the HTTP body as a single object and validate it against the empty-key validator. Add HTTP methods and headers stipulated by HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) and Calendaring Extensions to WebDAV (CalDAV). Allow for the HTTP request headers to be exported to the struct kreq object as both a list and, for common HTTP headers, an indexed map. Added kcgixml(3) bits for some simple XML support and added khtml_putc() and khtml_puts() to kcgihtml(3) for consistency. Specify that a NULL template passed to the khttp_template(3) functions simply causes the named file or buffer to be outputted without any processing. Lastly, recognise getentropy(2) as a white-listed system call in the systrace(4) sandbox.
  • Version 0.5.1 2015-03-14
    Allow kcgi_regress(3) to handle query strings and forms for validation testing. Pick out one assertion failure in the parser by using the American fuzzy lop. Add an a test harness for AFL as well. Recognise CONNECT, DELETE, HEAD, TRACE, PUT, and OPTIONS methods.
  • Version 0.5 2014-12-18
    Expand the functionality of the template system with khttp_templatex(3) and khttp_templatex_buf(3), which allow the writing function to be overriden. This allows templates to play nicely with other systems such as kcgijson(3) and general dynamic buffer creation. I use this for creating mails for sending with libcurl. Also create an automated test harness kcgi_regress(3) that, for internal automated regression testing, uses libcurl to create and parse HTTP messages over a local socket and a small server to translate the requests into CGI variables for kcgi. The internal tests can be run with make regress.
  • Version 0.4.4 2014-12-15
    Add the Boolean and NULL types to kcgijson(3). Fix control code serialisation in the same.
  • Version 0.4.3 2014-12-03
    Make sure that khttp_parse(3) returns an error code indicating what actually went wrong. This is a minor API change because the function no longer returns 0, but an enumeration (where 0, incidentally, means success). Also audit the parsing trailer to make sure that the parent doesn't exit in the event of memory exhaustion.
  • Version 0.4.2 2014-11-30
    Implement the kcgijson(3) library for convenient JSON functions, remove function prototype names as per OpenBSD's style(9), polish documentation.
© 2014–2019 Kristaps Dzonsons
Built with sblg on OpenBSD