Add significant regression tests to the Node.js validators,
normalising their behaviour in some cases.
Add ability to break apart
ort-nodejs(1) output into multiple files.
Re-write internals of auditing system. These now use the internal
ort_audit(3) function. The output of
ort-audit(1) has been formalised and
is now documented (it differs from the prior output).
Formalise the output of
ort-audit-json(1) and include
count operation in its output.
For coherence with the program name, output straight-up JSON and
ort-audit-gv, which isn't really useful.
Add callback mechanism to ort-json.ts.
Add some new methods to the generated
ort.Long output with
easier for front-ends to work with 64-bit integers. Add regression
tests for these.
Start work on a
ort-c-manpage (not yet released) that
will allow a C API to be output as a
I personally use this because it makes scanning for functions faster
than looking in the header files.
functions. No change to code, but adding
or and some
documentation. Also relax some needless constraints in
Fix a bug in ort-sqldiff(1) where
struct fields were considered SQL columns.
A number of improvements to the ort-json.ts machinery.
Probably not going to change any normal usage.
Introduce a significant change where names in
ort(5) configurations are internally
converted to lowercase. This means that all output will use the
lowercase versions of the read names, which affects APIs.
Why is this a good idea? Largely because it keeps APIs consistent
with changing (for example) structure names, which is already
not a change because structure names are case
It also cleans up some inconsistency where some fields (e.g., roles)
were lowercased already, while most (structure names, field names,
explicit query naming, etc.) were not.
Install the manual and package configuration for
which creates TypeScript front-end code.
Also tighten down role name case insensitivity and fix serialisation
of empty role objects in ort-json.ts.
Tiny changes to speed up TypeScript routines.
Minor fix in handling UTF-8 labels and converting to and from JSON.
Have ort-xliff(1) properly handle XML
special characters by on-the-fly encoding and decoding, lifting the
restriction of not being able to use
< in labels.
Fix parsing and writing of limit/offset pairs.
Allow parameter-less search statements.
Enable the exported library for
This is still experimental, however.
Fix how null references (a
struct to a foreign key that
null) was handled both in the generated C
source and in node.js output. Both failed for different reasons.
Continued work on ort-json.ts.
Relax the constraint where overlapping roles applied to the same
operation would error out.
Now we merely warn about them.
Include the ability to convert the JSON representation of a
configuration as provided by
ort-json(1) back into a
This is part of the ort-json.ts file.
Include a formatter for the JSON representation produced by
Enforce that the
default action for
actdel has either a
null field or a
default. This prevents the database from defaulting to a
null value, which may conflict with the configuration
and thus expectations of the code.
Do the same for the
nullify action, requiring that the
Rename some parts of the ort-json(1)
TypeScript definitions file.
Add in some forgotten fields to the definitions.
Make sure that
distinct on a
iterate correctly declare the
required macros in
Found by the expanded regression suite.
grouprow queries, which use
LEFT OUTER JOIN, don't prematurely terminate quoted
strings. Found by the expanded regression suite.
Fix how empty iterators would cause bad
ort-nodejs(1) to be emitted.
Internally, convert front-end utilities to properly emit output to a
passed-in file stream instead of assuming
way, these may be converted into libraries for use within other systems.
ort_lang_json(3), which converts
a configuration into JSON.
These are still very new, but heavily tested.
Export changes to the
roles statement in
ort_diff(3) field results.
Also add full support for the
This completes full difference coverage.
All (or near enough) of these differences are encoded in the
While doing all this, push plenty of documentation from the header
Reconsider white-space handling in comments. Prior to this,
consecutive white-space collapsed into the first white-space.
This was a problem for two reasons: first, Markdown and other common
comment formats recognise multiple newlines to separate blocks.
More significantly, a CRLF end of line would convert into a blank CR
(instead of a LF). Modify the system, thus, to accept comments
as-is except for CRLF, which is reduced to LF.
This makes diffing much more consistent as well.
Continue adding functionality to
No functional changes.
Enumerations, bit-fields, structure uniques, and structure fields
are now fully covered by the interface.
ort-diff(1), which works off the back of
ort_diff(3) in producing the semantic
differences between two configurations.
This is now being used within
ort-sqldiff(1), which has been
The function is still incomplete: it doesn't record all differences.
So far, it returns enough to be fully used by
While here, fully document all of the transformations possible in
ort-sqldiff(1) and also tighten
down on some destructive changes (e.g., allowing type changes) in
The new difference engine has introduced a significant number of
The documented transformations of the SQL engine are also now
exhaustively tested in the regression suite.
Start installing the ort library.
This includes header, library code, manpages, and a pkgconfig.
make sure that
handles numbers and strings when formatting. This was not caught
by the regression framework because
accepts the type-mangling code while
tsc does not.
Fixed and added regression tests that force the issue.
ort-c-source(1) was not generating
db_xxxx_fill_r functions in some cases,
even though required by query functions.
db_open_logging function in
Major change: integer values are now JSON-serialised as
strings intead of numbers.
ort-c-header(1)'s JSON export
ort-c-header(1)'s JSON import
interface, and most importantly,
string as well as of
to parse integers with more than 53 bits and the necessity to handle
full 64-bit numbers.
While JSON can accept arbitrary-precision numbers if used purely as
a transport protocol, it's more portable to just use a string
The only user-facing code this might affect is custom callbacks that
perform numeric operations.
These functions will now also need to handle string values.
This is reflected in the TypeScript type annotations, so any current
numeric operations on values passed to these functions should raise
errors or at least warnings.
There is now a
ortns.Long class modelled after
Long.js that's also
used internally which can perform 64-bit operations.
The code generated by
with deserialised data can handle either string or numeric
(old-style) values, so new front-ends can work with old back-ends.
The reverse is not true.
While the proper (and simpler) way to store integers in
BigInt, this isn't yet standard across browsers, and
would require alternative encoding during transit anyway (JSON
doesn't recognise it). The current implementation is portable
across all browsers.
Also, and without much of an introduction:
This experimental back-end generator builds a
This generates an API very similar to that of
ort-c-header(1), making it easy to
transition between languages.
For now, it does not provide comparable security to
the C implementation.
Output change: do not have delete statements return a
Output change: the output of
sensitive to fields that are not exported.
This applies to fully unexported (i.e., those marked with
noexport or passwords) and having optional fields
(those not exported per role) to be checked and behaviour if not
found documented and regression-tested.
Output change: full support for binary blobs in
base64 encoding throughout.
Those wishing to use the binary data will need to use custom
callbacks with (for example)
Buffer.from() to convert
from base64 to binary.
Considerable improvements to the output of
This has no functional change, but introduces more TypeScript idioms
Output and configuration change:
emits the un-documented and hard-coded
not given (with the
notgiven class) for labelling null enumeration
These are now represented by the isnull label documented in
This is similar to bit-fields.
This necessitates a minor version bump.
In doing so, use TypeDoc instead of
Split out lots of boilerplate code for
pulled in during parse.
The location of this boilerplate may be set with -S, just
like for ort-c-source(1).
This does not affect the behaviour of generated code.
checked="checked" instead of
checked="true" (same with
-enum-select class fills generated by
This is more standards compliant.
the undocumented string
unset was emitted for bit-fields that
don't have corresponding labels when formatted.
It is a documented empty string.
Also handle missing individual labels and so on.
Add a large regression suite to tests for the behaviour documented
the generated documentation.
Be more explicit as to why the
> character is not allowed
and further disallow empty labels.
Clarify the maximum and minimum values for bit-fields and
(Lift earlier restriction of 32 bits after testing on a variety of
In the documentation, clarify native, meta, and user-defined types.
Clarify that sub-structure references may not be recursive.
While there, also add some query examples and clarify the docs
for distinct and grouping queries.
Only accept logical operators on integer and bit types.
Allow string and unary operators (and document as such) for
Limit enumerations to 32-bit values and prevent overflow.
When generating validation functions for the C API, don't emit a
wrapper function if not emitting limiting code—instead,
simply use the kcgi(3)
function itself as the validator.
This saves on superfluous generated code.
count query type to have password equality
This never worked, but now we explicitly check for it.
db_xxx_fill_r functions if there are no
queries that require it.
This saves on superfluous generated code that would otherwise raise
warnings during compilation.
Bug-fix: if errors occur when parsing files with
ort-sqldiff(1), properly return
with an error code.
Earlier, this was not returning an error code.
Clarify the ort-sqldiff(1)
documentation with regard to exactly the cases that cause errors.
Create a regression suite to test for this.
enum fields to have a default type.
password fields to accept
Remove several fields in ort.h used only by the C front-end,
such as the capitalised name of structures. These add moving parts
with few discernable benefits.
Complete removal of
resolution fields from ort.h.
These fields were the superfluous names attached to (for example)
Earlier, these were required because the names were resolved after
parsing, during the linking phase.
A new, safer internal API is used instead.
This effort has resulted in a much cleaner API and a significant
number of regression tests.
grouprow and associated row operation
were not being written by ort(1).
order type was not being written by
distinct keyword on the local structure
was not being printed properly by ort(1).
When running ort-sqldiff(1), be
more selective on which
field attribute changes will
trigger an error.
Significant upgrade to the internals: instead of haphazardly
resolving names to object components (e.g., the field names in
foreign references), do so in a dedicated pass by way of resolver
This simplifies the final parsed tree: where before both the name
and resolved field were part of (for example) an update reference,
now only the resolved field object exists.
In doing so, this has made several structures unnecessary, such as
struct bref. This and others like it existed solely to
carry both the name of the field to resolve and the results
of that resolution.
Many many new regression tests have been added to reflect the checks
that occur during linking.
Other internal APIs and files were moved around for clarity: the
C-generating files are now prefixed with
c, the library no
longer contains front-end objects, etc.
When querying with a password entry, having a non-terminal password
field in the list of search fields would generate an incorrect
ordering when assigning to SQL parameters.
This has been fixed and the code (and documentation) cleaned up and
Allow unary password constraint operations in update queries.
Fix how password
null fields are checked for.
While there, consider how
NULL values passed as the
password are processed.
Before they were not at all.
Fix where auto-valued enumerations weren't auto-value-ing.
Enable and verify portability in the generated source code for a
large number of systems.
This made possible by the latest version of
Significant internal restructuring for clarity.
grouprow, which groups rows by column then
reduces them to a single row with a
At this time, this is limited to a single grouping column and
Possibly API-breaking changes: update functions
declared without constraints no longer emit a superfluous
by in the generated function names. That is,
update [xxx]; will no longer append a hanging
_by to the function name.
Anonymous update functions (i.e., without a
have their modification type emitted in the function name.
This is to prevent name collisions from, e.g.,
update foo inc and
update foo dec.
strset keyword for update modifiers set the
raw password value instead of hashing.
Bring up to date with latest
Have passwords accept both
Following the database library switch in release 0.8.0, update
functions no longer return <0 on error, as error conditions in
the database trigger program exit. The precise return value is now
zero on failure, non-zero on success; however, for backwards
compatibility, <0 shall not be returned so these code-paths are
strneq search operators and
update constraints for passwords.
(For non-passwords, these are equivalent to
These operate on the raw hash instead of the password.
For C source, don't generate
INSERT SQL if it's not
going to be used.
This has no impact on running code, but reduces generated code size
ever so slightly.
Possibly API-breaking changes: now using
sqlbox instead of
ksql for the database
This is due to the performance benefits and simpler API.
The library also mandates safe operation instead of making it
optional, so (for example) all database operations must be specified
Specifically, it will no longer accept SQL not passed at start.
Second, make -s the default for
The underlying database access library only supports split-process
mode, versus ksql which
also had an
Third, and most significantly, require using
struct ort instead of having direct access to
struct sqlbox (was
This mirrors the situation of roles being enabled.
If you have code that did not use roles and directly used the
struct ksql to interface with the database via custom
strings, the code will not to be re-written to use safer
format function of
null class name parameter, which switches into
While here, improve the
style(9) of the
generated code and add some extra documentation.
Also continue to add more content to the
manpage regarding namespacing and class construction.
Significantly improve manpage documentation.
Have generated code adhere more closely to
This release features lots of internal improvements to code
readability, both in terms of documentation and style.
Most importantly, it's now possible to use
when compiling the output of
count query type and its associated C API
This is essentiallythe
COUNT SQL command.
It has its own role entry like the other query types.
Clean up whitespace and line breaking in the C API output, while
Add boundary values for enumerations.
In the C interface, this is
xxx is the name of the enumeration.
For Java or TypeScript, this is
BITI__MAX on the
Also fix a line termination bug.
When using concatenation (the
concat operator) for a
field with possibly-
null values, wrap the field with a
This prevents the foo = empty || yourval expression, which evaluates
into another empty instead of the desired
Make code and documentation consistent for
Clarify the functionality of -d mode in ort-sqldiff(1).
Also make it work with bitfields and expand its operation for
enumerations so as not to raise false positives during an upgrade.
This means that all binaries prefixed with
are now prefixed with the simpler
kwebapp and code produced by the C API
with roles uses
struct ort and
ort_role instead of the earlier
While here, clean up a lot of documentation, especially in
ort(5), and also simplify several data
Several versions in a row where I screwed up small things found
only after release.
And then again.
(Then one more time.)
Add the string concatenation operator (
Now produces code usable on musl Linux thanks to
Duncaen (alongside the
earlier support for glibc).
Also produces code running properly on FreeBSD.
Stop using a deprecated function in
ksql and use the safe
TypeScript now errors out if
NodeListOf is used instead
HTMLCollectionOf, so use the proper invocation.
Forgot to zero some allocated memory in
Found with valgrind.
Allow kwebapp-sqldiff(1) to
drop table commands.
Fix that custom handlers are invoked for null values if null values
are allowed by the configuration.
fillArrayByClass, which is like
fillArray but invoked over matching classes.
Begin migration (internally) to new name, openradtool.
The former two captures common uses of
hiding or showing an element if there are non-zero elements in the
The latter are convenience methods of wrapping a
invocation with a
the defaults, instead of coming beforehand.
Document this behaviour.
(Before, the order was unspecified.)
Re-organise internals for better library support.
The library isn't exported, but may be (and is) used.
No functional change.
default keyword be handled when emitted
default support for text, emails, and dates.
Make sure that ksql(3) is
configured for foreign keys.
unique statements into account when warning
search statement might return multiple
Also add some examples for using
Fix a crash in kwebapp-xliff(1).
Same as 0.6.1 except a small fix for packaging.
Below is the message from 0.6.1.
Add ability to convert from JSON back into objects.
This uses jsmn,
as it's simple, fast, and easy to bundle directly into the sources.
-J flag in
kwebapp-c-source(1) for details.
Add ability to convert from JSON back into objects.
This uses jsmn,
as it's simple, fast, and easy to bundle directly into the sources.
-J flag in
kwebapp-c-source(1) for details.
Allow configuration files to be broken apart for several applications.
Now, all kwebapp utilities may be invoked
with multiple files per configuration.
This has made kwebapp-sqldiff(1)
and kwebapp-xliff(1) have
slightly awkward syntax for many-many invocations,
but the usual UNIX conventions of standard input and option parsing
The existing calling convention is also still usable.
introduce the concept of a default value that's used in
TABLE statements for new columns.
This currently is only defined for number types.
The default is documented in kwebapp(5).
for consistency (oops!), then add
Have several new class recognised in
fill methods as
xxx-yyy-bits-checked is now recognised for
bits types and it will check input
boxes whose value is covered by the field's value.
xxx-yyy-date, for ISO 8601 formatting of
xxx-yyy-value-checked for checking general
input boxes matching input values.
All of these is documented in the manpage and in the output jsdoc.
stipulate callback function types for the custom format handler.
Fix TypeScript generation for enumerations.
Invert logic for language selection so that language not found gets
Allow a copy mode -c in kwebapp-xliff(1).
Also don't allow tags (the < character) in
Fix specifying language
jslabel for special phrases in
Full translation capabilities!
jslabel strings may be translated out-of-band
into the appropriate
jslabel.xxx labels using the
industry-standard XLIFF 1.2 format.
Just use kwebapp-xliff(1) to
extract translatable strings, edit the content, then merge with the
for dynamic per-language formatting!
make output by putting version information
into its own header file.
Continue adding in TypeScript type information.
Multi-language support: re-tool the
jslabel feature to
allow for multiple languages that detect the HTML (or XML)
document's root language and only produce output for that language.
This uses the
Also warn more about missing labels and be clearer on what to do
when label information isn't found.
unset keyword has been changed to
isunset and the
isnull keyword has also been added.
There is no longer a hard-coded
not given value, and the
class set on null values is now
kwbp-null instead of
The system now supports compiling on Linux and other non-OpenBSD systems.
This has been enabled by bringing up to date with oconfigure
and adding portability shims for password hashing and other tools.
I'm also pleased to announce TypeScript
This is still under development, but now exports proper TypeScript
classes, interfaces, and so on.
Bring up to date with oconfigure
and also use newest safety mechanisms (the
Repair header inclusion (kcgijson.h
requires kcgi.h) for kwebapp-c-source(1).
Bring up to date with newest ksql, which requires
the default role have permission to open the database.
Fully switch over to ksql(3)'s roles for
the header file.
Significantly expand the auditing capability to show both the export
and access to data.
Before, the concepts were too closely linked.
Now, both the JSON and non-JSON output are careful to note which
data is simply accessable, and which may be exported as well (and
via which paths through the code).
Also fix code output by non-role-based systems.
noexport empty statement, which was causing
assertions when invoked in certain situations.
Also start using ksql(3)'s stored
statement functionality, which improves security.
Add partial support for roles within ksql(3) as well, but leave them
unimplemented for the time being.
allowing limits, offsets, and orderings to come after an empty
noexport in a structure's roles to accept an
empty name, which indicates that no fields should be exported.
Reduction of nearly 40% from the file size of the last version!
This is a savings in code size along—the bulk of the files
are often comments.
Support auto-numbering of enumerations.
The scheme is to start at the maximum assigned number plus one or
zero, whichever is greater, and assign incrementally in order of
date pseudo-type that aliases
epoch but uses the date validation input.
LIMIT set goes after
kwebapp-sql(1), and kwebapp-sqldiff(1).
The historical calling conventions of using kwebapp(1) with output flags is no
This tool allows configurations to be audited for role permissions.
It effectively takes a configuration and determines all possible
access (data access, modify, etc.) available to a given role and
produces a report.
Reports can be on the command-line or JSON.
A sample implementation of the JSON file is installed.
comment strings to be associated with roles.
The audit functionality is part of a project generously funded by
CAPEM Solutions, Inc.
delete to operate
See kwebapp(5) for syntax.
Also add transaction call-through functions to the generated C API.
This is necessary in role-based mode since the underlying
ksql API is shielded.
These are documented in kwebapp(1).
Fix a crash when having
limit statements with
Lift the restriction on alias counts (total number of inner joins)
from 26 to 17576.
This can easily be expanded further: the limitation is purely for
distinct field to queries.
This field, in its basic form, enforced the SQL
However, it can also be used for nested fields in a structure,
effectively only returning distinct rows of nested structures.
The canonical example is returning only distinct album fields that
struct foreign key references of a track table.
All of this is documented in kwebapp(5).
Bug-fixes in created C code to account for line-breaking long SQL
(This is an in-progress effort.)
Also add the
unset jslabel statement for bit-fields.
to provide a value if the field exists but has not been set with any
(This is yet to be documented for kwebapp(1) as the feature is still
Lots of new features!
order statement for ordering
It can also act as a way to define single results over non-unique
fields by combining the
search statement with
Allow for limit offsets by extending the
Lastly, introduce the
bit bit index type and the
bits top-level type for working with bit-fields.
All of these are documented in kwebapp(5).
(These will otherwise collide in the C namespace.)
Improve the documentation (function arguments) in the produced C
enum values—this is described
or logical (bit-wise)
operations for search constraints.
Finally allow comments!
These begin at any time with the hash mark and continue to the end
(My motivation was modelines for vim.)
Lastly, and the biggest, allow for having
null foreign keys.
This requires the system to perform extra
statements when the keys aren't
null (which is noted in
the generated API comments on a per-function basis), but overall
it's very useful.
like operator type.
This allows for primitive text searching.
(This will become more advanced in time.)
accept an array of functions for structure-field or structure in
bit type to work with kcgi's new bit
Lastly, add feature that allows an update
statement translate into updating all fields.
Repair the jsdoc by actually
Also add custom callback handling for individual structure/fields.
insert statement within a structure to
generate insertion functions.
Disambiguate the configuration grammar with respect to structure
struct types could specify the linkage from
another field to a foreign key.
Now, the linkage must be specified within the other field itself.
This clears up the situation where a structure and the referring
field had different types.
It also allows all foreign key information (e.g.,
delete modifiers) to exist unambiguously on the
referring field itself.
actdel modifiers on a
foreign key reference.
actdel cascade specifies
Properly wrap comments (at word boundaries) at 70 columns.
Introduce RBAC functionality: the
object defining the roles themselves such that each structure
insert) can be wrapped in a
roles statement limiting access.
When the top-level
roles is defined (even if empty),
ksql database handle is hidden from the API, and
instead an opaque
kwbp is used.
This allows for full role-based access to the database.
Read the kwebapp(5) for more
Also clarified some warnings and errors generated during the link phase.
CFLAGS during configuration and use -Wextra during compilation.
First, clarify how the root DOM element is handled as to whether it can have classes to be filled in.
Second, add a
fillInner() function to handle cases where the root shouldn't be filled in.
Finally, add a
fillArray that fills from an array of objects.
This removes the first element, clears the root element, then clones, fills in, and appends the first element
for all objects.
fillInner() functionality comes in handy to protect nested objects from clobbering each other.
Fix a bug that crept into the last release where iterate/list terms with an empty constraint set resulted in a
WHERE in the generated SQL.
Allow iteration and listing to accept no terms; i.e., they produce all rows for that structure.
split process model to be used when
-Fsplitproc is specified.
sqldiff mode support for enumerations.
This makes sure that enumeration objects are consistent (non-destructive) across configurations.
Add enumerations (
enum) as top-level objects.
This allows the C API to represent integers as sets of constants and makes validation more precise.
eq operator for validation.
(Probably not used for numeric, but useful for strings in some circumstances.)
This is just a string type with special validation.
Add the ability for
update fields to accept a modifier.
This translates into the SQL equivalent to
update foo set x = x + ?.
Right now, only the
dec modifiers are supported, but it's easy to add in new ones.
One-liner so that
unique fields can be foreign keys (but not
Also add the
epoch native type that uses the
time_t type for the C API.
For OpenBSD with 64-bit signed integral time, this is just an alias for
int except for the type
in the API.
Fix bug in recursive reference checker.
This now works properly and allows [properly] for self-referencing structures for native type references.
Add special function to JSON print iterate entries.
These work together with the iterate functions and accept their callback syntax.
Finally fix naming of anonymous query (search, list, iterate), update, and delete functions.
This now uses the basic syntax of
xx is the structure,
is the field, and
op is the operation.
op field disambiguates fields.
(The operation also makes the earlier double-underscore unneeded.)
Also add functions to print JSON arrays if and only if a structure has list iterators specified.
Flag day: use the -O flag to specify output type (hence minor bump).
Also move -v and -j into -Fvalids and -Fjson as part of the new -F flag.
This cleans up a lot of options and leaves the door open for new output formats and configuration options.
Also handle the
foo-xxxx-value class, where
foo is the structure and
xxxx is the
field, for auto-population of form elements.
This completes the chain from HTML form input to filling in HTML DOM trees with CGI script output.
Handle empty insertions by using
DEFAULT VALUES on the insertion.
Accept inequality operators for search, update, and delete constraints.
Add limit clauses to make validation more meaningful.
(See the limit field clause in kwebapp(5).)
Now input values can be limited.
Add full validation sequence required by kcgi(3), specifically
With this in place, applications can push the given structure directly into the HTTP parser without needing to
create the array themselves.
Add field validation routines.
See the -v flag in kwebapp(1).
This uses the field type (and modifiers) to create
kcgi(3) validation routines.
Require a semicolon after
This is future-proofing for options following the
Also added a
noexport field option that disallows fields from being JSON-exported.
Handle empty (
null) values as specified in the configuration when exporting to JSON.
Also, properly handle blob values as base64-encoded strings.
Introduce -j, which also emits JSON output functions using kcgi(3)'s interface.
These complete the chain of exporting data from a database directly to client applications.
Have the singleton search function (e.g.,
search foo: name bar) use the operator
get in generated
Before, these would be (following the example)
This is more consistent with the other search functions.
Renamed the -h flag to -C.
(Source in lowercase, header in uppercase.)
Initial stable internals.
I'm still going to change the generated API function names before adding features, but the system is already in