Updated March 2018
We open-sourced a lot more libraries since this first post, including our popular OCaml Linter - take a look on our Github page.
Original post:
The vast majority of the Cryptosense code base is built using OCaml. We're excited to announce that we're releasing a couple of the OCaml libraries we developed as free software: records and enumerators. Here we'll describe what the two libraries do and what we use them for.
Records
Most languages have a notion of record types: a collection of fields which each have a name and a type. The classic example is a point type with three integer fields x, y and z. In OCaml this is written:
A common problem however is that this notion is often not first class. For example, one can write a function that takes a point, a value and sets p.x to this value. One can also do the same for y:
But since fields are not values it's not possible to make a function that can set either x or y. Several solutions exist, the most popular being Van Laarhoven lenses, which abstract away the concepts of "getting" and "setting" things. Lenses can get a bit hairy, so we're using a simpler solution.
Our records library turns record types and fields into runtime values. This example, taken from our README, shows how we handle this.First, we create a layout. That's a value representing a record type.
Then, we can allocate and use a record value.
The last line outputs p in JSON format:
We have two use cases for this in our code base. First, it simplifies the way we deal with JSON data. This basically builds of_json and to_json functions automatically. The ppx_deriving_yojson library can also be used to this end, but sometimes more control is needed (in particular when we don't control the JSON format).Also, it makes it possible to create lists of heterogeneous records. For example, our app tracer parses a stream of JSON values identified by a command name (such as "encrypt" or "sign"), each associated to a record with the command's parameters. This is all bound together with GADTs, but that's another story!
Enumerators
When working with large data structures, you may need a way of controlling how your program allocates memory. Typically, inefficiencies can result from pieces of code like the following:
This constructs an horizontal cross from a straight line and successive spatial transformations, where a figure is represented by an array of point records. Using this code can be problematic because intermediate arrays are allocated at each step. With a large amount of points, it may take a significant amount of time or even fill up your computer's memory.To deal with these issues, we have built the enumerators library. An enumerator represents a finite sequence of elements, like an array does, but you can apply transformations to enumerators and combine them together in a lazy way, that is, by computing them only when needed and without storing intermediate values. Here is an improved version of the plotting of the cross:
To make it even more efficient, you could store the points using the records library mentioned previously.In addition to the functions used in this post, the library provides you with various less common combinators. At Cryptosense, we use functions such as `product` and `subset` to generate test cases for our Analyzer and Compliance Tester.
Get in touch
We'd love to hear about it if you use one of these libraries in an OCaml project. Also, we're recruiting.