TLDR

• I will briefly introduce the scope of the problem, and why I needed something like this
• Copyable gist file included
• How I implemented it
• Tests (also in a gist)

Mustache templates and JSON schema and us

if you are reading this, I hope you are familiar with Mustache templates or JSON schema, but I will introduce them shortly.

Mustache is a popular templating system. You can write text, and add placeholders/variables to it, which are later replaced with values. It's a very simple and powerful concept. An example:

1const Mustache = require('mustache');
2
3const view = {
4  title: "Joe",
5  calc: () => ( 2 + 4 )
6}; // params for the template
7
8const output = Mustache.render(
9  "{{title}} spends {{calc}}", // this is the template
10  view,
11  );
12// output: "Joe spends 6"

JSON schema is a way of describing the structure of a JSON object. It's an another very powerful concept, and it's also widely used. There are many tools for JSON schema, such as ajv as a validator, or json-schema-faker to generate fake data based on a schema, or json-schema-form to generate a form based on a schema. You have probably seen a JSON schema before, because openApi (swagger) uses it to describe the structure of the request and response bodies of an API. A simple schema for our previous view might look like this:

1{
2  "type": "object",
3  "properties": {
4    "title": {
5      "type": "string",
6    },
7    "calc": {
8      "type": "integer",
9    }
10  }
11}

My main problem with Mustache is that it can't validate the data, and I can't easily tell if I "broke" the "api" between the previous template and the new one. Also, in the previous example {} would be a valid view, and the output of that view is spends which is far from what I call correct, or what I expect.

You know what would be super cool? If I could generate a form for my template.

Implementation

Types

I have a scala background. I love types. Also, for JSON schema we want types too. Mustache has four operators (or at least I only care about these four):

• {{, {{& these means "inline variable" (the first one will escape the value, the second one won't)
  • we can have string or number as a type
• {{# means "iterate over an array" or "if it is not falsy" (yes it has two meanings)
  • in both cases this starts a block, and the block ends with an {{/
  • if it is an array, the block is repeated for each element of the array, and the variables are set to the current element
  • if we don't use a variable inside the block, we can have any type
  • if we use a variable inside the block, we can have an array of that type
• {{^. This is the same as {{#, but it will be executed if the value is falsy (starts a block)
  • we can have any type

One more thing, that we can have a template like {{test.a}} and {{test.b}}, where test is an object, and a and b are (inlined) properties of that object.

So we can have the following types:

1export type AllowedSchemas =
2  | typeof SingleField
3  | ArraySchema
4  | ObjectSchema
5  | typeof AnyField;
6const SingleField = { type: ["string", "number"], nullable: false };
7const AnyField = {
8  type: ["string", "number", "object", "array", "boolean", "null"],
9};
10type ArraySchema = {
11  type: "array";
12  items: AllowedSchemas;
13  nullable: false;
14  minItems: 0 | 1;
15};
16type ObjectSchema = {
17  type: "object";
18  properties: Record<string, AllowedSchemas>;
19  required: string[];
20  nullable: false;
21  additionalProperties: true;
22};

Only the AnyField can be nullable, and the ObjectSchema can have additional properties. For the sake of completeness, we will need to require all the properties of an object (which is not nullable), and we will need at least one element in an array if it is not used as conditions.

So given the following template I want the list to have at least one element in it:

1List
2{{#list}}
3 - {{.}}
4{{/list}}

But in this case I would allow an empty list:

1{{#list}}
2List
3{{/list}}
4{{#list}}
5 - {{.}}
6{{/list}}

Or here an empty array in repo is fine too:

1{{#github}}
2You can check the following github repo: {{repo}}
3{{/github}}
4{{^github}}
5We dont have a link for this repo.
6{{/github}}

Implementation

I don't want to go into fine details, because I have a gist with the code in it, but I will give a short overview.

First, we need to parse the template, and keep the "interesting" parts of it. Convert these parts to a "naive" representation, and then merge these parts into a single schema.

1const variableSpans = ["name", "&", "#", "^"];
2export const schemaGenFromMustacheTemplate = (
3  template: string
4): AllowedSchemas => {
5  const parsedTemplate = Mustache.parse(template);
6
7  const params = parsedTemplate.flatMap((e) => {
8    if (variableSpans.indexOf(e[0]) > -1) {
9      return [schemaOfParam(e)];
10    }
11    return [];
12  });
13
14  const innerSchema = mergeSchema(params);
15  return {
16    properties: innerSchema,
17    type: "object",
18    nullable: false,
19    required: Object.entries(innerSchema)
20      .filter(([, v]) => {
21        return !(
22          ("nullable" in v && v.nullable) ||
23          (Array.isArray(v.type) && v.type.indexOf("null") !== -1)
24        );
25      })
26      .map(([k]) => k),
27    additionalProperties: true,
28  };
29};

The only tricky part here is the list of required fields. We don't want to require a field that is nullable or can be null.

schemaOfParam is a function with a switch case, that converts name and &, to SingleField, and ^ to AnyField. # is converted like this:

1// we try to detect the sub-variables
2if (Array.isArray(param[4])) {
3    const spans: TemplateSpans = param[4];
4    const schemaList = spans
5      .flatMap((p) => [schemaOfParam(p)])
6      .filter(nonEmptyObject);
7    const itemDef = mergeSchema(schemaList);
8    // if we found any named one, this is an array of objects with the sub-vars
9    if (Object.keys(itemDef).length > 0) {
10      return {
11        [param[1]]: {
12          type: "array",
13          items: {
14            type: "object",
15            properties: itemDef,
16            required: Object.keys(itemDef),
17            nullable: false,
18            additionalProperties: true,
19          },
20          nullable: false,
21          minItems: 1,
22        },
23      };
24    }
25    // if we not found any named one this is an array of SingleFields
26    if (schemaList.length > 0) {
27      return {
28        [param[1]]: {
29          type: "array",
30          items: SingleField,
31          nullable: false,
32          minItems: 1,
33        },
34      };
35    }
36}
37// if we don't have any sub-variable block, this is an if condition so an AnyField
38return {
39    [param[1]]: AnyField,
40};

The hardest part both to implement and to understand is the mergeSchema function. It tries to construct objects from "a.b.c" like named variables. (Dropping the "."s.) It takes care if the same variable is used in different places, and if it is used as a condition or not. (Allowing minItems to be 0.)

Tests

This is an easily testable function, so I wrote some tests for it. The idea is to have a template and an input data, generate the schema from the template, validate the input data with it, and check if the validation matches the expected result or not. With code;

1interface TestGen {
2  name: string;
3  template: string;
4  schema: Record<string, any>;
5  expectedError: string | null;
6}
7
8const runTest = (testGen: TestGen) => {
9  test(testGen.name, () => {
10    const schema = schemaGenFromMustacheTemplate(testGen.template);
11    // console.log(JSON.stringify(schema));
12    const ajv = new Ajv({ allowUnionTypes: true, verbose: true, strict: true });
13    const validate = ajv.compile(schema);
14    const res = validate(testGen.schema);
15    if (testGen.expectedError === null) {
16      if (!res) console.log(validate.errors);
17      expect(res).toBeTruthy();
18      expect(validate.errors).toBeNull();
19    } else {
20      // console.log(validate.errors);
21      expect(res).toBeFalsy();
22      expect(validate.errors?.[0].message).toEqual(testGen.expectedError);
23    }
24  });
25};
26const testDescriptionsBasic: TestGen[] = [
27  {
28    name: "no param trutly",
29    template: "hello",
30    schema: {},
31    expectedError: null,
32  },
33];
34describe("mustache", () => {
35  testDescriptionsBasic.forEach((t) => runTest(t));
36});

Conclusion

I hope this has been useful for you. I think this is a good way to validate your mustache templates, and it was not that hard to implement. I was shocked that no one else published code for this before, so I hope this will help someone in the future.

If you want to generate forms from your mustache templates, you can do that with the convertToUISchema function also published in the gist.