Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/program-boilerplate/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
dist
dist
*.tsbuildinfo
8 changes: 8 additions & 0 deletions packages/program-boilerplate/.oxfmtrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"$schema": "./node_modules/oxfmt/configuration_schema.json",
"ignorePatterns": [],
"printWidth": 80,
"sortImports": {
"newlinesBetween": false
}
}
16 changes: 16 additions & 0 deletions packages/program-boilerplate/.oxlintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["typescript", "unicorn", "oxc"],
"categories": {
"correctness": "error",
"suspicious": "error"
},
"options": {
"typeAware": true,
"typeCheck": true
},
"rules": {},
"env": {
"builtin": true
}
}
178 changes: 18 additions & 160 deletions packages/program-boilerplate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,22 @@ programs.

Here is the basic architecture of the SaaSquatch program engine:

![](media/arch.png)

## Template

Each program has a template, written in JSON. The template is often referred to as the
"program schema", although this is technically incorrect as it is not a real schema. The
template defines several aspects of the program, including its rules, rewards, emails,
widgets, requirements, and more. The program template is stored in Contentful as a JSON
field in the larger "program" content type. The whole entry including the other fields is
known as the "program template". Program templates are retrieved and used by the Java
known as the "program template". Program templates are retrieved and used by the Java
backend and are never directly touched by the program. Programs can self-modify their
templates when activated by customers. This is called Introspection and will be discussed
in further detail below.

## Program Logic / Behavior
In addition to the program template, all programs contain their own business logic. This is
simply some NodeJS code that runs on Heroku. The program logic is completely stateless

In addition to the program template, all programs contain their own business logic. This
is simply some NodeJS code that runs on Heroku. The program logic is completely stateless
and only depends on the input given when it is "triggered". The different trigger types
will be discussed below.

Expand All @@ -38,9 +38,10 @@ are low-level tech specs that are used for unit testing. These also live in the
`blackbox-testing` repo in a sub-folder called `unit`.

SaaSquatch program logic consists of three major components (triggers):
* General program trigger handlers ([`PROGRAM_TRIGGER`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L39))
* Program introspection handler ([`PROGRAM_INTROSPECTION`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L54))
* Program validation handlers ([`PROGRAM_VALIDATION`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L66))

- General program trigger handlers ([`PROGRAM_TRIGGER`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L39))
- Program introspection handler ([`PROGRAM_INTROSPECTION`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L54))
- Program validation handlers ([`PROGRAM_VALIDATION`](https://github.com/saasquatch/program-tools/blob/master/packages/program-boilerplate/src/types/rpc.ts#L66))

Combined, these components form a program. All three triggers are optional and aren't
necessarily implemented by all programs.
Expand All @@ -49,14 +50,13 @@ necessarily implemented by all programs.

A general program trigger is one of the following

* `AFTER_USER_CREATED_OR_UPDATED` Triggered after a user is created or updated ("upsert")
* `AFTER_USER_EVENT_PROCESSED` Triggered after a user event has been processed by the
backend
* `REFERRAL` Triggered when a referral is created or updated
* `SCHEDULED` Triggered on a set schedule defined by the program template or during
introspection
* `REWARD_SCHEDULED` ??

- `AFTER_USER_CREATED_OR_UPDATED` Triggered after a user is created or updated ("upsert")
- `AFTER_USER_EVENT_PROCESSED` Triggered after a user event has been processed by the
backend
- `REFERRAL` Triggered when a referral is created or updated
- `SCHEDULED` Triggered on a set schedule defined by the program template or during
introspection
- `REWARD_SCHEDULED` ??

### Program Introspection

Expand All @@ -77,8 +77,8 @@ should be satisfied by the tenant before the program launches. The requirements
block the launch if they are not satisfied, but it is recommended to complete them. All
requirements can be automatically verified by the programs based on a GraphQL query.

In the program template or during introspection, the program requirements are added to the
template. Each requirement includes a key, name, query, long description and other
In the program template or during introspection, the program requirements are added to
the template. Each requirement includes a key, name, query, long description and other
fields (see `types/rpc/ProgramRequirement`). The queries defined here will be executed by
the backend and the results sent to the programs for validation. Based on the results of
the query, the program will return one or more results indicating the status of the
Expand All @@ -87,145 +87,3 @@ validation along with a message.
Since the program requirements reside in the template, they can be modified by the
introspection trigger. This means that requirements can be added/removed or modified
depending on the rules of the program.

# Creating a new program
If you are looking for instructions on how to modify an existing program, skip to the
section on the [program development workflow](#Program-development-workflow).

Before you create a new program you will need the following tools installed on your
computer:

* NodeJS / npm
* Heroku CLI (logged in with permission to access the `saasquatch-webtasks` team)

All of the commands in the tutorial, are written for a UNIX shell (MacOS/Linux). If
you're on Windows, good luck!

## Set up the program code on your local machine
Under the `programs` folder, create a new folder for your program:
```
mkdir <my-program>
cd <my-program>
```
Replace `<my-program>` with your program name for the rest of this tutorial. Program
names are kebab case by convention.

Initialize a new npm module:
```
npm init
```
You can reference the other programs for what to answer in the interactive prompt.

Add a couple scripts to your `package.json`
```json
"scripts": {
"start": "node dist/<my-program>.js",
"start:dev": "nodemon dist/<my-program>.js",
"build": "tsc --strict"
}
```

After your npm module is initialized, you may want to install some dependencies for your
program. There are a few that are shared by pretty much every program:
```
npm i @saasquatch/program-boilerplate
```
```
npm i -D typescript @types/node @types/express nodemon
```

After this you will need to setup your `tsconfig.json`. Unfortunately sharing a
`tsconfig` between programs is impossible due to the Heroku deployment process.
Copy/paste one from one of the other programs (they should all be the same).

Create your source tree:
```
mkdir -p src/schema
touch src/schema/<my-program>_schema.json
touch src/program.ts src/<my-program>.ts
```
The program template will be stored under `schema/<my-program>_schema.json`. The `schema`
naming is simply a legacy decision, perhaps in the future it could be changed to
`template/<my-program>_template.json`.

Having `program.ts` and `<my-program>.ts` files in the source root is standard for newer
programs. `program.ts` should export a `Program` type with the relevant handlers, and
`<my-program>.ts` should be the entry point for running the program on Heroku that starts
the Express server. You can take a look at the referral program for examples if you are
unsure.

The rest of the program structure is a matter of personal taste; take a look at the
referral program for an example to follow if you are unsure.

To get your program started, fill the source files with some boilerplate:

`program.ts`
```typescript
import {Program} from '@saasquatch/program-boilerplate';

export const program: Program = {
AFTER_USER_CREATED_OR_UPDATEDL: undefined,
AFTER_USER_EVENT_PROCESSED: undefined,
REFERRAL: undefined,
PROGRAM_INTROSPECTION: undefined,
SCHEDULED: undefined,
REWARD_SCHEDULED: undefined,
PROGRAM_VALIDATION: undefined,
};
```

`<my-program>.ts`
```typescript
import {program} from './program';
import {webtask, getLogger} from '@saasquatch/program-boilerplate';

const logger = getLogger();
const port = process.env.PORT ?? 3000;

webtask(program).listen(port, () => {
logger.notice(`My program running on port ${port}`);
});
```

Running `npm build && npm start` should now run your program locally (useless for now,
but a good test to make sure things are working).

## Set up your program template
As mentioned previously, your program also needs a template which defines its rules,
rewards, emails etc. You could try to write your template from scratch, but it would
likely be easier to copy/paste one from another program and edit it accordingly. You can
probably skip this step and come back to it later.

## Provisioning cloud infrastructure
Next it is time to provision your cloud infrastructure that will host your program
components.

### Creating a Heroku pipeline
In the Heroku console, switch to the `saasquatch-webtasks` team. Select `New` -> `Create
new pipeline`. The name of the pipeline should match the name for your program that was
chosen in the previous step. Do not connect to GitHub.

Next, add the staging and prod apps to the pipeline by clicking `Add app` -> `Create new
app...` under the respective staging and production pipeline stages. By convention, the
app names are `<my-program>-staging` and `<my-program>-prod`.

In each Heroku app add `APP_BASE` as a configuration variable and set it to the path of
the program in the SaaSquatch core repo.

#### Setting up the Heroku app
Now that your apps are created, you can configure them to run your program. It's easiest
to do this from the command line. The first thing you will want to do is use the
`heroku.sh` script to set up the git remotes.

From the `programs` folder, run:
```
./heroku.sh setup-remotes
```
This will set up the git remotes for making deployments to the various programs. It's OK
if you get a bunch of `fatal: remote <some_remote> already exists.` errors.

**WARNING**:
The `setup-remotes` command will only work if the folder name for your program matches
the names of the Heroku programs, ie. you have a folder: `my-program` and Heroku programs
`my-program-staging` and `my-program-prod`.

30 changes: 19 additions & 11 deletions packages/program-boilerplate/__tests__/jsonata.test.ts
Original file line number Diff line number Diff line change
@@ -1,37 +1,45 @@
import * as assert from "node:assert";
import { describe, test } from "node:test";
import jsonata from "jsonata";
import { safeJsonata, timeboxExpression } from "../src/jsonata";
import { safeJsonata, timeboxExpression } from "../src/jsonata.ts";

// oxlint-disable typescript/no-floating-promises

describe("#timeboxExpression", () => {
const infExpr = jsonata("( $inf := function(){$inf()}; $inf())");

test("infinite loops throw error", () => {
test("infinite loops throw error", { timeout: 7000 }, () => {
timeboxExpression(infExpr);

let error: any;
expect(() => {
assert.throws(() => {
try {
infExpr.evaluate(undefined);
} catch (e) {
error = e;
throw e;
}
}).toThrow();
expect(error!.code === "U1001" || error!.code === "U1002").toBe(true);
}, 7000);
});

assert.strictEqual(
error!.code === "U1001" || error!.code === "U1002",
true,
);
});
});

describe("#safeJsonata", () => {
const infExpr = "( $inf := function(){$inf()}; $inf())";

const expr = "( event.key = 'purchase' ? 111 )";
const input = { event: { key: "purchase" } };
test("infinite loops do not throw, but still exit", () => {
expect(() => {
test("infinite loops do not throw, but still exit", { timeout: 7000 }, () => {
assert.doesNotThrow(() => {
safeJsonata(infExpr, undefined);
}).not.toThrow();
}, 7000);
});
});

test("jsonata is evaluated as normal", () => {
expect(safeJsonata(expr, input)).toBe(111);
assert.strictEqual(safeJsonata(expr, input), 111);
});
});
19 changes: 8 additions & 11 deletions packages/program-boilerplate/__tests__/logger.test.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
import { getLogger, setLogLevel } from "../src/logger";
import * as assert from "node:assert";
import { describe, test } from "node:test";
import { getLogger } from "../src/logger.ts";

// oxlint-disable typescript/no-floating-promises

const logger = getLogger("notice");
describe("#getLogger", () => {
test("first call initializes the logger at the given level", () => {
expect(logger);
expect(logger.level).toBe("notice");
assert.ok(logger);
assert.strictEqual(logger.level, "notice");
});

test("logger is only initialized once", () => {
expect(getLogger("warn")).toBe(logger);
});
});

describe("#setLogLevel", () => {
test("sets the log level of an initialized logger", () => {
setLogLevel("crit");
expect(logger.level).toBe("crit");
assert.strictEqual(getLogger("warn"), logger);
});
});
Loading
Loading