Thumper

Thumper is a traffic generator for SpiceDB and Authzed. It will issue CheckPermission, Read/Write Relationships, ExpandPermissionTree and LookupResources to a SpiceDB or Authzed endpoint, and exposes Prometheus metrics about those operations.

Setting up

Run the migration script required by your thumper scripts against your permission system.

thumper migrate --endpoint grpc.authzed.com:443 --token t_some_token --insecure false ./scripts/schema.yaml

Running

Running against an Authzed instance

thumper run --endpoint grpc.authzed.com:443 --token t_some_token --insecure false ./scripts/example.yaml

Running against a local SpiceDB

thumper run --token presharedkeyhere ./scripts/example.yaml

Configuring

Changing the QPS

By default, Thumper will run a single goroutine, which will issues calls once per second, at rates configured in thumperrunner/definitions.go.

To increase the number of calls issued, the THUMPER_QPS environment variable or --qps flag can be used:

thumper run --token presharedkeyhere --qps 5

The above will spawn 5 goroutines, which will each issue calls once per second.

Script Format

Thumper config files are yaml files with go template preprocessing supported. The final yaml generated by the templates must validate with the schema in schema.yaml.

Example

name: create org, tenant, and add client
weight: 1
steps:
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  expectNoPermission: true
  consistency: AtLeastAsFresh
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 0
  consistency: AtLeastAsFresh
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: {{ .Prefix }}organization:org_{{ randomObjectID }}
    subject: {{ .Prefix }}platform:plat_{{ randomObjectID }}
    relation: platform
  - op: TOUCH
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}organization:org_{{ randomObjectID }}
    relation: organization
  - op: TOUCH
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}client:client_{{ randomObjectID }}#token
    relation: writer
  - op: TOUCH
    resource: {{ .Prefix }}client:client_{{ randomObjectID }}
    subject: {{ .Prefix }}token:t_{{ randomObjectID }}
    relation: token
    caveat:
      name:  {{ .Prefix }}caveat_name
      context:
        bool_field: true
        string_field: value
        int_field: 4
        float_field: 3.14159
        null_field: null
        nested_object:
          abc: def
        nested_list:
        - 1
        - 2
        - 3
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  consistency: AtLeastAsFresh
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: permission_with_caveat
  consistency: AtLeastAsFresh
  context:
    field_name: field_value
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 1
  consistency: AtLeastAsFresh
- op: WriteRelationships
  updates:
  - op: DELETE
    resource: {{ .Prefix }}organization:org_{{ randomObjectID }}
    subject: {{ .Prefix }}platform:plat_{{ randomObjectID }}
    relation: platform
  - op: DELETE
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}organization:org_{{ randomObjectID }}
    relation: organization
  - op: DELETE
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}client:client_{{ randomObjectID }}#token
    relation: writer
  - op: DELETE
    resource: {{ .Prefix }}client:client_{{ randomObjectID }}
    subject: {{ .Prefix }}token:t_{{ randomObjectID }}
    relation: token
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 0
  consistency: AtLeastAsFresh
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  expectNoPermission: true
  consistency: AtLeastAsFresh

Types

The following common types are used in various operations:

Type	Example(s)	Used In
Permission/Relation Name	reader, writer, view	*
Object Reference	objecttype:objectid	CheckPermission, ExpandPermissionTree, LookupSubjects, WriteRelationships
Subject Reference	subjecttype:subjectid, subjecttype:subjectid#optionalrelation	CheckPermission, ReadRelationships, DeleteRelationships, ExpandPermissionTree, WriteRelationships
Object Type	objecttype	LookupResources, LookupSubjects
Object Filter	objecttype, objecttype:objectid	ReadRelationships, DeleteRelationships

Go Template Properties

The following properties are available to be used from within go templates:

enumerate(length)

This function will generate an array with the specific length filled with the natural numbers. This array can be ranged over to repeat a script fragment a number of times with a varying identifier.

Example:

name: many checks
steps:
{{- range $i := enumerate 100 }}
- op: CheckPermission
  resource: document:{{ $i }}
  subject: user:stacy
  permission: read
{{- end }}

randomObjectID

This function returns a different random object ID per worker allowing many workers to work on the same flow in parallel. Because this function returns a randomObjectID per worker, it will require you to load a set of scripts for every worker. This can significantly increase the Thumper initialization time for high QPS tests.

Example:

name: check permissions on random document
weight: 1
steps:
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: document:{{ randomObjectID }}
    subject: user:stacy
    relation: reader
- op: CheckPermission
  resource: document:{{ randomObjectID }}
  subject: user:stacy
  permission: read

Prefix

This parameter contains the value of the --prefix command line parameter followed by a /, and can be used to isolate schemas and data between different instances of thumper.

Example:

name: check permissions on random tenant
weight: 1
steps:
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: {{ .Prefix }}document:1
    subject: {{ .Prefix }}user:stacy
    relation: reader
- op: CheckPermission
  resource: {{ .Prefix }}document:1
  subject: {{ .Prefix }}user:stacy
  permission: read

IsMigration

This parameter contains a boolean that specifies whether the script is being run under the thumper migrate command. This can be used to write a script that contains both a migration and the actual test scripts.

Example:

{{- if .IsMigration }}
---
name: write schema
steps:
- op: WriteSchema
  schema: |
    definition user {}
    definition document {
      relation reader: user
    }
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: document:1
    subject: user:stacy
    relation: reader
{{- else }}
---
name: check permissions
weight: 1
steps:
- op: CheckPermission
  resource: document:1
  subject: user:stacy
  permission: reader
{{- end }}