Feature availability: This article discusses Custom Tests, which may require an upgrade or add-on. Refer to Vanta Plans and Pricing for details.
Custom Tests lets Vanta users build and manage their own automated tests, map them to controls and frameworks, and validate security program requirements using their own integration data. If Custom Tests is not available on your plan, the Documents page can be used for similar purposes.
Ways to Create Custom Tests
All of the options below are part of the Custom Tests feature. You can create custom tests in one of three ways:
Configure a test parameter (on select Vanta-built tests)
Copy an existing test (Vanta-built or custom)
Create a Custom Test
Configure a Test Parameter
Some Vanta-built tests allow you to customize a parameter like the number of days before an account should be deactivated or the branch name for production repositories.
Go to the Tests page
Open the test you’d like to configure
Select the Evidence tab (formerly called Source Data)
Adjust the available field(s) to match your policy
Please note: Only a limited set of Vanta-built tests support configurable parameters. If you think a test would benefit from customization, you can provide feedback to your Customer Success team.
Copy an Existing Test
Some Vanta-built and custom tests can be copied and modified to suit your needs.
From the Tests page, select the test you want to copy
Click the Evidence tab (formerly called Source Data)
Choose Copy test
Edit the name, logic, and details to create a new test
You can also copy your own existing custom tests to make variants.
Please note: When you copy a test in Vanta, there are a few important things to keep in mind. The new test won’t carry over any control mappings, so you’ll need to manually add those after copying. You’ll also need to assign an SLA category and test owner. And because custom tests can’t be edited after they’re created, any future changes will require making a new copy of the test.
Create a Custom Test
Single-resource custom test
From the Tests page, select + Create custom test
Add:
Test name
Description
How to fix/remediate instructions
From the drop-down, select the integration the custom test will be associated with, you cannot currently create a custom test that is not tied to an integration
Use the simple logic builder to build the test
Select Create
2-resource custom test
Overview
Custom tests now let you pull and join data from two resource types: a primary resource plus one secondary. The secondary can come from the same integration or a different one. You define a join condition (for example, User.email == GitHubMembership.user_email) that tells the Test Builder how to correlate records; the builder fetches and combines matching records so fields from both resources are available for scoping and pass/fail evaluation.
This enables tests that require correlated data across resources (for example, IdP user data joined to cloud access records). The Test Builder currently supports exactly one primary + one secondary resource per custom test (maximum of 2 data sources).
Setup guide
The creation wizard in Vanta has four steps: Test data, Match data sources, Evaluation logic, and Test details.
Step 1: Test data
From the Tests page, click + Create test
Select the integration and primary resource type
Click +Add data source to add a second integration and resource type
Scoping conditions (optional): Under each selected data source, you can click + Add condition to narrow the resources in scope for that data source. You can also click Show available fields to browse the fields you can filter on.
Note: Scoping conditions on the secondary resource are applied before the join. Only secondary resources that pass their scoping filter will be available for matching with the primary resource.
Click Continue
Step 2: Match data source
This step only appears when you have two data sources selected
Pick a property from each resource type to define how records should be matched. The join is equality-based (i.e. a primary record matches a secondary record when the selected fields have the same value).
Important join behavior to be aware of:
Join matching is type-strict: the number
1will not match the string"1", and the booleantruewill not match the string"true".Null, undefined, or missing values on either side will not produce a match.
Only scalar values (strings, numbers, booleans, and dates) are supported as join fields. Objects and arrays will not match.
If a primary resource has no matching secondary resources, it will still be evaluated — but with an empty set of secondary data. How this affects the outcome depends on your evaluation logic (see below).
Click Continue
Step 3: Evaluation logic
This step has 3 sections:
Test outcome (reducer strategy) - Choose whether the overall test passes based on all or any primary resources:
All - The test passes only if every in-scope primary resource passes the evaluation logic. If any single resource fails, the overall test fails.
Any - The test passes if at least one in-scope primary resource passes. Resources that don't pass are marked as N/A rather than Fail.
Resource outcome (pass/fail rules) - Use the logic builder to define the conditions that determine whether each individual primary resource passes or fails. The field picker shows fields from both the primary and secondary resources, so you can reference either in your conditions. When referencing secondary resource data in your conditions, you can also specify whether the condition should be met by any or all of the matched secondary resources for a given primary.
Evaluation preview - Review the evaluation preview to validate the join and evaluation behavior against your actual data. The preview displays: the overall test outcome (pass/fail), counts of total primary resources matched, passing, failing, and N/A, and a table of individual resource-level outcomes showing how each primary resource would be evaluated.
Tip: If a primary resource has no matching secondary records after the join, a condition checking for "at least one matching secondary where…" will cause that primary to fail. Keep this in mind when designing your logic.
If no resources appear in the preview, try adjusting your data source selections or scoping conditions.
Click Continue
Step 4: Test details
Finally, complete the Test details section by creating a Test name, adding a Description, and providing Vanta with Remediation instructions (optional).
Click Save test.
After creating a 2-resource test
After saving, be sure to:
Map the test to controls (this is not automatic – see “Mapping the test to controls” section below)
Assign an SLA category
Assign a test owner
Deactivate the original test if this replaces one you no longer need
Mapping the Test to Controls
From the Tests page, select the Custom tab
Search for your test
Select the test
Open the Controls tab
Select Add control
Choose the controls you would like mapped to this test, and click Add
Finalizing and Managing Custom Tests
After creating or copying a test, be sure to:
Map it to controls (this is not automatic)
Assign an SLA category
Assign test owners
Deactivate the original test if you no longer need it
These final steps ensure the test integrates into your program and is monitored properly.
Please note: Custom tests can now support both automated checks and manual inputs. This feature allows creating unique tests to validate security program elements either automatically or through manual evidence when required.