Skip to main content

Automated Test History

S
Written by Shannon DeLange
Updated today

Understanding how automated tests behave in Vanta is key to confidently managing your compliance posture. Depending on whether a test falls under Vanta’s SLA (Service Level Agreement) tracking system, the history view may vary; some tests track the overall pass/fail status over time, while SLA-enabled tests focus on how long individual failing entities have remained unresolved. This distinction helps teams prioritize remediation efforts based on urgency and duration of non-compliance.

History Tab - SLA Category Tests

The History tab on an automated test’s detail page displays past test results, but how that history is shown depends on whether the test is part of Vanta’s SLA (Service Level Agreement) system. Many automated tests are linked to an SLA category, which means Vanta doesn't just track whether the test passes or fails as a whole; instead, it monitors how long individual failing items (such as users, devices, or resources) remain non-compliant. This helps teams focus on resolving long-standing issues, not just the most recent ones.

Screenshot 2025-05-05 at 4.58.48 PM.png

If you are pursuing multiple frameworks at once, the status of this test may vary by framework over time. The provided filters can help you narrow down the results.

  • Select the dropdown under Remediation history to select the desired resource.

Screenshot 2025-05-05 at 5.02.03 PM.png

Additional filters

  • With due date / without due date

  • Remediation status

    • Within SLA

    • SLA Missed

Screenshot 2025-05-05 at 5.03.31 PM.png

History Tab - Non-SLA Category Tests

Some automated tests in Vanta don’t evaluate individual entities (like users or devices). Instead, they track whether the test as a whole is passing or failing. For these tests, the History tab shows a horizontal timeline bar:

  • Green segments indicate periods when the test was passing

  • Red segments indicate when the test was failing

  • Gray segments show when the test wasn’t run or wasn’t applicable during that time

This visual helps you quickly understand how consistent your compliance has been for a specific control over time.

Audit Page: History in the Automated Test Evidence View

To Access the audit page:

  • Select Audits from the left-hand navigation panel

  • Open the audit

  • Select the evidence you would like to view the details of

Screenshot 2025-05-05 at 5.22.44 PM.png

The “View History” in the Details tab of the evidence drawer provides a more detailed look at the test’s performance over time. This allows you to demonstrate consistent compliance or explain any remediation work done if issues were previously flagged. These views give auditors the context they need to validate ongoing control effectiveness. The "View history" button will store history throughout the audit period.

  • For automated SLA system tests, we render a table view showing entities not remediated by the SLA deadline.

  • We will show a line graph if a test does not use the SLA system.

Audit Page: Audit Activity

The Audit Activity tab inside the automated test’s evidence drawer shows a summary of when the test was reviewed or used as audit evidence. This helps track activity during your audit period.

Screenshot 2025-05-05 at 5.16.35 PM.png

Automated Test Evidence List Determination

There are a few factors that can influence whether a test appears in a particular audit:

A Note on Test Applicability

  • To see an automated test appear in the evidence view or on the tests page, the test must be considered applicable to your Vanta setup. This means Vanta needs the right data to run the test. For example, if a test relies on data from an integration you haven’t connected (like AWS or GitHub), Vanta will automatically mark that test as not applicable. The same applies if you’ve scoped out certain assets or frameworks using Vanta’s integration scoping or framework scoping features. This helps keep your view focused and prevents irrelevant tests from cluttering your dashboard.

Deleting an integration is a permanent action that can impact your compliance visibility. If you remove an integration, any Vanta-built automated tests relying on data from that connection may disappear from your tests page and audit evidence view. Be sure you understand what’s connected before making changes.

Choosing an Audit Window that Includes Past Dates

When creating an audit in Vanta, it's essential to understand how test history is tracked, especially if your audit window includes dates in the past.

For example, if you create an audit on May 1st and set the audit window to span April 1 to May 30, some test history, like results from April, will already exist and may appear in your audit evidence. This can lead to two different types of historical behavior, depending on how you're viewing the test:

  • Audit Activity Tab: Tracks Usage from the Date the Audit Is Created. This tab logs when a test is actively used for audit purposes. It only starts tracking when the audit is created (in this case, May 1).

So, even if the test failed on April 6, you won’t see audit activity recorded for that event because it happened before the audit was created.

  • “View History” Button: Displays full test results within the audit window. This button provides the actual test results across the entire audit period, including dates before the audit was created.

So if your audit runs from April 1 to May 30, you’ll still see failing test results from April 6, even though the audit was only created on May 1.

This allows auditors to view the compliance status throughout the whole window, not just from the point of audit creation.

Please note: If the test becomes inapplicable (due to scoping or integration removal) on or before May 1, it won’t appear in the audit evidence view, even if it failed earlier in the audit window.