Skip to main content

Security Testing Orchestration (STO) FAQs

This article addresses some frequently asked questions about Security Testing Orchestration (STO).

What is a security scanner?​

A security scanner is a tool that scans a software target and detects security issues and vulnerabilities. Examples of target types include code repos, container images, and websites. Developers, DevOps engineers, SecOps engineers, and others often include scanners at different phases of the software development life cycle. A wide variety of scan tools are available, such as:

  • Static Application Security Testing (SAST) tools that scan source code, binaries, or byte code.
  • Dynamic Application Security Testing (DAST) tools that scan running applications.
  • Software composition analysis (SCA) tools that scan the open source software in used in code repos.
  • Container scanning tools that detect issues in running containers.

The following pictorial highlights a typical SDLC process.

Proactive security testing

How many scanners does STO support?​

STO supports many popular scanners. For a list of all supported tools, go to the Supported Security Scanners.

Do similar category tools show identical issues?​

In theory, similar scanners should identify similar issues for the same target. In practice, however, different scanners often identify different vulnerabilities for the same target. Even similar scanners can produce results that have no similar issues.

What is an STO Target?​

Every instance of a scanner has a specific target, which is the object or software/code artifact it is set up to scan. It might be a repo, a container, or an instance and for some scanner categories, an artifact maybe collection of configuration files. A target baseline refers to a baseline variant, such as a main branch or a latest image tag, against which developers want to compare new code.

STO supports the following test targets and form factors:

  • Repository
  • Container
  • Instance
  • Configuration

The following table specifies where the target to be tested is located.

Target nameTarget type
azurerepository
bitbucketrepository
githubrepository
gitlabrepository
local_imagecontainer
docker_v2container
jfrog_artifactorycontainer
aws_ecrcontainer
websiteinstance

Can I use STO with custom or unsupported scanners?​

Yes. You can easily ingest Static Analysis Results Interchange Format (SARIF) data from any tool that supports this format. SARIF is an open data format supported by many scan tools, especially tools available as GitHub Actions.

STO also supports a generic JSON format for ingesting results from custom or unsupported scanners.

By ingesting your custom issues, you can benefit from STO's refinement, deduplication, and correlation features. Harness handles your issue data the same way as data from supported scanners.

How does STO identify new issues and display them in the UI?​

You can define a baseline for each target. The baseline is the object that you want to update. For example, your scan target might be a codebase for a specific service that you're looking to update. You specify the main branch as the baseline. You run scans on the main branch and any number of non-main branches. Each scan has its own set of identified issues.

STO identifies an issue as "new" like this:

  • If you're scanning the baseline, or if the target has no baseline, any issue found in the latest scan but not in a previous scan is considered new.

  • If you're scanning a non-basline object, such as a feature branch, any issue found in that object but not in the baseline is considered new.

    Suppose you introduce a new library with known vulnerabilities into your feature branch. STO shows those issues as new if the scanner found them in the feature branch only, not in the baseline. When you merge your feature branch into main, any vulnerabilities that get merged in are no longer considered new.

Why do I need to use Docker-in-Docker? Why is root privilege required?​

When Harness acquired ZeroNorth in 2021, the STO development team used Docker-in-Docker to speed up the integration between the ZeroNorth and Harness platforms. Some scanners explicitly require Docker-in-Docker, which requires Privileged mode.

On March 14, 2023, Harness introduced a set of new UIs (scanner templates) for Aqua Trivy, Bandit, SonarQube, and other popular scanners. These steps do not require Docker-in-Docker unless you are scanning a container image on Kubernetes or Docker build infrastructures. For more information, go to Docker-in-Docker requirements for STO.

info

Harness is currently working to remove this requirement for other scanners that don't need it and to provide other ways to orchestrate scans.

What RBAC roles does STO support?​

Harness supports two RBAC roles specifically for STO users. You can customize these roles as needed in the Harness platform.

  • Developer role — Permissions needed for developer workflows.

  • AppSec role — Permissions needed for Application Security or Security Operations staff. This role includes all Developer permissions and also allows users to approve security exemptions.

These workflows are covered in Add Security Testing roles.

How do I set up a CheckMarx scan step in YAML?​

Here's an example. See also the Checkmarx scanner reference.

- step:
  type: Security
  name: Checkmarx Scan
  identifier: Checkmarx_Scan
  spec:
    privileged: true
    settings:
      policy_type: orchestratedScan
      scan_type: repository
      product_name: checkmarx
      product_config_name: default
      product_team_name: /its/in/this/format
      product_domain: https://codescan.example.com
      product_access_id: <+secrets.getValue("account.checkmarx_access_id")>
      product_access_token: <+secrets.getValue("account.checkmarx_access_password")>
      product_project_name: <redacted>
      repository_branch: <+codebase.branch>
      fail_on_severity: HIGH
    failureStrategies: []

What is the purpose of the Exemptions page in STO?​

The page (or tab) exists to grant and manage security exemptions. For more information, go to Exemptions to override Fail on Severity thresholds for specific issues in STO.

What open-source scanner integrations does STO support?​

For detailed information about supported scanners, both open-source and commercial, go to Harness STO scanner support.

What is the difference between an STO pipeline, a CD pipeline, and a CI pipeline?​

In Harness NextGen, there are no hard distinctions between CI, CD, and STO pipelines. You can create integrated pipelines that implement any mix of CI, CD, and STO use cases. Harness platform components are generic to all three modules: you can use and reuse delegates, connectors, secrets, and other components across across multiple stages and pipelines.

You can easily include scans in CI workflows. To run a scan, you add a standalone Security stage or a Security step to a CI Build stage. The pipeline uses the same platform components to build code and run scans. For an example of this type of workflow, go to Create a build-scan-push pipeline (STO and CI).

For CD workflows, Harness recommends that you add a standalone Security stage (for static scans) before the deployment and another stage (for dynamic scans) after the deployment.

Does STO require Harness CI or Harness CD?​

No, you can run STO as a standalone product. STO supports integrations with external tools using standard triggers and webhooks. Thus you can use STO to add security scans to your CI and CD workflows if you're using an external tool such as GitHub.

What infrastructures are supported for running STO scans?​

STO supports running scans on Harness Cloud, Kubernetes, Docker, and VMs. STO uses CI build infrastructures to orchestrate scans and ingest issues.

For more information, go to What's supported in Harness STO > Harness SaaS > Build Infrastructure.

How does STO deduplicate detected issues?​

The following diagram illustrates the three stages in the STO deduplication pipeline:

  1. Normalization: The pipeline normalizes the "raw" scan results using STO-defined parameters and attributes that are common across supported scanners.

  2. Deduplication: The pipeline deduplicates based on CVE, CWE, package name, and other reference identifiers found in the scan results.

  3. Prioritization: For each issue, the pipeline calculates a Severity score based on the NIST Common Vulnerability Scoring System (CVSS) version 3.0.

I don't have any scanner licenses and I'm just getting started with security scanning. Where do I begin?​

STO maintains its own set of container images pre-installed with open-source scanners such as Bandit, OWASP, Zap, Prowler, and Aqua Trivy. This makes it easy to get started running scans and generating results. The STO UI uses one format to report all detected issues and includes in-depth information about each detected issue: location, severity, and links to in-depth information, and so on.

To get started, go to Get started with STO.

Why don't I see results from specific scans in the Security Testing Dashboard?​

The most likely reason is that the scanned target does not have a baseline defined. The Security Testing Dashboard is intended to show vulnerabilities that potentially affect production-ready artifacts. For this reason, the dashboard shows detected vulnerabilities for targets that have baselines defined.

How does STO ensure High Availability and data protection for my scan data and vulnerability information?​

Harness and STO adopt the highest standards towards data privacy and data security. To see the general operational status of Harness production environments, go to Harness Status.

All of our Cloud SQL Prod databases (including STO) have both an HA and DR instance, as well as point-in-time recovery to any moment in the previous seven days. The HA instance is in a different Google Cloud Platform zone in the same region, and the DR instance is in a different region. Backups are retained for seven days.

Harness Engineering has a roadmap plan to support a similar infrastructure for Self-Managed Enterprise Edition.

Can I prevent the user who requested an exemption from approving it?​

To approve exemptions, users must have the Approve/Reject permission. Currently, Harness STO doesn't have a setting to prevent the user who requested the exemption from also approving it.

Can the size of the container image impact pod eviction during a scan?​

Yes, the size of the container image contributes to resource utilization, especially large images (around 4GB). Make sure the container has sufficient resources allocated to prevent eviction during resource-intensive tasks, such as Aqua scans. For more information, go to

Does STO support execution on ARM64 architecture?​

Currently, STO doesn't support running on ARM64 platforms.

Can I use a specific image tag for STO images?​

Yes. For instructions and information about customizing your STO images, go to:

Can I use an API to extract data on vulnerabilities detected by STO?​

Public APIs for this functionality are on the STO roadmap.

Does Harness STO support ingesting scan results from non-native scanners?​

STO supports a generic JSON format for ingesting data from unsupported scanners that cannot publish to SARIF. This means that you can ingest custom issues from any scanning tool, even those that do not have direct integration with STO. For more information, go to Ingest from unsupported scanners.

What is SARIF, and how does it relate to scanning tools and STO?​

SARIF (Static Analysis Results Interchange Format) is an open data format supported by many scanning tools. If your scanner supports SARIF, Harness recommends publishing your results in this format. For more information, go to Ingest SARIF scan results.

Which scanners are supported natively with Harness STO?​

  1. SAST (Static Application Security Testing)
  2. SCA (Software Composition Analysis)
  3. Secret Scanning
  4. DAST (Dynamic Application Security Testing)
  5. Container Scanning For more details, go to What's supported.

What are the requirements for STO to ingest scan results in JSON format?​

For STO to successfully ingest your scan results, the ingestion file must adhere to the specific JSON format described here: Ingest from unsupported scanners

STO dashboards​

These is no Test Execution Summary widget in the list of dashboard widgets​

To use this widget, you must have the CI_TI_DASHBOARDS_ENABLED feature flag enabled for your account. This feature flag enables the Unit Tests Metrics dashboard. Contact Harness Support to enable this feature flag.

Why doesn't the STO dashboard populate the data from targets?​

This happens when scan executions don't have baselines set. You must set test target baselines to show this data on your STO dashboards.

Every scanned target needs a baseline to enable the full suite of STO features. For more information, go to Target baselines.

I'm trying to create a dashboard to display the STO metrics for my project, but my organization is not getting listed in the filter. Hence, I'm not able to get my project filtered.​

The dashboard only populates projects that had a baseline scan run previously. Check that your scanned targets have baselines defined if they don't appear in the dashboard.

Every scanned target needs a baseline to enable the full suite of STO features. For more information, go to Target baselines.

Aqua scans​

Pod evicted during an Aqua scan​

Pod eviction during an Aqua scan can be attributed to resource constraints, especially with a large image size (around 4GB).

To address pod eviction during an Aqua scan, increase container resource limits by adjusting the resource requests and limits for the container.

Can the Aqua Security scan use an image built with PLUGIN_NO_PUSH?​

No. The Aqua Security scan can't pick up a local image.

AWS ECR scans​

How do I configure a session token in the AWS ECR scan step?​

You can set the AWS_SESSION_TOKEN in the Authentication settings.

BlackDuck scans​

During BlackDuck scans in my pipeline, I get a "Could not connect to addon client after max retries" error, but this error doesn't occur locally​

The "Could not connect to addon client after max retries" error typically indicates that the container running the BlackDuck scan step is terminated abruptly due to insufficient resources. To address this issue, Harness recommends increasing the resources allocated to the BlackDuck step.

You can begin by adjusting the resource allocation to memory: 1Gi and cpu: "1.0". Then, monitor the memory and CPU consumption of the container during the scan to gauge its resource requirements accurately. Based on this observation, you can further refine the resource allocation as needed to prevent container termination and ensure successful BlackDuck scans in your pipeline.

For more information, go to Optimize STO pipelines.

Grype scans​

Grype exception "db could not be loaded: the vulnerability database was built n weeks ago (max allowed age is 5 days)"​

Go to Troubleshoot "vulnerability database build date exceeds max allowed age" exception in the Grype scanner reference.

OWASP scans​

OWASP step generates exception when initializing Yarn Audit Analyzer​

Go to Troubleshoot Yarn Audit Analyzer exceptions in the OWASP scanner reference.

Prisma Cloud scans​

How do I add labels to a Prisma Cloud scan when my build infrastructure is Kubernetes or Docker?​

To add labels such as JOB_NAME to your Prisma Cloud scans, add key-value pairs to Settings (optional) in your Prisma Cloud scan step. These key-value pairs will be added as labels when you run the scan.

Sonar scans​

Go to Troubleshoot Sonar Scans in the SonarQube scanner reference. This section discusses the following:

Why am I getting the error Missing target_name for scan_type [repository] scan.​

This error ocurrs if there's no scan target in the Scanner configuration. To fix this, please ensure that the Scan Step configuration properly selects a target.

How does the SonarQube integration work when attempting to perform a branch scan with SonarQube Enterprise?​

An enhancement has been made to ensure the orchestration step always downloads results for the branch specified in the step, instead of downloading results only for main or master branches. For more information, go to the STO 1.83.1 release notes.