Configuring a custom parser

This section describes the syntax for configuring a custom parser in TRON.ASOC, supported expressions and functions, parser builder limitations, and how to debug the configuration. For information about creating, assigning, and deleting parsers, see Working with parsers.

warning

The parser builder uses the standard JSONata language, customized for the TRON.ASOC data model. Some language constructs aren't supported, and some work differently than they do in the standard implementation.

This results in two practical rules:

• Use only the constructs and functions listed in Mapping fields and Available functions—using an unsupported construct disables parsing.
• The result of checking an expression in a third-party tool (try.jsonata.org) doesn't guarantee the same result in TRON.ASOC. Use it only to check field paths against a real report. Check the expression logic against a sample report in the system. See Creating and validating a configuration.

Differences from standard JSONata

The parser builder uses the standard JSONata language, customized for parsing reports in TRON.ASOC. The constructs in the table below work differently in the builder than they do in the standard language implementation. This is the most common reason an expression that looks correct returns an empty value.

Construct	Standard JSONata	TRON.ASOC parser builder
or operator	Strictly logical: "a" or "b" returns true	Returns the first nonempty value (fallback expression): "a" or "b" returns "a"
$match(expr, pattern)	Requires a regex literal: $match(expr, /CVE-\d+/)	Accepts the pattern as a string: $match(expr, "CVE-\\d+")
Ternary operator with a condition based on a missing field	x ? a : b returns b if the x field isn't present in the finding	Returns an empty value; the b branch isn't applied
Field names with hyphens	Require backticks: info.`cwe-id`	Backticks aren't supported. Specify the field as-is: info.cwe-id
:= (variable assignment)	Supported	Not supported
{ ... } (object literal)	Supported	Not supported
% (reference to the parent context)	Supported	Not supported
$number(), $exists(), $count()	Supported	Not supported

The ternary operator with a condition based on a field that isn't present in all findings returns a value only for findings where the field is present. For other findings, the field remains empty, even if the else branch contains a nonempty expression. For optional fields, use the or operator. See Common techniques.

danger

If at least one field in a section contains an unsupported construct, the section doesn't return any findings. The system doesn't display an error, and the parsing result is 0 vulnerabilities, with no indication of the specific field.

Therefore, add fields incrementally while developing a configuration instead of adding them all at once. See Creating and validating a configuration.

Configuration structure

A parser configuration is a JSON object that maps each TRON.ASOC model field to a JSONata expression.

You can define a configuration in two ways:

• One section—specify mapping fields at the top level of the object.

{
  "items_path": "runs[0].results",
  "category": "...",
  "severity": "..."
}

• Multiple sections—specify mapping fields inside the sections array, with one object for each finding type.

{
  "sections": [
    { "items_path": "Results[].Vulnerabilities", "category": "...", "severity": "..." },
    { "items_path": "Results[].Misconfigurations", "category": "...", "severity": "..." }
  ]
}

The system combines findings extracted by all sections into one result for the report.

Configuration section description

items_path

items_path specifies the path to the findings array in the report. The parser evaluates all other section expressions relative to an individual item in this array, not relative to the report root.

Report structure	items_path value
Findings in one nested array	runs[0].results
The report root is the findings array	$
Two nesting levels	Results[].Vulnerabilities
Three nesting levels	site[].alerts[].instances

warning

If items_path doesn't start with the $ prefix, the system processes the path with a simplified handler that understands only a sequence of segments and array expansion with []. Predicate filters ([field = "value"]) aren't applied in this format.

To use a filter, start the path with $:

"items_path": "$.issues[type.key = \"Sca\"].ptOsvs"

Filtering by field value isn't supported for all path formats. If the section returns 0 findings after you add a filter, expand the array without the filter and exclude unnecessary findings at the mapping-field level.

The following incorrect formats occur most often:

• "[]" isn't a path on its own. Square brackets expand an array when used with a path segment ("[].messages"). To expand the report's root array, use "$".
• An empty string enables automatic detection of the findings array. Automatic detection doesn't work with all report structures, so specify items_path explicitly in custom parsers.

sections and __parent

Use the sections array when a report contains multiple independent findings arrays of different types—for example, a Trivy report contains vulnerabilities, misconfigurations, and secrets—or when one report item should produce multiple findings.

The service object __parent is available inside a section. It contains the object that held the expanded items_path array.

warning

__parent provides access only to scalar fields of the parent object. Nested objects and arrays aren't available through __parent: an expression such as __parent.type.displayName returns no value and, like any unsupported reference, can disable the entire section.

If the required value is in a nested object of the parent, use a scalar field at the same level that provides the equivalent value—for example, an existing text component identifier—or move the expansion one level higher.

Example: one SCA component contains an array of vulnerabilities, and each vulnerability must become a separate TRON.ASOC finding.

{
  "sections": [
    {
      "items_path": "issues",
      "category": "type.displayName",
      "severity": "$lookup({\"10\":\"1\",\"20\":\"1\",\"30\":\"2\",\"40\":\"3\",\"50\":\"4\"}, $string(level.severity))"
    },
    {
      "items_path": "issues[].ptOsvs",
      "category": "summary",
      "cve": "$match($join(sources, \"; \"), \"CVE-\\\\d{4}-\\\\d+\").match",
      "source_file": "__parent.parentItem"
    }
  ]
}

info

A section with a more general items_path also extracts items processed by the second section. The system then combines matching findings by a composite key. See Combining findings. Therefore, the total number of findings is usually less than the sum of findings from all sections.

Available functions

The table below lists the functions supported by the TRON.ASOC parser builder.

Function	Purpose
$lookup(map, key)	Returns a value from a lookup table by key
$string(expr)	Converts a value to a string
$join(array, separator)	Joins array items into a string using a separator
$split(string, separator)	Splits a string into an array
$substring(string, start, length)	Returns a substring
$replace(string, pattern, replacement)	Replaces pattern matches
$uppercase(string)	Converts a string to uppercase
$match(string, pattern)	Searches for a pattern match and returns an object with the match, index, and groups keys
$max(array)	Returns the maximum value in an array of numbers
$truncBytes(string, n)	Truncates a string to n bytes without splitting multibyte characters
$tailBytes(string, n)	Returns the last n bytes of a string
$linesJoin(lines)	Joins nonempty Content values from an array of objects in the form {"Content": "..."} with line breaks
$sanitize(html)	Removes HTML tags from a string and trims whitespace
$cwe(value)	Returns the first identifier in the form CWE-XX found in a string or array of strings
$cweAll(value)	Returns all unique CWE identifiers found in a string, including links such as cwe.mitre.org/data/definitions/79.html, joined with ;

Functions not listed in the table aren't supported by the parser builder. Using an unsupported function disables the entire section. See the warning in Differences from standard JSONata.

Behavior of individual functions

$match returns an object, not a string. To get the matching text, access the match key:

"cve": "$match($join(sources, \"; \"), \"CVE-\\\\d{4}-\\\\d+\").match"

$max works only with numbers. If a report field contains a score as a string ("5.3"), $max returns an error and disables the section. Most reports contain no more than one score for each CVSS version in a finding, so you can use the [0] index instead of $max.

$join doesn't accept null. An explicit null in a report and a missing key are handled differently: a missing path safely returns an empty value, while null causes an error. Add a check before the function call:

"cve": "info.classification.cve-id ? $join(info.classification.cve-id, \"; \") : \"\""

A ternary check is valid here because an empty value is the expected result when the field is missing. Add the same check before $uppercase and other functions that receive a value from an optional field. Specifying a nonempty default value in the else branch doesn't help: if the field is missing, the branch isn't applied. See Differences from standard JSONata.

Common techniques

Fields with hyphens in their names. Specify them as-is without backticks:

"source_file": "matched-at or url"

A value from multiple possible sources. The or operator returns the first nonempty value, so list alternatives in a chain:

"line": "startLine or numberLine or beginLine"

A default value for an optional field. If a field isn't present in all findings, specify a fallback source with or instead of a ternary operator:

"source_object": "matcher-name or template-id"

For findings without a matcher-name field, an expression such as "matcher-name ? matcher-name : template-id" returns an empty value.

Selecting required array items. If an array contains identifiers from different systems (CVE, GHSA, and BDU), don't access an item by position—the first item might be an identifier from another system. Join the array into a string and find the required identifier by pattern:

"cve": "$match($join(sources, \"; \"), \"CVE-\\\\d{4}-\\\\d+\").match"

Combining multiple values into one field. Combine classifications and other heterogeneous data through concatenation, checking each source:

"message": "(owasp.value != \"\" ? (\"OWASP \" & owasp.value & \" | \") : \"\") & (sans.value != \"\" ? (\"SANS \" & sans.value) : \"\")"

Values from nested arrays of objects. A path through an array of objects applies automatically to each item, so you don't need a separate function: pcidss.displayName returns an array of values from all items.

Field value limits

Finding field values have length limits in bytes, and each field has its own limit. In Cyrillic text, one character takes two bytes, so the limit is reached approximately twice as quickly as in Latin text.

If an error occurs while parsing a report, reduce the value length with $truncBytes:

"category": "$truncBytes(summary, 500)"

Truncating a string with standard string functions can split a multibyte character and cause an error when the finding is saved. Use $truncBytes specifically for truncation.

Combining findings

Findings with the same values in key fields are combined into one record by a composite key. The keys are:

• tool_type_name;
• cve;
• cwe;
• lib_name;
• lib_version;
• line;
• matched_code;
• source_file;
• source_object;
• category.

If a parser configuration doesn't populate some composite-key fields, different findings can receive the same key and be combined into one. This most often occurs with finding types for which the report doesn't contain a file path or line number—SCA findings and detected secrets. When source_file and line are empty, the key is reduced to the category and cwe pair.

If the expected number of findings is less than the actual number of records in the report, compare the difference with the number of matching composite keys. This is expected behavior, not data loss.

Report requirements

The report must be a syntactically valid JSON document in accordance with RFC 8259. The parser evaluates expressions against the parsed document, so you can't use the parser configuration to work around errors in the file itself:

• Invalid escape sequences. After \, only ", \, /, b, f, n, r, t, and u followed by four hexadecimal digits are allowed.
• // comments aren't part of the JSON grammar.
• Unescaped line breaks and other control characters inside string values.
• Unescaped double quotation marks inside string values.

If the system doesn't accept the report, first check it with a JSON validator. Only then investigate the parser configuration.

info

SARIF is a data schema built on JSON, not an alternative file format. A report in SARIF format is also a JSON file.

However, the same security tool can export results in multiple incompatible schemas—for example, SARIF and its own JSON format. A parser configuration is tied to a specific schema: a parser written for a tool's SARIF export won't parse that tool's report in its own format. Before creating a configuration, make sure the sample report was produced using the same export method that will be used later.

Creating and validating a configuration

Configuration creation procedure

Follow these steps to create a valid parser configuration:

1. Identify the report structure: determine which array contains the findings, how many nesting levels it has, and whether the same array contains different finding types.
2. Create a minimal configuration with only items_path, category, and severity with a constant value, such as "1".
3. Validate the configuration and make sure the number of findings matches the expected number.
4. Add fields one at a time or in small groups, validating the configuration after each step.
5. If the configuration contains multiple sections, validate each section separately and combine them only after validation.

This procedure is recommended because an error in any field disables the entire section and doesn't identify the specific field. In a configuration with 20 fields, you can't identify the source of the error without adding fields step by step.

Errors that occur when creating a configuration

The following are the most common errors you might encounter when creating and validating a configuration.

Error	Possible cause
0 findings in a report known to contain findings	An incorrect items_path or an unsupported construct in one of the section fields
0 findings for any items_path value, including a known-invalid value	The security tool has a different parser assigned to it. Check the Parser field in the tool settings
One finding instead of several	All findings received the same composite key. See Combining findings
Some findings are missing	Matching composite keys for some findings or an unaccounted-for value in the $lookup table
One field is empty for all findings	A typo in the path, backticks around a field name with a hyphen, or the value is missing from the report itself
One field is empty for some findings	The report contains multiple finding types with different structures—add alternative sources with or. If the field uses a ternary operator, check whether the condition field is present in all findings. If it's missing, the expression returns an empty value
Error saving findings	Parsing succeeded, but a problem occurred during saving. See Field value limits

tip

If the result doesn't change after you modify the configuration, change one parameter at a time. Changing multiple fields at once makes it impossible to determine which change affected the result.

Validating a configuration against a sample report

The system doesn't have a separate mode for validating a configuration against a sample report. To validate a configuration, import the sample report into a security check with the parser assigned, then evaluate the result by reviewing the detected vulnerabilities and their populated fields. For the procedure, see Creating a parser for a security tool that TRON.ASOC doesn't support.

To keep test runs separate from production data, run the validation in a separate layer or security check.

Modifying the configuration of an assigned parser

warning

Changes to a parser configuration are inherited by all security tools and security checks that use the parser, but they apply only to reports processed after the change. The system doesn't recalculate previously saved vulnerabilities: new fields won't appear, and changed values won't be updated.

To make parsing results match the new configuration, process the report again. During reprocessing, the system closes vulnerabilities that the updated parser doesn't find and combines newly detected vulnerabilities with previously detected ones when their hashes match.

An invalid configuration therefore results not only in missing new vulnerabilities but also in previously detected vulnerabilities being closed. Validate the configuration in a separate layer or security check before assigning the parser to a security tool with production data.