What should I know about telemetry data collection?

Data collection prerequisites

All DCA functions, including baselines, indicators, and OAM testing, depend on telemetry data. Telemetry data collection must be configured before any function in the Data Collection and Analysis views can be used.

NSP has the following prerequisites for telemetry data collection.

• The NEs from which you want to collect data must be discovered, managed, and reachable by the NSP.

• Mediation policies must be configured in the discovery rules used to discover and manage the NEs:

  • SNMP telemetry requires an SNMP mediation policy

  • gNMI telemetry requires a GRPC mediation policy

  See “Device discovery” in the NSP Device Management Guide for more information about NE discovery and mediation.

• If you want to use a telemetry subscription to manage MIB statistics from classic NEs, statistics collection must be configured in the NFM-P.

• Cloud native telemetry requires the gnmiTelemetry installation option.

• All required NE adaptors and telemetry custom resources must be installed on the NSP server:

  • for the list of required telemetry artifacts for an NE family, see the artifact guide for the NE family

  • for information on installing the telemetry CRs, see How do I install telemetry artifacts?.

• If you want to use statistics for Analytics reports with granularity other than raw data, you need aggregation rules for the telemetry types you want to collect; see the Telemetry tutorial on the Network Developer Portal

• You need to configure the aggregation time zone.

Accounting telemetry

Accounting telemetry requires retrieval of accounting file information from NEs. The NSP polls the NE directories specified in the device helper custom resource for accounting file updates. The file on the NE is transformed to JSON format, compressed, and saved in the NSP file server.

CAUTION: All accounting files found on the NE are transferred to NSP, processed, then deleted from the NE. The file is no longer available for collection by NFM-P or any other external application. This applies to both Classic and model-driven NEs.

If communication with the NE is lost, files that were stored on the NE during the loss of communication are retrieved when communication is reestablished, provided the files have not been purged by the NE.

The following prerequisites apply for accounting telemetry:

• an FTP NORM mediation policy must be assigned to the NE. FTP mediation policies are created and assigned using a REST API; see the LSOM Framework and APIs tutorial on the Network Developer Portal.

• The NE must be configured to allow parallel SSH sessions; see the NE documentation.

• an accounting policy must be configured on the NE. The accounting policy can be configured via CLI; see the NE documentation, or in the Device Management, Device Configuration views using the icm-file-accounting intent type; see the NSP Device Management Guide.

• a file purge policy in the NSP is recommended; see “How do I configure file purge policies?” in the NSP System Administrator Guide.

Data available for collection

When NSP collects telemetry from NE devices using the gNMI protocol, only data in statistics containers is available for collection. Furthermore, NSP may only make a subset of the attributes in the statistic containers available in its telemetry types. If you cannot find the counter you are looking for in the NSP telemetry types, verify that it is available in a statistics container in the relevant NE state Yang path.

For example, the SR Linux search tool can be found here: (https://yang.srlinux.dev/). Click on the NE release of interest and filter for the paths you need, as shown in the following example:

For more information about supported telemetry attributes for your NE family, see the NE artifact guide. For more information about telemetry definitions, see the Telemetry collection API documentation on the Network Developer Portal.

Automatic subscriptions

Subscriptions are automatically created for Baseline Analytics, NSP Indicators, and OAM testing. To collect statistics for another purpose, create a subscription and set its state to enabled.

RESTCONF APIs are available for telemetry collection and aggregation; see the Network Assurance API documentation on the Network Developer Portal.

Data storage

You can enable database storage for MDM managed NEs as part of subscription creation. By default, the data collected is stored in Postgres unless there is an auxiliary database enabled, in which case all collected data is stored in the auxiliary database. For NFM-P collected SNMP or accounting statistics, the database parameter in the subscription is ignored.

Historical data, that is, data that is stored in a database, is retained according to the age-out policy; see How do I edit an age-out policy?.

Note: For statistics to be available to the Analytics application for aggregated reports, they must be stored in the auxiliary database.

The Telemetry data API also provides the functionality to subscribe, stream, and plot historical and real-time data; see Use Case 5 in the Telemetry Collection tutorial on the Network Developer Portal.