Zero touch provisioning

The Nokia-specific TLV is NOKIA-DCTOR-AUTOCONFIG. The location of the BOF for each system to use is configured in the optional autoboot file parameter, which is a standard Option 43 value initialized at the beginning of the process. The BOF location is sent in Option 43 as part of the DHCP offer and Ack messages from the DHCP server to the system. The system uses the location specified in Option 43 to initiate an FTP download of the BOF.

This section describes DHCP discovery options.

Unlike IPv4 DHCP offers, which contain the prefix and default route, IPv6 DHCP offers only contain the IP address assignment. The IPv6 route advertisement (RA) provides the default router and the prefix is set to /128 for the IP address supplied by the DHCP server.

For further information about RA support, see IPv6 DHCP/RA details. For further information about DHCP server offers, see DHCP server offer options.

Option 3 (default route) and Option 121 (classless static route) are supported for IPv4 DHCP.

For identical routes with different next hops, only the first route is installed and the second route is kept as a backup route. ECMP is not supported.

There is no route limit for Option 121.

IPv6 DHCP offers do not contain an IP prefix. The IPv6 prefix is usually obtained from the IPv6 RA arriving from the upstream router. Because the SR OS router is the host for the ZTP process, the system assigns a /128 prefix to the IPv6 address obtained from the DHCP offer.

SR OS supports the use of an IPv6 RA to install IPv6 default and static routes from upstream routers. The system installs all the routes advertised using the RA. If the same route has been advertised from multiple upstream routers (next hops), the system installs the route with the highest preference. SR OS does not support ECMP if the same route is advertised from multiple next hops by multiple RAs.

In accordance with RFC 4861 recommendations, SR OS ensures that all RAs are obtained before the auto-provisioning process is started for IPv6. RFC 4861 recommends that the host (in this case, the SR OS router) send a minimum of three route solicitations to increase the likelihood of at least one route solicitation being received by the upstream routers. Each route solicitation is followed by a 4-second timeout, so the third route solicitation is sent 8 seconds after the first. The upstream routers must respond within 0.5 seconds. As a result, the SR OS router receives all RAs and routes within 8.5 seconds of the first route solicitation, and waits a maximum of 9 seconds to receive all RAs; ZTP always waits 20 seconds to receive all RAs, however, only the first RA received is used.

The ZTP timeout is user-configurable with a default value of 30 minutes. See Options and option modification and Configuring the ZTP timeout in the provisioning file for more information. After each ZTP timeout, the node reboots and reattempts the ZTP process. If the ZTP timeout interval expires while the node is executing a DHCP offer or downloading files, the node does not reboot. The DHCP offer is executed until it succeeds or fails, at which point the node reboots. If the offer is successful, the node comes up in normal operation mode.

The DHCP timeout interval is 20 seconds. If a DHCP offer is not received within the DHCP timeout interval, the auto-provisioning process is reattempted using the next valid interface.

ZTP can be reinitiated any time by setting the auto-boot flag and configuring the flag options in the BOF. After the auto-boot flag is set, any reboot forces the node into ZTP mode, including DHCP discovery, and downloading and reprocessing the provisioning file. The old BOF is kept in the storage medium until the ZTP process is successful, then the old BOF information is overwritten. If an unsuccessful ZTP process is interrupted and the auto-boot flag is removed, the node boots using the old BOF.

Nokia recommends that the provisioning file should download all files to the SD card, and the BOF should point to the SD card for imaging and configuration.

The BOF itself does not support loading from the network using HTTP or HTTPS.

By default, the auto-boot process scans all ZTP-enabled ports to find a port with an operational link. The scanned ports include:

• out-of-band management port (Mgmt port)

• Ethernet ports on the first two card or MDA slots (used for in-band management)

  Note: For breakout connectors, only the first breakout port in the connector can be used for ZTP.

ZTP attempts to discover the node IP via DHCP and identifies the node using DHCP client ID Option 61 (IPv4) or Option 1 (IPv6). The client ID uses the chassis serial number by default. The chassis serial number is visible on the shipping box of the chassis.

Supported DHCP client options for ZTP lists the default DHCP client options for ZTP. Some client options can be manually configured in the BOF using the bof auto-boot command.

The optional auto-boot configuration options are as follows:

• management port

  Specify that ZTP should only be performed using the out-of-band management port (Mgmt port).

• in-band VLAN

  Specify ZTP should only be performed using Ethernet ports on the first two card or MDA slots. The VLAN ID can be used to specify an in-band VLAN to use for the auto-boot process.

• IPv4, IPv6

  Specify that IPv4 discovery, IPv6 discovery, or both, should be performed. If both are specified, the system dual-stacks.

• client identifier

  Identify the node to the DHCP server and find a pool for DHCP offers. This information is sent using Option 61 (IPv4) or Option 1 (IPv6). If the client-identifier options are not configured, the chassis serial number is sent by default. This option is used for both IPv4 client ID and IPv6 DUID Type 2.

• include user class

  Specify to include Option 77.

• timeout

  Specify in minutes the timeout for the ZTP process to be executed successfully before the node is rebooted and ZTP is retried because of an unsuccessful ZTP completion. The default ZTP timeout is 30 minutes.

  See Configuring the ZTP timeout in the provisioning file for information about how to configure the ZTP timeout in a ZTP provisioning file.

The auto-boot options can be modified using the bof auto-boot command, or by interrupting the bootup process and manually modifying the bof.cfg file.

The auto-boot process is executed in the background and does not block the CLI. The user can enter CLI commands while the auto-boot process runs in the background. A warning message is displayed to notify the user that the auto-boot process is being executed. Any configurations performed using the CLI may be lost when the node reboots following successful auto-boot and auto-provisioning processes. After the node has finished booting and if the auto-boot flag is set in the BOF, the node displays the login prompt.

The user can access the CLI using a console and can change and save the BOF configuration; as such, the user can remove or modify the auto-boot option in the BOF.

The auto-boot process can be interrupted using the tools auto-boot terminate command. After the auto-boot process is terminated, use the bof auto-boot command to modify the auto-boot flag.

The node can perform VLAN discovery if it is shipped in ZTP mode. VLAN discovery is supported only for the in-band management port. It is not supported for the out-of-band management ports.

After the node is installed and powered up:

1. ZTP is attempted on the null (untagged) port first, including the out-of-band management port, and then on all in-band management ports with operational links.

   1. SR OS scans each port with an operational link and sends IPv4 DHCP discovery messages.

   2. SR OS waits for the DHCP offer within the DHCP timeout.

2. The first VLAN with a valid offer that includes the IPv4 DHCP Options 66 and 67, or Option 67 or 43, or IPv6 DHCP Option 59 or 17 is selected as the working VLAN and the ZTP process is executed on this VLAN.

   Note: If there is no offer or the offer does not have the relevant or correct options, SR OS floods the network with DHCP discovery messages on all remaining non-reserved VLANs (1 to 4094).

3. When a VLAN is discovered, the ZTP process is executed on the respective VLAN as described in the following sections.

   Note: If there is no offer on any VLAN or the offer does not have the relevant or correct options, the node starts over from step 1.

After the node enters ZTP mode, the auto-discovery process is executed to provision the necessary hardware for node discovery.

The following are the operational steps of the auto-discovery process.

1. DHCP is used to discover the IP address of the node.

2. Options 66 and 67, or Option 43 is used to find and download the provisioning file.

   The provisioning file includes the location of necessary files, such as configuration information, system image, and licenses, along with the DNS needed to resolve these location URLs. The file also includes BOF information required to boot the node into operational mode.

3. The provisioning file is executed to download the named files to the node.

4. After all files are successfully downloaded, the node is rebooted and the auto-boot flag is cleared from the BOF.

After the node reboots, it comes up in normal operational mode.

The node can be put back into ZTP mode by editing the BOF to include the auto-boot flag and saving the BOF. Doing this causes the node to enter ZTP mode after it is rebooted.

Use one of the following methods to run the auto-provisioning process.

• automatic execution

  The auto-boot process automatically executes the auto-provisioning process if the auto-boot flag is set in the BOF.

• manual execution

  The auto-provisioning process can be executed manually using the following command.

  tools perform system auto-node-provisioning

  If the auto-provisioning process is executed manually, only interfaces without IP addresses are considered part of the discovery mechanism. Additionally, while the process is running, it attempts to discover DHCP servers using all card or MDA slots and ports with Layer 3 interfaces that do not have IP addresses.

  Note: Using the following command while the auto-boot process is running is not allowed.

  tools perform system auto-node-provisioning

The auto-provisioning process can use the out-of-band management port (Mgmt port), or in-band management on Ethernet ports.

The node attempts the auto-provisioning process using any port with an operational link, starting with the out-of-band management port. If the node cannot be discovered using the out-of-band management port, either because the port is down or the port is not receiving a DHCP offer from the DHCP server, the process is reattempted using Ethernet ports. If the node cannot be discovered using the Ethernet ports, the process is reattempted using the out-of-band management port and the cycle repeats.

The following operational guidelines apply to in-band and out-of-band management ports:

• Out-of-band management and in-band management support untagged frames.

• Out-of-band management does not support dot1q (VLAN tags).

• In-band management supports dot1q interfaces if the VLAN is correctly configured in the BOF.

• In-band ports support VLAN discovery for IPv4 by default, if not disabled in the BOF.

If out-of-band management is used, no card or MDA provisioning is necessary and the auto-provisioning process executes as soon as an active link is detected on the Mgmt port.

To use out-of-band management exclusively, use the following command:

bof auto-boot management-port

To use in-band management exclusively, use the following command:

bof auto-boot inband vlan

The provisioning file location is discovered using DHCP offer Options 66 and 67 or Option 43, and is downloaded using HTTP or FTP.

The provisioning file URL can be resolved using DNS, in which case the IP addresses for up to three DNS servers should be present in the DHCP offer using Option 6 (IPv4). The DHCP DNS is only used for resolving the provisioning file URL, and not for resolving the URLs of the files named within the provisioning file.

ZTP does not support Option 15 domain names; the URL of the provisioning file should be in ‟host/domain” format, or a simple IP address should be used.

If the downloaded provisioning file includes a DNS IP in the DNS section of the file, the URLs of the files in the provisioning file must be resolved using this DNS server or the DNS server listed in the DHCP offer.

Up to three DNS addresses (primary, secondary, tertiary) can be listed in the DNS sections of the provisioning file. If all three DNS addresses are listed, they are attempted in the listing order to resolve the file URLs.

Up to three locations can be set for each file type, using the primary-url, secondary-url, and tertiary-url fields. The auto-provisioning process attempts to download all files using the primary-url information for each file. If this attempt is unsuccessful, the process reattempts using the secondary-url information for each file. If this attempt is not successful, the process reattempts using the tertiary-url information.

A ZTP operation is considered successful when all files named in the provisioning file are downloaded. If all file locations are attempted and all named files are not successfully downloaded, the auto-provisioning process fails and ZTP reattempts the provisioning process using the next valid interface.

The ZTP timeout is the total time allowed for the ZTP process to execute successfully. If the ZTP process fails to run successfully within the ZTP timeout duration, the node is rebooted and the ZTP process is retried. The default timeout is 30 minutes.

See Options and option modification for information about how to configure the ZTP timeout using CLI commands.

ZTP timeout information can be included in the provisioning file to configure a value different from the default. For a provisioning file example, see Example provisioning file.

set {
  timeout hours 1
  timeout minutes 30
}

The image file can be downloaded in the following ways:

• Extract the images from the OLCS CFLASH file and explicitly list them in the provisioning file. Each of the .tim files is validated and the version of the software is checked. The hash-value must be explicitly listed.

  download {
    image "$(BOOT-PATH)/both.tim" {
      primary-url "$(FILESERVER)/both.tim"
        verification {
            hash-algorithm sha256
            hash-value "53473e2727caf55f3a38fa466622af2147762e26a8587e9248240a572cdee849"
        }
  
    }
    ……..
  }
  

• Zip the image files and perform a single image download. ZTP can extract the images and verify the .tim file and software version. An additional file can also be listed for all files to be verified using sha256 or md5.

  download {
    image "$(BOOT-PATH)/images.zip" {
        primary-url "$(FILESERVER)/images.zip"
        unzip
        verification {
            hash-algorithm sha256
            hash-value "53473e2727caf55f3a38fa466622af2147762e26a8587e9248240a572cdee849"
        }
    }
    # SHA-256 Checksum Files - File of a list of SHAR256 checksum values and file names (must be absolute)
    sha256 "$(BOOT-PATH)/somefile.txt" {
        primary-url "$(FILESERVER)/sha256-checksums.txt"
        verification {
            hash-algorithm sha256
            hash-value "53473e2727caf55f3a38fa466622af2147762e26a8587e9248240a572cdee849"
        }
    }
  }
  

This section provides examples of provisioning file information.

set {
  hours 1
  minutes 30
}
dns {
      primary 192.0.2.1
      secondary 192.0.2.2
      tertiary 192.0.2.3
      domain sample.domain.com
}
download {
  image "cf3:/both.tim" {
    primary-url "http://192.168.40.140:81/both.tim"
    secondary-url "http://192.168.40.140:81/both.tim"
    tertiary-url "http://192.168.40.140:81/both.tim"
  }
  image "cf3:/support.tim" {
    primary-url "http://192.168.40.140:81/support.tim"
    secondary-url "http://192.168.40.140:81/support.tim"
    tertiary-url "http://192.168.40.140:81/support.tim"
  }
  config "cf3:/config.cfg" {
    primary-url "ftp://ftpserv:name@192.168.194.50/./images/dut-a.cfg"
    secondary-url "http://192.168.41.140:81/dut-a.cfg"
    tertiary-url "http://192.168.42.140:81/dut-a.cfg"
  }
  file "cf3:/license.txt" {
    primary-url "ftp://ftpserv:name@192.168.194.50/./images/provision_example.cfg"
    secondary-url "http://192.168.41.140:81/dut-a.cfg"
    tertiary-url "http://192.168.42.140:81/dut-a.cfg"
  }
}
bof {
  primary-image cf3:/both.tim
  primary-config cf3:/config.tim
  address 192.168.100.1 active
  autonegotiate
  duplex full
  speed 100
  wait 3
  persist off
  console-speed 115200
}

import {
    client {
        cert "cf3:/client.crt" {
            format pem 
            primary-url http://10.10.10.67:81/client.crt 
        } 
        key "cf3:/client.key" { 
            format pem 
            primary-url http://10.10.10.67:81/client.key 
        } 
    } 
    trust-anchor TRUST_ANCHOR{ 
        cert "cf3:/ca.crt" { 
            format pem 
            primary-url ftp://user-name:password@10.10.10.66//user-name/logs/
fileserver-4/ca.crt 
        } 
        crl "cf3:/ca.crl" { 
            format der 
            primary-url ftp://user-name:password@10.10.10.66//user-name/logs/
fileserver-4/ca.crl 
        } 
    } 
} 
<snip> 
    download { 
      config "cf3:/ztp/ztp_dut-a.cfg" { 
        primary-url "https://10.10.10.64:81/ztp_node-2.cfg" 
        primary-trust-anchor "TRUST_ANCHOR" 
  } 

HTTP and HTTPS can connect to public servers using a proxy. The proxy is in URL format and the URL must be resolved using the provisioning file DNS.

The proxy can include a username and password. Proxy Auto-Configuration (PAC) is not supported.

Proxy information formatting is as follows:

http://user@hostname:file-path

https://user@hostname:file-path

proxy http://ip-or-url user@hostname:port

The HTTP (or HTTPS) proxy support information is included in file commands and in the ZTP provisioning file. The following example shows HTTP proxy information in the provisioning file.

image "cf3:/both.tim" {
    primary-url "http://200.150.40.140:81/both.tim" 
    secondary-url "http://200.150.40.140:81/both.tim"
    tertiary-url "http://200.150.40.140:81/both.tim"
    primary-proxy http://132.2.3.1:8080
    secondary-proxy http://133.3.4.1:8080
  }

On multi-slot routers, the discovered IOMs, MDAs, IMMs, and fabric (that is, the discovered hardware) are part of the day 0 configuration. ZTP automatically adds the CLI configuration to the correct location in the day 0 configuration in the provisioning file.

This process ensures that the nodes for different sites are populated with the correct hardware, regardless of what physical hardware is used for a specific site.

The day 0 configuration process for multi-slot routers is as follows:

1. The provider creates a day 0 configuration template with the required services and interfaces.
2. ZTP provisions the IOMs, MDAs, IMMs, and fabric.
3. ZTP discovers the port and VLAN on which the DHCP server is connected.
4. ZTP adds the discovered and provisioned hardware to the day 0 configuration in the provisioning file

5. ZTP uses the day 0 configuration in provisioning file to create an initial configuration file and saves it to the specified location on the SD card of the node.

   The BOF section of the provisioning file must specify the configuration file generated from the day 0 configuration.

6. After rebooting, the node boots up using the day 0 configuration file.

Use symbols to generalize and reuse a single provisioning file across multiple ZTP deployments and for value substitutions and conditions in the day 0 configuration template of the provisioning files. The same symbol can appear as many times as required and can be used for both conditional statements and substitutions.

The following is a sample of a day 0 configuration template included in a provisioning file.

# Generate Day-0 Configuration
# ----------------------------
Write "cf3:/config-template.cfg" {
#########################
# !!DAY0 CONFIG START!! #
#########################
exit all
configure
#--------------------------------------------------
echo "System Configuration"
#--------------------------------------------------
    system
        name "chassis-name"
        snmp
        exit
        login-control
            idle-timeout disable
        exit
        time
            ntp
                server 192.168.194.202
                no shutdown
            exit
        exit
        thresholds
            rmon
            exit
        exit
    exit
#--------------------------------------------------
echo "System Security Configuration"
#--------------------------------------------------
    system
        security
            telnet-server
            ftp-server
            snmp
                community "private" rwa version both
                community "public" r version both
            exit
        exit
    exit
#--------------------------------------------------
echo "Log Configuration"
#--------------------------------------------------
    log
        snmp-trap-group 90
            trap-target ‟”
        exit
        log-id 90
            from main change
            to snmp
        exit
    exit
exit all
configure
    router
#--------------------------------------------------
echo "IP Configuration"
#--------------------------------------------------
        interface "IF-1/1/c1/1"
            port $(ztp.bootstrap.uplink)
            address 192.168.0.1/31
            ipv6
            exit
            no shutdown
        exit
    exit
exit all
#########################
# !!DAY0 CONFIG END!!   #
#########################
}
# Generate New BOF File
# ---------------------
bof {
  primary-image  cf3:/image.both
  primary-config cf3:/config-template.cfg
  autonegotiate
  duplex full
  speed 100
  wait 3
  persist off
  console-speed 115200
}

The conveyed information file (also referred to as conveyed-info.ztp file) contains the certificates, keys, and trust anchors required to establish the TLS connection. This is the minimum information that the node requires to start SZTP after staging at the installation site. The initial file must be added to cf3: by copying it on the CF manually or using regular ZTP procedures and the provisioning file.

import {
   client {
      cert "cf3:/artifacts/node.cert"
      key "cf3:/artifacts/node.key" {
         format der
      }
   }
   trust-anchor BOOTSERVER {
      cert "cf3:/artifacts/bootserver.cert"
   }
}

import {
   client {
      encrypt
      cert "cf3:/artifacts/node.cert"
      key "cf3:/artifacts/node.key" {
         format der
      }
   }
   trust-anchor BOOTSERVER {
      encrypt
      cert "cf3:/artifacts/bootserver.cert"
   }
}

import {
   client {
      encrypt
      cert "cf3:/artifacts/node.cert"
      key "cf3:/artifacts/node.key" {
         format der
      }
   }
   trust-anchor BOOTSERVER {
      encrypt
      cert "cf3:/artifacts/bootserver.cert"
   }
}

redirect-information {
   boot-server "https://mybootserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
   boot-server "https://backupserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
}

import {
   client {
      encrypt
      cert "cf3:/artifacts/node.cert"
      key "cf3:/artifacts/node.key" {
         format der
      }
   }
   trust-anchor BOOTSERVER {
      encrypt
      cert "cf3:/artifacts/bootserver.cert"
   }
}

redirect-information {
   boot-server "https://mybootserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
   boot-server "https://backupserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
}
onboarding-information {
   boot-image
      download-uri https://images.com/$(sys.platform).zip
   pre-configuration-script "https://config.com/provisioning.cfg"
}

The onboarding information is required to obtain all critical resources to boot the node in the normal mode of operation with the latest boot image and configuration. When processing the onboarding information, the device must first process the boot image information (if any), then execute the preconfiguration script (if any).

The onboarding information is present in the conveyed information file only. See Conveyed information.

onboarding-information {
   boot-image
      download-uri https://images.com/$(sys.platform).zip
   pre-configuration-script "https://config.com/provisioning.cfg"
}

onboarding-information {
   boot-image {
      # Download-URI(Mandatory): URL to ZIP file of images. Up to 3 for primary, secondary, tertiary
      # -> Base directory is "cf3:/"
      download-uri "https://server.download.com/images.zip"
      download-uri "https://server2.download.com/images.zip"
      download-uri "https://server3.download.com/images.zip"
      # VERIFICATION (Optional): File within ZIP file to verify images
      image-verification {
         hash-algorithm sha256
         hash-value "53473e2727caf55f3a38fa466622af2147762e26a8587e9248240a572cdee849"
      }
   }

   # Another provisioning file that can be loaded. Downloaded as "cf3:/autoboot.cfg.1,2,3" based on chaining
   pre-configuration-script "https://server.file.com/someotherprovisioningfile.txt"
}

After SR OS authenticates successfully to the bootstrap server, the node can download the conveyed information using HTTPS. The operator can choose the name of the conveyed information file.

The SR OS conveyed information is trusted and does not require an additional signature verification.

onboarding-information {
   boot-image
      download-uri https://images.com/$(sys.platform).zip
   pre-configuration-script "https://config.com/provisioning.cfg"
}

onboarding-information {
   boot-image
      download-uri https://images.com/$(sys.platform).zip
   pre-configuration-script "https://config.com/provisioning.cfg"
}
import {
   client {
      cert "cf3:/artifacts/node.cert"
      key "cf3:/artifacts/node.key" {
         format der
      }
   }
   trust-anchor BOOTSERVER {
      cert "cf3:/artifacts/bootserver.cert"
   }
}

redirect-information {
   boot-server "https://mybootserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
   boot-server "https://backupserver.com/" {
      port 50
      trust-anchor BOOTSERVER
      file "conveyed.info"
   }
}