Verifying the vSIM installation

Overview

This section describes the basic procedures for verifying your vSIM virtual machine (VM) installation. Common problems that you may encounter are highlighted and possible solutions to resolve these issues are provided.

Note: This section provides instructions on verifying and troubleshooting VMs deployed on Linux hosts using the QEMU-KVM hypervisor. Similar techniques can also be applied to VMs deployed in a VMware environment.

Verifying host details

Successful installation of a vSIM VM requires the host machine to be set up properly. Use the commands described in this section to display host information for Linux systems (running Centos, Red Hat).

General system information

To display the Linux kernel version, enter the following:

uname -a ↵

To verify that your Linux kernel is 64-bit, enter the following:

uname -m ↵

The command output should be x86_64.

Linux distribution type

To display the type of Linux distribution and version, enter the following:

lsb_release -a ↵

Note: Depending on your Linux distribution, you may have to install a package such as redhat-lsb-core to run this command.

PCI devices

To view all PCI devices, enter the following:

lspci ↵

A partial example output of the command is as follows:

[user@host ~]# lspci
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
05:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
06:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

The first entry indicates that there is a PCI device attached to bus 04, with device ID 00 and function 0 (04:00.0) and that it is an 82574L Gigabit Ethernet controller made by Intel Corporation.

To view PCI device details, including capabilities such as the maximum bus speed and the number of lanes (for example x4), enter the following:

lspci -vvv ↵

CPU processor information

To view details about all the CPU processors available to the host, enter the following:

cat /proc/cpuinfo ↵

When hyper-threading is enabled on Intel CPUs, every hyper-thread appears as a separate processor, as shown in the following partial example output:

[user@host ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
stepping        : 4
cpu MHz         : 2593.614
cache size      : 15360 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 x2apic popcnt aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 5187.22
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

To run vSIM VMs, the cpu family value must be 6 (Intel) and the model should be greater than or equal to 42 (in most cases). In addition, several CPU flags are critical for the vSIM and must be passed through to the guest. These include:

x2apic
support for an advanced interrupt controller introduced with Intel Nehalem processors. Support for the x2apic is mandatory; the flag must not be emulated.
lm
long mode, indicating a 64-bit CPU, which is necessary to support 64-bit guests
vmx
support for Intel virtualization technologies such as VT-d/VT-x

Host memory

To view details about the host memory, enter the following:

cat /proc/meminfo ↵

The following is a partial example output of this command:

[user@host ~]# cat /proc/meminfo
MemTotal:       16406144 kB
MemFree:         9442676 kB
MemAvailable:   11272708 kB
Buffers:          648220 kB
Cached:          1352744 kB
SwapCached:            0 kB
Active:          4898888 kB
Inactive:        1723664 kB

The MemFree value must be at least 4194304 kB if you want to create another vSIM VM on this host.

Host capability

To show summary information about the host and its virtualization capabilities, enter the following:

virsh capabilities ↵

Note: The libvirt package must be installed to run this command on the host.

The command output must confirm that the system is capable of supporting guests with the x86_64 architecture (64-bit guests).

QEMU and libvirt information

To show libvirt and QEMU version information, enter the following:

virsh version ↵

Note: The libvirt package must be installed to run this command on the host.

Loaded modules

To list all the kernel modules installed on the host, enter the following:

lsmod ↵

Some key modules are: bridge, kvm, kvm_intel, vhost_net, tun, macvtap and openvswitch.

Host virtualization setup

To check that dependencies for virtualization are installed correctly on the host, enter the following:

virt-host-validate ↵

The following is an example output of the command:

[user@host ~]# virt-host-validate 
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking for device /dev/kvm                                         : PASS
  QEMU: Checking for device /dev/vhost-net                                   : PASS
  QEMU: Checking for device /dev/net/tun                                     : PASS
   LXC: Checking for Linux >= 2.6.26                                         : PASS
[user@host ~]#

Verifying the creation of VMs

Before attempting to log in to a vSIM and to check for successful boot of its VMs, ensure that the VMs have been created as expected on all the host machines.

If you are using libvirt, you can view the list of VMs on a specific host by entering the following command:

virsh list ↵

The following is an example output of this command:

[user@host ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 1     CPMA                           running
 2     CPMB                           running
 3     IOM1                           running

Because each QEMU-KVM VM is a process with two or more threads, you can also use a sequence of commands, such as the following, to get more details about a running VM:

[user@host ~]# ps -ef | grep CPMA
qemu      6304     1  5 Sep10 ?        05:03:50 /usr/libexec/qemu-kvm.real 
name CPMA -S -machine rhel6.0.0, accel=kvm, usb=off 
cpu SandyBridge, +erms, +smep, +fsgsbase, +rdrand, +f16c, +osxsave, +pcid, +pdcm, +
tpr, +tm2, +est, +smx, +vmx, +ds_cpl, +monitor, +dtes64, +pbe, +tm, +ht, +ss, + acp
, + ds, +vme -m 6144 -realtime mlock=off -smp 2, sockets=2, cores=1, threads=1 
uuid nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn -nographic -no-user-config -nodefaults 
chardev socket, id=charmonitor, path=/var/lib/libvirt/qemu
CPMA.monitor, server, nowait -mon chardev=charmonitor, id=monitor, mode=control 
rtc base=utc -no-kvm-pit-reinjection -no-shutdown -no-acpi -boot strict=on 
device piix3-usb-uhci, id=usb, bus=pci.0, addr=0x1.0x2 -device virtio-serial
pci, id=virtio-serial0, bus=pci.0, addr=0x7 -drive file=/path
disk1.qcow2, if=none, id=drive-virtio-disk0, format=qcow2, cache=none 
device virtio-blk-pci, scsi=off, bus=pci.0, addr=0x8, drive=drive-virtio
disk0, id=virtio-disk0 -netdev tap, fd=23, id=hostnet0, vhost=on, vhostfd=24 
device virtio-net
pci, netdev=hostnet0, id=net0, mac=nn:nn:nn:nn:nn:nn, bus=pci.0, addr=0x3, bootinde
=1 -netdev tap, fd=25, id=hostnet1, vhost=on, vhostfd=26 -device virtio-net
pci, netdev=hostnet1, id=net1, mac=nn:nn:nn:nn:nn:nn, bus=pci.0, addr=0x4 
chardev socket, id=charconsole0, host=0.0.0.0, port=2500, telnet, server, nowait 
device virtconsole, chardev=charconsole0, id=console0 -device virtio-balloon
pci, id=balloon0, bus=pci.0, addr=0x6
[user@host ~]# ps -T 6304
root@bksim4204 6304]# ps -T 6304
  PID  SPID TTY      STAT   TIME COMMAND
 6304  6304 ?        Sl     0:19 /usr/libexec/qemu-kvm.real -name CPMA -S -machine rhel6.0.0,accel=k
 6304  6310 ?        Sl   169:47 /usr/libexec/qemu-kvm.real -name CPMA -S -machine rhel6.0.0,accel=k
 6304  6311 ?        Sl   134:52 /usr/libexec/qemu-kvm.real -name CPMA -S -machine rhel6.0.0,accel=k

These example command outputs indicate that the VM called CPMA is running as process ID 6304 in the host machine. There are three threads associated with this process.

You can obtain a real-time view of the host system impact of all running VMs by entering the following commands:

top ↵

htop ↵

The following is an example output of the command:

[root@bksim4204 bin]# top
top - 14:00:10 up 5 days,  4:44,  2 users,  load average: 4.09, 4.08, 4.09
Tasks: 184 total,   2 running, 182 sleeping,   0 stopped,   0 zombie
%Cpu(s): 51.2 us,  0.1 sy,  0.0 ni, 48.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  16406144 total,  6970448 used,  9435696 free,   649488 buffers
KiB Swap:        0 total,        0 used,        0 free.  1328612 cached Mem

  PID USER    PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND        
               
 6349 qemu    20  0  4860828 2.445g   5912 S 402.1 15.6   24522:22 qemu-kvm.real                  
 6304 qemu    20  0  6736452 0.981g   5916 S   4.7  6.3  305:44.16 qemu-kvm.real                  
 6321 qemu    20  0  6736424 0.980g   5916 S   4.3  6.3  280:32.31 qemu-kvm.real                  
 6308 root    20  0        0      0      0 S   0.3  0.0    4:45.88 vhost-6304                     
 6324 root    20  0        0      0      0 S   0.3  0.0    1:23.12 vhost-6321

Verifying host networking

You can use different methods to provide network connectivity between the vSIM VMs and external destinations.

Host network troubleshooting lists some useful commands that can help you troubleshoot networking at the host level.

Table 1. Host network troubleshooting
Command syntax Description

ip -d link show

Shows details of all host network interfaces, including physical NIC ports and logical interfaces, such as vNIC interface constructs on the host

ip link set dev <interface-name> mtu <value>

Explicitly sets the MTU (Maximum Transmit Unit) of a host interface

You are required to set the MTU of network interfaces associated with vSIM internal fabric interfaces to 9000 bytes

ip addr show

Displays the IP addresses associated with host network interfaces

Note: In a virtualization environment, many interfaces do not have any IP addresses assigned to them.

ip route show

Displays the IP routing table of the host

tcpdump -i <interface name>

Captures packets on the selected interface and outputs them for analysis

brctl show

Displays all the Linux bridges

ovs-vsctl show

Displays all the Open vSwitch bridges

ethtool -S <interface name>

Displays statistics collected by the physical NIC for a selected interface

Verifying vSIM software

After you have verified that the vSIM VMs have been created successfully on the respective hosts, check the SR OS operating system to verify that it has booted up properly on each VM and that the vSIM is functional. You will typically need console access to the vSIM to perform these checks. This is the main reason for adding a serial console port to vSIM VMs. With console access, you can log in to each vSIM and perform the checks that are described in this section.

Check the status of the system BOF

The output of this command depends on the SMBIOS text string used for the VM and the saved BOF configuration.

MD-CLI

Use the following command to check the status of the system BOF.

admin show configuration bof

Status of the system BOF

# TiMOS-C-22.10.B1-25 cpm/x86_64 Nokia 7750 SR Copyright (c) 2000-2022 Nokia.
# All rights reserved. All use subject to applicable license agreements.
# Built on Fri Dec 2 18:20:32 PST 2022 by builder in /builds/c/2210B/B1-25/panos/main/sros
# Configuration format version 22.10 revision 0

# Generated MON DEC 05 20:38:08 2022 UTC

bof {
    configuration {
        primary-location "ftp://*:*@[192.168.193.192]/./dut-a.cfg"
    }
    console {
        speed 115200
    }
    image {
        primary-location "cf3:\timos\"
    }
    li {
        local-save false
        separate false
    }
    license {
        primary-location "ftp://*:*@[192.168.193.192]/./license.txt"
    }
    router "management" {
        interface "management" {
            cpm active {
                ipv4 {
                    ip-address 192.168.231.86
                    prefix-length 18
                }
                ipv6 {
                    ipv6-address 3000::c0a8:e756
                    prefix-length 114
                }
            }
            cpm standby {
                ipv4 {
                    ip-address 192.168.231.87
                    prefix-length 18
                }
                ipv6 {
                    ipv6-address 3000::c0a8:e757
                    prefix-length 114
                }
            }
        }
        static-routes {
            route 135.0.0.0/8 {
                next-hop 192.168.193.191
            }
            route 138.0.0.0/8 {
                next-hop 192.168.193.191
            }
        }
    }
    system {
        base-mac-address fa:ac:ff:ff:10:00
        fips-140-2 false
        persistent-indices false
    }
}

# Finished MON DEC 05 20:38:08 2022 UTC

Classic CLI

Use the following command to check the status of the system BOF.

show bof

Status of the system BOF

===============================================================================
BOF (Memory)
===============================================================================
    primary-image    cf3:\timos\
    primary-config   ftp://*:*@[192.168.193.192]/./dut-a.cfg
    license-file     ftp://*:*@[192.168.193.192]/./license.txt
    address          192.168.231.86/18 active
    address          192.168.231.87/18 standby
    address          3000::c0a8:e756/114 active
    address          3000::c0a8:e757/114 standby
    static-route     135.0.0.0/8 next-hop 192.168.193.191
    static-route     138.0.0.0/8 next-hop 192.168.193.191
    autonegotiate
    duplex           full
    speed            100
    wait             3
    persist          off
    no li-local-save
    no li-separate
    no fips-140-2
    console-speed    115200
    system-base-mac  fa:ac:ff:ff:10:00
===============================================================================

Check the chassis type

Verify that the vSIM VM chassis type is set correctly. If the chassis type does not match the one encoded in the SMBIOS text string, there is likely an error in the SMBIOS text string. Use the following command to show the chassis information.

show chassis

Chassis information

===============================================================================
System Information
===============================================================================
  Name                              : Dut-A
  Type                              : 7750 SR-12
  Chassis Topology                  : Standalone
  Location                          : (Not Specified)
  Coordinates                       : (Not Specified)
  CLLI code                         :           
  Number of slots                   : 12
  Oper number of slots              : 12
  Num of faceplate ports/connectors : 148
  Num of physical ports             : 128
  Critical LED state                : Off
  Major LED state                   : Off
  Minor LED state                   : Off
  Over Temperature state            : OK
  Base MAC address                  : fa:ac:ff:ff:10:00
  Admin chassis mode                : d
  Oper chassis mode                 : d
  Fabric Speed                      : S4
  FP Generations                    : FP3 FP4 
  System Profile                    : none

===============================================================================
Chassis Summary
===============================================================================
Chassis   Role                Status
-------------------------------------------------------------------------------
1         Standalone          up
===============================================================================

Check the card types equipped in the system

Verify that correct (virtualized) card types are equipped in the system. If a card type does not match the one encoded in the SMBIOS text string of the corresponding VM, there is likely an error in that SMBIOS text string. Use the following command to show the card information.

show card

Card information

=============================================================================== 
Card Summary
===============================================================================
Slot      Provisioned Type                         Admin Operational   Comments
              Equipped Type (if different)         State State         
-------------------------------------------------------------------------------
1         iom5-e:he1200g+                          up    up            
2         iom5-e:he1200g+                          up    up            
3         iom4-e-b                                 up    up            
4         iom4-e-hs                                up    up            
5         imm48-1gb-sfp-c                          up    up            
6         iom4-e                                   up    up            
7         imm-2pac-fp3                             up    up            
8         iom4-e                                   up    up            
9         iom4-e                                   up    up            
10        imm-2pac-fp3                             up    up            
A         cpm5                                     up    up/active     
B         cpm5                                     up    up/standby    
===============================================================================

Check the vSIM system licenses

Verify that the vSIM has valid licenses. Use the following command to show the system license information.

show system license

System license information

===============================================================================
System  License
===============================================================================
License status : monitoring, valid license record
Time remaining : 141 days 10 hours
===============================================================================
===============================================================================
Active  License [CPM A]
===============================================================================
License status : monitoring, valid license record
Time remaining : 141 days 10 hours
-------------------------------------------------------------------------------
License name   : name@organization.com
License uuid   : cb0ba837-07db-4ebb-88ea-694271754675
Machine uuid   : cb0ba837-07db-4ebb-88ea-694271754675
License desc   : vSIM license1a
License prod   : Virtual-SIM
License sros   : TiMOS-[BC]-15.0.*
Current date   : WED JUL 12 14:21:52 UTC 2017
Issue   date   : THU JUN 01 22:57:48 UTC 2017
Start   date   : THU JUN 01 00:00:00 UTC 2017
End     date   : FRI DEC 01 00:00:00 UTC 2017
===============================================================================
===============================================================================
Standby License [CPM B]
===============================================================================
License status : monitoring, valid license record
Time remaining : 141 days 10 hours
-------------------------------------------------------------------------------
License name   : name@organization.com
License uuid   : 60ca42cf-d45a-4124-afd0-81057f167bf4
Machine uuid   : 60ca42cf-d45a-4124-afd0-81057f167bf4
License desc   : vSIM license1b
License prod   : Virtual-SIM
License sros   : TiMOS-[BC]-15.0.*
Current date   : WED JUL 12 14:21:52 UTC 2017
Issue   date   : THU JUN 01 22:57:48 UTC 2017
Start   date   : THU JUN 01 00:00:00 UTC 2017
End     date   : FRI DEC 01 00:00:00 UTC 2017
===============================================================================