SR Linux applications

The SR Linux-provided default cgroup profile is located in the /opt/srlinux/appmgr/cgroup_profile.json directory.

If the default cgroup profile fails to parse or be read by the app_mgr, the SR Linux does not start.

The default cgroup_profile.json file definition is shown below:

{
 "profiles": [
  {
   "name": "workload.primary",
   "path": "workload.slice/primary",
   "controller": {
    "memory": {
     "max": 0.8,
     "swap_max": 0,
     "low": 0
    },
    "cpu": {
     "weight": "10000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.datapath",
   "path": "workload.slice/datapath",
   "controller": {
    "memory": {
     "max": 0.8,
     "swap_max": 0,
     "low": 0
    },
    "cpu": {
     "weight": "10000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "all",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.secondary",
   "path": "workload.slice/secondary",
   "controller": {
    "memory": {
     "max": 0.5,
     "swap_max": 0,
     "low": 0
    },
    "cpu": {
     "weight": "10000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "user.default",
   "path": "user.slice/default",
   "controller": {
    "memory": {
     "max": 0.25,
     "swap_max": 0,
     "low": 0
    },
    "cpu": {
     "weight": "1000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  }
 ]
}

Default cgroup profile parameters describes the default cgroup profile parameters.

Customers can configure cgroup profiles in the /etc/opt/srlinux/appmgr/cgroup_profile.json directory. The app_mgr creates this directory at boot up if it does not exist.

If a customer-defined cgroup profile fails to load, the system continues to function and applications that are loaded into the customer cgroup are loaded using Nokia defaults, listed below.

• Nokia-written applications run in the workload.slice/primary cgroup along with any processes that are started by linuxadmin, including sr_cli.

• Non-Nokia-written applications run in the workload.slice/secondary cgroup. If a customer builds an application and launches it using the app_mgr without specifying a cgroup, the application runs in this cgroup

• All interactive user applications run in the user.slice/default cgroup, including sr_cli when not started by linuxadmin.

The admin user is treated as any other user in the system. Its processes fall into the user.slice/default cgroup.

The kernel calculates the oom_score using the formula 10 × percentage of memory used by the process. The maximum score is 10 × 100% = 1000. The oom_score of a process can be found in the /proc directory (/proc/$pid/oom_score). An oom_score of 1000 means the process is using all the memory, an oom_score of 500 means it is using half the memory, and an oom_score of 0 means it is using no memory.

The OOM killer checks the oom_score_adj file in the /proc/$pid/oom_score_adj directory to adjust its final calculated score. The default value is 0.

The oom_score_adj value can range from -1000 to 1000. A score of -1000 results in a process using 100% of the memory and in not being terminated by the OOM killer. However, a score of 1000 causes the Linux kernel to terminate the process even when it uses minimal memory. A score of -100 results in a process using 10% of the memory before it is considered for termination, as its score remains 0 until its unadjusted score reaches 100.

The oom_score_adj value for each process is defined in its corresponding YAML definition file. The system groups the processes based on their score, which the SR Linux OOM killer uses as the hierarchy for terminating a rogue process, as follows:

• group1 = -998, for processes that should never be killed, such as app_mgr, idb_server, and all processes on the IMM.

• group2 = -200, for processes that ideally should not be killed because doing so has a substantial impact on the system, such as mgmt_server, chassis_mgr, and net_inst_mgr. A score of -200 means the process gets to use 20% of the memory before their memory use starts being counted.

• group3 = 0, for process that should be killed if they are using too much memory such as BGP, ISIS, and OSPF.

• group4 = 500, for processes that should be preferentially killed, such as cli and oam_mgr

OOM adjust score per process lists the OOM adjust score for each process.

The SR Linux provides CLI commands to perform the following:

• check the usage of existing cgroups

• show information about the OOM adjust score of applications managed by the app_mgr

• show information about the cgroups that are associated with the applications that are managed by the app_mgr

• list all of the applications associated with a specified cgroup

The following Linux-provided CLI commands are available for debugging cgroups:

• the systemd-cgls command

• the systemd-cgtop command

The systemd-cgls command dumps the cgroup hierarchy. The output below shows an example of the systemd-cgls command.

[linuxadmin@srlinux unified]$ systemd-cgls
+-1 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
+-system.slice
  +-rngd.service
  | +-2725 /sbin/rngd -f
  +-systemd-udevd.service
  | +-868 /usr/lib/systemd/systemd-udevd
  +-system-serial\x2dgetty.slice
  | +-serial-getty@ttyS0.service
  |   +-1120 login -- linuxadmin
  |   +-8320 -bash
  |   +-9262 sendacct "systemd-cgls"
  |   +-9263 systemd-cgls
  |   +-9264 less
  +-srlinux.service
  | +- 3092 /usr/bin/sudo /opt/srlinux/bin/sr_linux
  | +- 3120 runuser --user srlinux --group srlinux -- /opt/srlinux/bin/sr_linux
  | +- 4019 /bin/bash /opt/srlinux/bin/sr_linux --user-env-switched
  | +- 4039 ./sr_app_mgr
  | +- 4051 ./sr_supportd --server-mode
  | +- 4068 ./sr_device_mgr
  | +- 4219 ./sr_idb_server
  | +- 4229 ./sr_eth_switch
  | +- 4288 ./sr_aaa_mgr
  | +- 4299 ./sr_acl_mgr
  | +- 4314 ./sr_arp_nd_mgr
  | +- 4324 ./sr_chassis_mgr
  | +- 4337 ./sr_dhcp_client_mgr
  | +- 4365 ./sr_evpn_mgr
  | +- 4381 ./sr_fib_mgr
  | +- 4395 ./sr_l2_mac_learn_mgr
  | +- 4411 ./sr_l2_mac_mgr
  | +- 4423 ./sr_lag_mgr
  | +- 4445 ./sr_linux_mgr
  | +- 4494 ./sr_log_mgr
  | +- 4510 ./ntpd -c /etc/ntps.conf -g
  | +- 4526 ./sr_mcid_mgr
  | +- 4574 ./sr_mgmt_server
  | +- 4608 ./sr_net_inst_mgr
  | +- 4626 ./sr_oam_mgr
  | +- 4640 ./sr_sdk_mgr
  | +- 4658 ./sr_sflow_sample_mgr
  | +- 4681 ./sr_xdp_cpm
  | +- 5197 /usr/bin/sudo -Enu root /usr/bin/sudo -Enu gnmirpc bash -c ./sr_gnmi
  | +- 5224 /usr/bin/sudo -Enu root /usr/bin/sudo -Enu jsonrpc bash -c ./sr_json
  | +- 5243 /usr/bin/sudo -Enu gnmirpc bash -c ./sr_gnmi_server
  | +- 5248 ./sr_qos_mgr
  | +- 5261 /usr/bin/sudo -Enu jsonrpc bash -c ./sr_json_rpc
  | +- 5290 ./sr_gnmi_server
  | +- 5302 ./sr_json_rpc
  | +- 5693 /usr/sbin/sshd -f /etc/ssh/sshd_config_mgmt
  | +- 6361 /usr/sbin/rsyslogd -i /var/run/srlinux/rsyslogd.pid
  | +-21936 ./sr_bfd_mgr
  | +-22018 ./sr_bgp_mgr
  | +-22132 ./sr_isis_mgr
  | +-22242 ./sr_lldp_mgr
  | +-22321 ./sr_mpls_mgr
  | +-22437 ./sr_ospf_mgr
  | +-22588 ./sr_plcy_mgr
  | +-22701 ./sr_static_route_mgr
  | +-22796 sr_json_wkr
  | +-22797 python3 -m srlinux.mgmt.json_rpc.json_rpc_server_main --bind=[::]:40
  | +-22953 ./ntpd -c /etc/mntp.conf -g
  | +-23023 python3 -m srlinux.mgmt.json_rpc.json_rpc_server_main --bind=[::]:40
  | +-23736 sr_json_wkr
  | +-23741 python3 -m srlinux.mgmt.json_rpc.json_rpc_server_main --bind=[::]:40
  | +-24009 python3 -m srlinux.mgmt.json_rpc.json_rpc_server_main --bind=[::]:40
  | +-27488 ./dnsmasq --conf-file=/etc/dnsmasq.conf
  +-polkit.service
  | +-2701 /usr/lib/polkit-1/polkitd --no-debug
  +-ztpapi.service
  | +-1114 /opt/srlinux/ztp/virtual-env/bin/python -m ztp.ztphttp
  +-systemd-journald.service
  | +-840 /usr/lib/systemd/systemd-journald
  +-sshd.service
  | +-985 /usr/sbin/sshd -D
  +-crond.service
  | +-1133 /usr/sbin/crond -n
  +-sr_watchdog.service
  | +-1155 sr_wd
  +-dbus.service
  | +-991 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile
  +-systemd-logind.service
    +-1062 /usr/lib/systemd/systemd-logind

The systemd-cgtop command dumps the current usage of each cgroup. The output below shows an example of the systemd-cgtop command.

Path                                     Tasks   %CPU   Memory  Input/s Output/s
 
/                                          186      -        -        -        -
/system.slice/crond.service                  1      -        -        -        -
/system.slice/dbus.service                   1      -        -        -        -
/system.slice/polkit.service                 1      -        -        -        -
/system.slice/rngd.service                   1      -        -        -        -
/system.slice/sr_watchdog.service            1      -        -        -        -
/system.slice/srlinux.service               53      -        -        -        -
/system.slice/sshd.service                   1      -        -        -        -
/system.slic...ial-getty@ttyS0.service       3      -        -        -        -
/system.slice/systemd-journald.service       1      -        -        -        -
/system.slice/systemd-logind.service         1      -        -        -        -
/system.slice/systemd-udevd.service          1      -        -        -        -
/system.slice/ztpapi.service                 1      -        -        -        -