SR Linux applications

The SR Linux-provided default cgroup profile is located in the /opt/srlinux/appmgr/cgroup_profile.json directory.

If the default cgroup profile fails to parse or be read by the app_mgr, the SR Linux does not start.

The following shows the default cgroup_profile.json file definition:

{
 "profiles": [
  {
   "name": "workload.slice",
   "path": "workload.slice",
   "controller": {
    "memory": {
     "max": 0.8,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "1024"
    },
    "cpu": {
     "weight": "1000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "all",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.primary",
   "path": "workload.slice/primary",
   "controller": {
    "memory": {
     "max": 0.8,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "1024"
    },
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "all",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.primary.nondatapath",
   "path": "workload.slice/primary/nondatapath",
   "controller": {
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.primary.datapath",
   "path": "workload.slice/primary/datapath",
   "controller": {
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.primary.devicemgr",
   "path": "workload.slice/primary/devicemgr",
   "controller": {
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.primary.appmgr",
   "path": "workload.slice/primary/appmgr",
   "controller": {
    "cpu": {
     "weight": "1000",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.secondary",
   "path": "workload.slice/secondary",
   "controller": {
    "memory": {
     "max": 0.5,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "0"
    },
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.secondary.default",
   "path": "workload.slice/secondary/default",
   "controller": {
    "memory": {
     "max": 0.5,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "0"
    },
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "workload.secondary.eventmgr",
   "path": "workload.slice/secondary/eventmgr",
   "controller": {
    "memory": {
     "max": 0.1,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "0"
    },
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  },
  {
   "name": "userload.default",
   "path": "userload.slice/default",
   "controller": {
    "memory": {
     "max": 0.25,
     "swap_max": 0,
     "low": 0,
     "max_unoccupied_mem": "0"
    },
    "cpu": {
     "weight": "100",
     "period": "100000",
     "quota": "0"
    },
    "cpuset": {
     "cpus": "",
     "mems": ""
    }
   }
  }
 ]
}

Default cgroup profile parameters describes the default cgroup profile parameters.

Customers can configure cgroup profiles in the /etc/opt/srlinux/appmgr/cgroup_profile.json directory. The app_mgr creates this directory at boot up if it does not exist.

If a customer-defined cgroup profile fails to load, the system continues to function and applications that are loaded into the customer cgroup are loaded using the following Nokia default behaviors:

• Nokia-written applications run in the workload.slice/primary cgroup along with any processes that are started by linuxadmin, including sr_cli.

• Non-Nokia-written applications run in the workload.slice/secondary cgroup. If a customer builds an application and launches it using the app_mgr without specifying a cgroup, the application runs in this cgroup.

• All interactive user applications run in the user.slice/default cgroup, including sr_cli when not started by linuxadmin.

The admin user is treated as any other user in the system. Its processes fall into the user.slice/default cgroup.

Customers can configure up to three cgroups in the /etc/opt/srlinux/appmgr/cgroup_profile.json directory. Customer applications are assigned to these groups. Any more than three configured cgroups are ignored. The depth of cgroups is limited to two levels where, for example, workload is one level and primary/secondary are two levels. Any levels beyond this are also ignored.

If a cgroup with the same name is used in multiple customer-defined profiles, the system ignores it and uses the cgroup defined in the default profile.

The kernel calculates the oom_score using the formula 10 × percentage of memory used by the process. The maximum score is 10 × 100% = 1000. The oom_score of a process can be found in the /proc directory (/proc/$pid/oom_score). An oom_score of 1000 means the process is using all the memory, an oom_score of 500 means it is using half the memory, and an oom_score of 0 means it is using no memory.

The OOM killer checks the oom_score_adj file in the /proc/$pid/oom_score_adj directory to adjust its final calculated score. The default value is 0.

The oom_score_adj value can range from –1000 to 1000. A score of –1000 results in a process using 100% of the memory and in not being terminated by the OOM killer. However, a score of 1000 causes the Linux kernel to terminate the process even when it uses minimal memory. A score of –100 results in a process using 10% of the memory before it is considered for termination, as its score remains 0 until its unadjusted score reaches 100.

The oom_score_adj value for each process is defined in its corresponding YAML definition file. The system groups the processes based on their score, which the SR Linux OOM killer uses as the hierarchy for terminating a rogue process, as follows:

• group1 = –998

  For processes that should never be killed, such as app_mgr, idb_server, and all processes on the IMM.

• group2 = –200

  For processes that ideally should not be killed because doing so has a substantial impact on the system, such as mgmt_server, chassis_mgr, and net_inst_mgr. A score of –200 means the process gets to use 20% of the memory before their memory use starts being counted.

• group3 = 0

  For process that should be killed if they are using too much memory such as BGP, ISIS, and OSPF.

• group4 = 500

  For processes that should be preferentially killed, such as cli and oam_mgr.

OOM adjust score per process lists the OOM adjust score for each process.

The SR Linux provides CLI commands to perform the following:

• check the usage of existing cgroups

• show information about the OOM adjust score of applications managed by the app_mgr

• show information about the cgroups that are associated with the applications that are managed by the app_mgr

• list all of the applications associated with a specified cgroup

The following Linux-provided CLI commands are available for debugging cgroups:

• the systemd-cgls command

• the systemd-cgtop command

The systemd-cgls command dumps the cgroup hierarchy. The following output shows an example of the systemd-cgls command.

[linuxadmin@srlinux unified]$ systemd-cgls
Control group /:
-.slice
├─workload.slice (#6685)
│ ├─primary (#6750)
│ │ ├─nondatapath (#6792)
│ │ │ ├─22368 ./sr_supportd --server-mode
│ │ │ ├─22481 top -b -d 600 -H -i -w 512
│ │ │ ├─23140 ./sr_idb_server
│ │ │ ├─23284 ./sr_aaa_mgr
│ │ │ ├─23350 ./sr_acl_mgr
│ │ │ ├─23401 ./sr_arp_nd_mgr
│ │ │ ├─23465 ./sr_bfd_mgr
│ │ │ ├─23516 ./sr_chassis_mgr
│ │ │ ├─23563 ./sr_dhcp_client_mgr
│ │ │ ├─23602 ./sr_evpn_mgr
│ │ │ ├─23671 ./sr_fib_mgr
│ │ │ ├─23741 ./sr_l2_mac_learn_mgr
│ │ │ ├─23795 ./sr_l2_mac_mgr
│ │ │ ├─23848 ./sr_lag_mgr
│ │ │ ├─23891 ./sr_linux_mgr
│ │ │ ├─23963 ./sr_log_mgr
│ │ │ ├─24023 ./sr_mcid_mgr
│ │ │ ├─24085 ./sr_mfib_mgr
│ │ │ ├─24131 ./sr_mgmt_server
│ │ │ ├─24188 ./sr_net_inst_mgr
│ │ │ ├─24250 ./sr_radius_mgr
│ │ │ ├─24289 ./sr_sflow_sample_mgr
│ │ │ ├─24346 ./sr_stat_id_mgr
│ │ │ ├─25575 ./sr_grpc_server
│ │ │ ├─25614 ./sr_plcy_mgr
│ │ │ ├─25655 ./sr_qos_mgr
│ │ │ └─25693 ./sr_vxlan_mgr
│ │ ├─devicemgr (#6846)
│ │ │ └─22567 ./sr_device_mgr
│ │ ├─datapath (#6819)
│ │ │ └─24385 sr_xdp_lc_1 --slot_num 1 --complex_num 2
│ │ └─appmgr (#6873)
│ │   └─20843 ./sr_app_mgr
│ └─secondary (#6900)
│   └─default (#6942)
│     └─26090 sshd: /usr/sbin/sshd -f /etc/ssh/sshd_config_mgmt

The systemd-cgtop command dumps the current usage of each cgroup. The following output shows an example of the systemd-cgtop command.

Path                                     Tasks   %CPU   Memory  Input/s Output/s
 
/                                          186      -        -        -        -
/system.slice/crond.service                  1      -        -        -        -
/system.slice/dbus.service                   1      -        -        -        -
/system.slice/polkit.service                 1      -        -        -        -
/system.slice/rngd.service                   1      -        -        -        -
/system.slice/sr_watchdog.service            1      -        -        -        -
/system.slice/srlinux.service               53      -        -        -        -
/system.slice/sshd.service                   1      -        -        -        -
/system.slic...ial-getty@ttyS0.service       3      -        -        -        -
/system.slice/systemd-journald.service       1      -        -        -        -
/system.slice/systemd-logind.service         1      -        -        -        -
/system.slice/systemd-udevd.service          1      -        -        -        -
/system.slice/ztpapi.service                 1      -        -        -        -