Why you shouldn't hate and disable NetworkManager anymore

Note that this article has an update covering the changes moving from the initial release to the version of NetworkManager in EL7.3

Introduction

One of the quirks of distributions has always been the difference in how they handle network configuration.

Red Hat based systems have generally used /etc/sysconfig/network(-scripts) and debian based systems /etc/network/interfaces

These have generally called out to ifconfig (or more recently ip) to carry out various actions to configure the network stack appropriately. This methodology was (mostly) fine for static interfaces that never change but when WiFi began to grow more prevalent more flexible ways to handle these or switch between configurations became desirous.

A few tools appeared (such as wicd) but there has been some effort to consolidate on NetworkManager as a common point. This common point has native configuration in /etc/NetworkManager but most often the plugin infrastructure is used to read/write the network config in the standard location for the distro.

The first release seen in Enterprise Linux was EL6. This was fine for a laptop user in a graphical environment connecting to WiFi but pretty useless in almost every server scenario and the recommendation was to disable it and use the standard network service. In EL7 this has changed and the recommendation is to use NetworkManager where possible unless there is a specific edge case preventing doing so.

So what changed to make this preferable?

  • Changing an interface name with the network service required a server reboot, or at the least careful manipulation of manual ipconfig/ip commands. The scripts get very upset when interfaces change name and fail to find/create appropriately.
  • Changing configuration and restarting the network service affects all interfaces on a system rather than just an interface you care about or adding/removing a slave/ip.
  • With some network dependent based services they may not recover gracefully or even notice a network failure or restart.

These are all examples which the standard network scripts do not handle but NetworkManager provides an answer to.

NetworkManager information gathering

Focusing on the CLI the tool to manipulate NetworkManager is nmcli

As with many recent tools (see iproute2) the full operation name does not need to be entered - only enough to make a request unambiguous.

For instance nmcli general show can be shortened to nmcli g s and nmcli connection show to nmcli c s

A quick walk through of the various areas of the nmcli command is warranted before specific examples are delved into.

nmcli general

The general area is useful for a high level look at the current status and what permissions the user has to edit policy.

nmcli general show

STATE                   CONNECTIVITY  WIFI-HW  WIFI      WWAN-HW  WWAN    
connected (local only)  limited       enabled  disabled  enabled  enabled 

nmcli general permissions 

PERMISSION                                               VALUE 
org.freedesktop.NetworkManager.enable-disable-network    yes   
org.freedesktop.NetworkManager.enable-disable-wifi       yes   
org.freedesktop.NetworkManager.enable-disable-wwan       yes   
org.freedesktop.NetworkManager.enable-disable-wimax      yes   
org.freedesktop.NetworkManager.sleep-wake                no    
org.freedesktop.NetworkManager.network-control           yes   
org.freedesktop.NetworkManager.wifi.share.protected      yes   
org.freedesktop.NetworkManager.wifi.share.open           yes   
org.freedesktop.NetworkManager.settings.modify.system    yes   
org.freedesktop.NetworkManager.settings.modify.own       yes   
org.freedesktop.NetworkManager.settings.modify.hostname  auth  

nmcli networking

The networking area is used to modify high level networking behaviour. This could be disabling network connectivity entirely or checking current connectivity levels.

nmcli networking connectivity check

limited

nmcli radio

The radio area allows for the modification and status checks of radio based connectivity - this could be WiFi , WiMAX or mobile device. Note that as of recent NetworkManager releases the NM package has been split out so that things like wifi, wwan and bluetooth connectivity are only installed if required. If wifi connectivity through NM is desired for instance ensure the NetworkManager-wifi package is installed.

nmcli radio 

WIFI-HW  WIFI      WWAN-HW  WWAN    
enabled  disabled  enabled  enabled 
 

nmcli connection

The connection area is where profiles are set up to define the possible connections on the system. A laptop is likely to show many connections present due to hotspots being remembered. The device field is only populated if the connection is actually active.

nmcli connection

NAME                UUID                                  TYPE             DEVICE     
speedway            509e14aa-09d8-4824-ab9b-17f3b404b436  802-11-wireless  --         
p4p1                6f3baca8-f3a2-4e50-9358-2283c09396e1  802-3-ethernet   --         
333333              e7efc758-3b1f-4b94-99f1-35e4319069bb  802-11-wireless  --         
NewRouter           df362577-0fc4-4a72-8bb8-526f9dc98b16  802-11-wireless  --         
virbr1              a4050e95-edae-44f8-99ac-8555278276bb  bridge           --   

nmcli connection show p4p1
onnection.id:                          p4p1
connection.uuid:                        6f3baca8-f3a2-4e50-9358-2283c09396e1
connection.interface-name:              --
connection.type:                        802-3-ethernet
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.timestamp:                   1431589394
connection.read-only:                   no
connection.permissions:                 
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.secondaries:                 
connection.gateway-ping-timeout:        0
802-3-ethernet.port:                    --
802-3-ethernet.speed:                   0
802-3-ethernet.duplex:                  --
802-3-ethernet.auto-negotiate:          yes
802-3-ethernet.mac-address:             D4:BE:D9:7E:F3:CE
802-3-ethernet.cloned-mac-address:      --
802-3-ethernet.mac-address-blacklist:   
802-3-ethernet.mtu:                     auto
802-3-ethernet.s390-subchannels:        
802-3-ethernet.s390-nettype:            --
802-3-ethernet.s390-options:            
ipv4.method:                            auto
ipv4.dns:                               
ipv4.dns-search:                        
ipv4.addresses:                         192.168.0.1/24
ipv4.gateway:                           --
ipv4.routes:                            
ipv4.route-metric:                      -1
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.never-default:                     no
ipv4.may-fail:                          yes
ipv6.method:                            auto
ipv6.dns:                               
ipv6.dns-search:                        
ipv6.addresses:                         
ipv6.gateway:                           --
ipv6.routes:                            
ipv6.route-metric:                      -1
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --

nmcli device

The device area provides for display and manipulation of any actual devices themselves. A device doesn't have a configuration but when a connect request is made it will try to find a suitable connection that matches - which is where the configuration actually comes from.

nmcli device 
DEVICE             TYPE      STATE         CONNECTION 
virbr0             bridge    connected     virbr0     
virbr1             bridge    connected     virbr1     
virbr2             bridge    connected     virbr2     
virbr0-nic         tap       connected     virbr0-nic 
98:D6:F7:BA:15:3C  bt        disconnected  --         
AC:22:0B:9A:E0:FF  bt        disconnected  --         
p4p1               ethernet  unavailable   --         
wlp2s0             wifi      unavailable   --         
lo                 loopback  unmanaged     --         
virbr1-nic         tap       unmanaged     --         
virbr2-nic         tap       unmanaged     --        

nmcli device show p4p1
GENERAL.DEVICE:                         p4p1
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         D4:BE:D9:7E:F3:CE
GENERAL.MTU:                            1500
GENERAL.STATE:                          20 (unavailable)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
WIRED-PROPERTIES.CARRIER:               off

nmcli device show virbr0

GENERAL.DEVICE:                         virbr0
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         52:54:00:39:62:BE
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     virbr0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/1
IP4.ADDRESS[1]:                         192.168.122.1/24
IP4.GATEWAY:                            
IP6.GATEWAY:                            

With the high level information gathering out of the way it's example time.

It should be noted that changes made with nmcli are persistent - NetworkManager will write them to the files on disk. This could be in the /etc/NetworkManager/system-connections area in some cases or via the plugin mechanism the standard location for network configuration files (in Red Hat world /etc/sysconfig/network-scripts). The default behavior is not to change connection behaviours if the underlying files are changed automatically. If the files are altered directly then nmcli conn reload should be used to read in the new configuration and update the connection profile.

Configuring a static wired interface

nmcli conection modify <conn-name> connection.autoconnect yes ipv4.method manual ipv4.addr "10.0.0.1/24" ipv4.dns "10.0.1.1, 10.0.1.2" ipv4.gateway 10.0.0.254

This configures an interface statically. More than one address can be in a comma separated list to include aliases. To add a new address to an existing connection + can be used.

nmcli connection modify <conn-name> +ipv4.addr "10.0.0.2/24"

Configuring a wireless interface

nmcli device wifi connect "Free Hotspot" password c0ff33 name "Local Cafe"

This would create a new connection to the SSID called "Local Cafe" called "Free Hotspot". Once this is created it can be changed via nmcli connection modify "Free Hotspot" for any other details.

Overriding MAC address

For a wired interface:

nmcli connection modify <conn-name> 802-3-ethernet.cloned-mac-address 00:11:22:33:44:55

For a wireless interface:

nmcli connection modify <con-name> 802-11-wireless.cloned-mac-address 55:44:33:22:11:00

Configuring wireless hotspot

nmcli connection add type wifi ifname '*' con-name my-hotspot autoconnect no ssid my-local-hotspot
nmcli connection modify my-hotspot 802-11-wireless.mode ap 802-11-wireless.band bg ipv4.method shared

This uses a couple of interesting features. The '*' for interface name tells NetworkManager to use the first availiable interface that is not used yet and will meet the requirements (wifi in this instance). This is useful when you don't know the interface in advance such as a wifi USB adaptor. No IP settings are needed here as NetworkManager has a preconfigured behaviour for these quick hotspot setups. It will automatically configure dnsmasq to provide DHCP and DNS. NetworkManager handles the SSID behaviour itself rather than needing another tool such as hostapd.

Specify a particular firewalld zone for a connection

nmcli connection modify connection.zone 

This is particularly useful with the above hotspot behaviour as the zone can be preconfigured with the allowed traffic rules and then when the hotspot is brought up it will be bound by those straight away rather than facing a potential race condition or needing to know the interface name to write the rules directly in iptables.

Configuring bonded interface

Create a bond called mybond:

nmcli connection add type bond con-name mybond mode active-backup

Add some slaves to it:

nmcli connection add type bond-slave ifname eth2 master mybond
nmcli connection add type bond-slave ifname ens9 master mybond
nmcli connection add type bond-slave ifname ens10 master mybond

Bring up the slaves:

nmcli connection up bond-slave-eth2
nmcli connection up bond-slave-ens9
nmcli connection up bond-slave-ens10

Interesting note here. when carrying out the above the old connections were still listed initially, but only as "Wired Connection 1" rather than interface name and nmcli c sh on that connection showed no interface associated with it. After restarting the system the "Wired Connection X" vanished. The configuration files in /etc/sysconfig/network-scripts were removed on adding the interfaces to the bond.

The MAC address taken by the bond is the first link to be up. If using DHCP reservations this shoudl be kept in mind as the first link up is non-determinate. However you can set the MAC on each of the links to all be the same cloned-mac-address to ensure what the bond MAC address is safely.

When the slave interface is down ip li sh will show the real address of the interface and when the slave interface is brought up it will have the cloned MAC address.

In /proc/net/bonding these will be named nm-bond, nm-bond1 ... nm-bondX

Configuring teamed interfaces

Create a team called myteam:

nmcli connection add type team con-name myteam

Add the slaves:

nmcli connection add type team-slave ifname eth2 master myteam
nmcli connection add type team-slave ifname ens9 master myteam
nmcli connection add type team-slave ifname ens10 master myteam
nmcli connection add type team-slave ifname ens11 master myteam

As a note here unlike bonding where the original wired interface configurations were disassociated from the actual interface I encountered issues unless nmcli con del "Wired Connection X" was carried out to 'free' the interface for team use first.

The default runner (the type of 'bonding' used) is roundrobin as can be seen in teamdctl nm-team config dump.

Change this to active-backup:

nmcli connection modify myteam team.config '{runner: {"name": "activebackup"}}'

Changing the runner requires bringing down and then up the team.

nmcli connection down myteam
for i in eth2 ens9 ens10 ens11 ; do nmcli connection up $i ; done

Similar to the bonding driver to ensure the MAC address is static (rather than the first link that is activated) set 802-3-ethernet.cloned-mac-address on each link and then bring up the links to activate the team with that MAC address.

Note that since this does not use the bonding driver /proc/net/bonding does not exist and teamdctl is used to check the running config.

Configuring a tagged interface

Create the tagged interface:

nmcli connection add type vlan con-name myvlan dev ens9 id 60

The dev can be an interface name or a connection name. In this example it will tag all frames with vlan 60.

Configuring a bridge

Create the bridge:

nmcli connection add type bridge con-name mybridge

Add an interface to the bridge:

nmcli connection add type bridge-slave ifname ens9 master mybridge

By default STP is enabled so multiple active interfaces with the same bridge should result in one forwarding and the rest blocking. The package bridge-utils (brctl command) is deprecated and not installed by default. The replacement is part of the iproute2 package and is 'bridge' ...

bridge link
2: ens9 state UP :  mtu 1500 master nm-bridge state blocking priority 32 cost 100 
3: ens10 state UP :  mtu 1500 master nm-bridge state blocking priority 32 cost 100 
4: ens11 state UP :  mtu 1500 master nm-bridge state blocking priority 32 cost 100 
5: eth3 state UP :  mtu 1500 master nm-bridge state forwarding priority 32 cost 100 

When using libvirt (via virt-install or virt-manager) this bridge interface can be provided to the guest to pass traffic through.

If there is reason to disable STP (think carefully before do this or you risk network loops) then this config can be provided by:

nmcli connection modify mybridge bridge.stp no

Since this is a software bridge a loop is likely to stop you being able to even enter any commands, in which case rescue mode may be required to change the configuration via the files to STP=yes.

Tying the above together for VM guests

There is a limitation here at present still being worked on. It is not possible to use nmcli to link vlan, bond or teamd interfaces using it to a bridge using the bridge-slave interface type. This is being worked on according to bugzilla 1183420.

If adding anything other than something of type ethernet to the bridge then a slightly round the houses way is presently required.

NetworkManager is capable of reading the configuration files and setting up the more complicated setup so either edit the files directly and reboot (or nmcli con reload) or configure it by setting the properties of the interfaces correctly.

Create the bond(s) needed

nmcli connection add type bond con-name bond0 mode active-backup
nmcli connection add type bond-slave ifname eth2 master mybond
nmcli connection add type bond-slave ifname ens9 master mybond
nmcli connection add type bond-slave ifname ens10 master mybond

Create the bridge(s) needed

nmcli connection add type bridge con-name bridge0 ifname bridge0
nmcli connection add type bridge con-name bridge60 ifname bridge60
nmcli connection add type bridge con-name bridge100 ifname bridge100

Create the vlan(s) needed

nmcli connection add type vlan con-name vlan-60 dev nm-bond id 60
nmcli connection add type vlan con-name vlan-100 dev nm-bond id 100

Tie everything into the bridges

nmcli connection down mybond
nmcli connection down vlan-60
nmcli connection down vlan-100
nmcli connection modify bond0 connection.master bridge0 connection.slave-type bridge
nmcli connection modify vlan-60 connection.master bridge60 connection.slave-type bridge
nmcli connection modify vlan-100 connection.master bridge100 connection.slave-type bridge
nmcli connection up bond-slave-eth2
nmcli connection up bond-slave-ens9
nmcli connection up bond-slave-ens10
nmcli connection up bond0
nmcli connection up bridge0
nmcli connection up vlan-60
nmcli connection up bridge60
nmcli connection up vlan-100
nmcli connection up bridge100

Once that is complete (or just reboot after the connection modify commands) then when creating virtual guests they can just be connected to the appropriate bridge for the correct vlans.

Remember that if any IP configuration is required is most be set up on the bridge interface and not any of the underlying vlans or bond slaves.

Attempts to use the teaming driver instead failed with links not activating properly - not sure of the specifics why but this will be investigated further in future.

Add new comment