Teleport
Upgrade Teleport Cloud Agents on Linux
Version preview- Older Versions
This guide explains how to upgrade Teleport Cloud agents running on Linux servers. On Teleport Cloud, Auth Service and Proxy Service upgrades are managed for you. To keep agents up to date, you can either enroll them in automatic upgrades or upgrade them manually using the method you used to install Teleport.
Prerequisites
-
Familiarity with the Upgrading Compatibility Overview guide, which describes the sequence in which to upgrade components in your cluster.
-
A Teleport Enterprise Cloud account. You can determine the current version of the Auth Service and Proxy Service by running the following command, where
mytenant
is the name of your Teleport Enterprise Cloud account. This requires thejq
CLI tool:curl -s https://mytenant.teleport.sh/webapi/ping | jq '.server_version' -
The
tctl
andtsh
client tools version >= 15.4.4.You can download these from Teleport Cloud Downloads.
-
To check that you can connect to your Teleport cluster, sign in with
tsh login
, then verify that you can runtctl
commands using your current credentials.tctl
is supported on macOS and Linux machines.For example:
tsh login --proxy=teleport.example.com --user=email@example.comtctl statusCluster teleport.example.com
Version 17.0.0-dev
CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
If you can connect to the cluster and run the
tctl status
command, you can use your current credentials to run subsequenttctl
commands from your workstation. If you host your own Teleport cluster, you can also runtctl
commands on the computer that hosts the Teleport Auth Service for full permissions.
Teleport supports automatic agent updates for systemd-based Linux distributions
using apt
, yum
, and zypper
package managers. The Automatic Update
Architecture guide describes
how agent updating works. Automatic agent upgrades require:
-
A Teleport Cloud account that supports automatic updates. To determine if your account supports automatic updates, run the following command, replacing
mytenant.teleport.sh
with the address of your Teleport Cloud account:curl -s https://mytenant.teleport.sh/webapi/ping | jq '.automatic_upgrades'true -
At least one Teleport Enterprise agent, started via systemd on a distribution using the
apt
oryum
package managers.So you can access each agent, we assume that each agent runs the Teleport SSH Service in addition to any other services. See the Deploy Teleport Agents with Terraform guide for an example.
Step 1/2. Find agents to upgrade
To list agents that are not enrolled in automatic updates, use the tctl inventory ls
command with the --upgrader=none
flag. This command also uses
the --services
flag to constrain the search to Teleport processes running
agent services:
tctl inventory ls \ --services=node,kube,db,app,windows_desktop,discovery,okta \ --upgrader=noneServer ID Hostname Services Version Upgrader------------------------------------ ------------- -------- ------- --------00000000-0000-0000-0000-000000000000 ip-10-1-6-130 Node v13.4.5 none
If you have a lot of agents on different versions and want to prioritize enrolling
your oldest agents, you can limit your search using the --older-than
filter:
tctl inventory ls --upgrader=none --older-than=v1.2.3Server ID Hostname Services Version Upgrader------------------------------------ --------------- -------------- ------- --------00000000-0000-0000-0000-000000000000 old.example.com Node v1.1.1 none00000000-0000-0000-0000-000000000001 teleport-proxy Proxy v14.3.6 none00000000-0000-0000-0000-000000000002 teleport-auth Auth,Discovery v14.3.6 none...
Note that the inventory ls
command will also list teleport-auth
and teleport-proxy
services.
You do not need to update these services or enroll them in automatic updates.
Teleport Cloud manages updates for these services.
When you enroll each agent in automatic updates in the next section, you can run the following commands to fetch the hostname for each agent and access it via Teleport.
For each agent ID returned by the tctl inventory ls
command, copy the ID and
run the following tctl
command to access the host via tsh
:
HOST=00000000-0000-0000-0000-000000000000USER=roottsh ssh "${USER?}@${HOST?}"
You can then follow the steps in the next section on each agent to enroll it in automatic updates.
Step 2/2. Enroll agents in automatic updates
Complete the following instructions on each agent you would like to enroll into automatic updates:
-
Ensure the Teleport repository is added and Teleport Enterprise is installed.
To verify if the Teleport repository was added to the system, check if either of the follow files exists:
ls /etc/apt/sources.list.d/teleport.listor
ls /etc/yum.repos.d/teleport.repoThe upgrader checks the repository for available releases, so make sure that it is up to date.
-
If the repository was added, make sure the Teleport binary installed on the agent can run the automatic updater:
which teleport-upgrade || echo "Install the upgrader"Install the upgrader -
If the Teleport repository is not found, or the Teleport binary you installed does not include the upgrader, add the appropriate repository and reinstall Teleport:
Add the Teleport repository to your repository list:
Download Teleport's PGP public key
sudo curl https://apt.releases.teleport.dev/gpg \-o /usr/share/keyrings/teleport-archive-keyring.ascSource variables about OS version
source /etc/os-releaseAdd the Teleport APT repository for cloud.
echo "deb [signed-by=/usr/share/keyrings/teleport-archive-keyring.asc] \https://apt.releases.teleport.dev/${ID?} ${VERSION_CODENAME?} stable/cloud" \| sudo tee /etc/apt/sources.list.d/teleport.list > /dev/nullProvide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.comexport TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"Update the repo and install Teleport and the Teleport updater
sudo apt-get updatesudo apt-get install "teleport-ent=$TELEPORT_VERSION" teleport-ent-updaterSource variables about OS version
source /etc/os-releaseAdd the Teleport YUM repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")sudo yum install -y yum-utilssudo yum-config-manager --add-repo "$(rpm --eval "https://yum.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-yum.repo")"Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.comexport TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"Install Teleport and the Teleport updater
sudo yum install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updaterTip: Add /usr/local/bin to path used by sudo (so 'sudo tctl users add' will work as per the docs)
echo "Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin" > /etc/sudoers.d/secure_path
Source variables about OS version
source /etc/os-releaseAdd the Teleport YUM repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")Use the dnf config manager plugin to add the teleport RPM repo
sudo dnf config-manager --add-repo "$(rpm --eval "https://yum.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-yum.repo")"Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.comexport TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"Install Teleport and the Teleport updater
sudo dnf install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updaterTip: Add /usr/local/bin to path used by sudo (so 'sudo tctl users add' will work as per the docs)
echo "Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin" > /etc/sudoers.d/secure_path
Source variables about OS version
source /etc/os-releaseAdd the Teleport Zypper repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")Use Zypper to add the teleport RPM repo
sudo zypper addrepo --refresh --repo $(rpm --eval "https://zypper.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-zypper.repo")Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.comexport TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"Install Teleport and the Teleport updater
sudo zypper install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updater -
If you changed the agent user to run as non-root, create
/etc/teleport-upgrade.d/schedule
and grant ownership to your Teleport user. Otherwise, you can skip this step:sudo mkdir -p /etc/teleport-upgrade.d/sudo touch /etc/teleport-upgrade.d/schedulesudo chown <your-teleport-user> /etc/teleport-upgrade.d/schedule -
Verify that the upgrader can see your version endpoint by checking for updates:
sudo teleport-upgrade dry-runYou should see one of the following messages, depending on the target version you are currently serving:
no upgrades available (1.2.3 == 1.2.3) an upgrade is available (1.2.3 -> 2.3.4)
The upgrader may display warnings about not having a valid upgrade schedule. This is expected immediately after install as the maintenance schedule might not be exported yet.
-
Ensure that the upgrader is running:
sudo systemctl status teleport-upgrade
Version locking
As of Teleport 15.1.10
, a version locking mechanism is enabled by the updater.
This mechanism locks the version of Teleport and prevents manual updates of the Teleport
package. This prevents unintentional updates during routine system maintenance, or
an accidental update by a user. The updater still has the capability to update the
Teleport package, and all updates of Teleport are expected to be performed by the
updater.
The version locking mechanism is implemented using the features of the package managers. In case a user would like to manually update the Teleport version, this can be done with the following commands.
With the APT package manager CLI, the --allow-change-held-packages
flag must be provided
to bypass the version lock.
apt-get install --allow-change-held-packages "teleport-ent=<target-version>"
With the YUM package manager CLI, the --disableexcludes="teleport"
flag must be provided
to bypass the version lock.
yum install --disablerepo="*" --enablerepo="teleport" --disableexcludes="teleport" "teleport-ent-<target-version>"
With the Zypper package manager CLI, the lock must be disabled and then re-enabled after the update.
zypper removelock "teleport-ent"zypper install --repo="teleport" "teleport-ent-<target-version>"zypper addlock "teleport-ent"
Updating the updater
The updater is designed to be minimal and relatively stable, but the updater will
receive updates on occasion. Currently, the updater does not have the capability
to update itself. Updates to the teleport-ent-updater
package must be done manually.
The teleport-ent-updater
will be backwards compatible with older versions of Teleport,
so you can always update the teleport-ent-updater
package to the latest available
version.
Troubleshooting
The teleport-upgrade
tool provides some basic commands to verify and perform an
update of the Teleport agent.
teleport-upgrade helpUSAGE: /usr/sbin/teleport-upgrade <command>
Tool for automatic upgrades of Teleport agents.
Commands: run check for and potentially apply a teleport upgrade. dry-run check for new teleport version but do not upgrade. version print the current version of /usr/sbin/teleport-upgrade. help show this help text.
The dry-run
command can be used to check for an available update without performing
an update.
Example output when teleport is already on the latest compatible version.
teleport-upgrade dry-run[i] no upgrades available (14.3.14 == 14.3.14) [ 582 ]Example output when an update is available.
teleport-upgrade dry-run[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] within maintenance window, upgrade will be attempted. [ 596 ]
The run
command performs an update if available.
Successful teleport update from 13.4.14 to 14.3.14.
teleport-upgrade run[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] within maintenance window, upgrade will be attempted. [ 596 ][i] attempting apt install teleport-ent=14.3.14... [ 480 ][...][i] gracefully restarting Teleport (if already running) [ 449 ]Teleport updates are not attempted when outside the maintenance window.
teleport-upgrade run[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]
Teleport agents are not updated immediately when a new version of Teleport is released, and agent updates can lag behind the cluster by a few days.
If the Teleport agent has not been automatically updating for several weeks, you
can view the logs of the teleport-upgrade.service
to help troubleshoot the problem.
journalctl -t teleport-upgradeApr 16 16:44:32 example teleport-upgrade[168983]: [i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]Apr 16 16:50:32 example teleport-upgrade[169084]: [i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ]Apr 16 16:50:33 example teleport-upgrade[169084]: [i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]Apr 16 16:56:32 example teleport-upgrade[169187]: [i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ]...
To suspend automatic updates, disable the teleport.upgrade.timer
systemd timer:
sudo systemctl disable --now teleport-upgrade.timer
To enable and start the systemd timer after suspending:
sudo systemctl enable --now teleport-upgrade.timer