.. _upgrade: =============================== Upgrade from a previous version =============================== Overview ======== This page explains how to upgrade |sse| to the latest stable version from a previous version. These upgrade instructions cover an upgrade from |sse| version |last_major_release| to version |stable|. .. include:: _includes/terminology-change.rst To upgrade |sse|, you will need to: * Back up your data, including certain files and directories that are crucial to your specific installation of |sse| * Upgrade PostgreSQL (optional, but recommended) * Upgrade your Salt infrastructure (optional, but recommended) * Download the upgrade files * Upgrade the |raas| node * Upgrade any |masters-salt| using the |plugin| This page explains how to complete each of these steps in more detail. .. Warning:: If you are upgrading your |sse| installation, refer to the following upgrade instructions. Do not attempt to install using the automated installer (single-node or multi-node) or manual installation instructions. .. include:: _includes/salt-best-practices.rst Upgrading versions earlier than |last_major_release| ---------------------------------------------------- As a best practice, always upgrade from the latest major version of |sse| to the new release. If you are upgrading from a version earlier than |last_major_release|, you might see the best results if you upgrade in increments from one major release to the next. For instructions on upgrading to earlier |sse| versions, see the specific upgrade instructions for the releases you are upgrading from. The upgrade instructions for previous releases are included in the installation guide PDF for the subsequent release. For example, if you need to upgrade from 5.5 to 6.0, see the upgrade instructions in the 6.0 installation PDF. .. list-table:: :widths: 25 75 :header-rows: 1 * - Version - Installation and Upgrade Guide PDF * - 8.3 - :download:`8.3 Installation and Upgrade Guide ` * - 6.4 - :download:`6.4 Installation and Upgrade Guide ` * - 6.3 - :download:`6.3 Installation and Upgrade Guide ` * - 6.2 - :download:`6.2 Installation and Upgrade Guide ` * - 6.1 - :download:`6.1 Installation and Upgrade Guide ` * - 6.0.1 - :download:`6.0.1 Installation and Upgrade Guide ` * - 6.0 - :download:`6.0.0 Installation and Upgrade Guide` * - 5.5.1 - :download:`5.5.1 Installation and Upgrade Guide ` Best practices when upgrading ----------------------------- When preparing to perform an upgrade of |sse|, the following recommendations could be helpful: * **Back up your data.** In order to prevent data loss, back up your data. For an explanation of which files and directories must be backed up before upgrading, see `Back up your data`_. * **Perform upgrades during hours of slow network activity.** Database upgrades require re-indexing data. Depending on the complexity of your data, a database upgrade could potentially take several hours. To prevent service disruptions, consider upgrading the database during slower business hours or trimming your database prior to an upgrade. * **Check the database for any old commands being stored.** In some cases, the PostgreSQL database stores old commands that haven't run. These commands might run during the upgrade process, when you restart the |plugin|. To prevent this from happening, check whether any old commands are stored in the database, and enable skipping jobs that are older than a defined time. * **Test the upgrade before deploying.** If possible, you could try running a dress rehearsal in a test environment to get a sense of how long the upgrade could take. * **Read through the whole guide first.** Consider also reading through this entire guide one time before you implement an upgrade so that you have a good sense of the tasks that are required and whether they require planning from your team or if stakeholders must be notified of pending changes. Back up your data ================= Before upgrading |sse|, you should always back up your data. The following sections explain which data needs to be backed up so that it can be restored after you complete the upgrade process. Back up |sse| files and directories ----------------------------------- The following files and directories contain your custom |sse| configurations and need to be backed up before upgrading: #. On the |raas| node, back up these entire directories: * ``/etc/raas/raas`` * ``/etc/raas/raas.secconf`` * ``/var/log/raas`` * ``/etc/raas/pki/`` .. Note:: The ``pki`` directory contains hidden files, so ensure you back up the entire directory. Backing up the log files in the ``/var/log/raas`` directory is optional. During the upgrade process, you'll clear the log files to provide a clean log file if troubleshooting is necessary. #. On each |master|, back up the ``/etc/salt/master.d/raas.conf`` and ``/etc/salt/master.d/eAPIMasterPaths.conf`` files. .. Note:: Depending on how you initially installed |sse|, the eAPI |master| paths could instead be in the ``/etc/salt/master.d/raas.conf`` file instead. After backing up the |sse| and |master| files, proceed to the next section. Back up your database schema ---------------------------- When upgrading your |raas| node, the database schema is updated. For that reason, ensure you create a backup of your database before the upgrade. To back up your database you need to first look up your PostgreSQL database name and then copy the contents: #. On the PostgreSQL server, back up these files: * ``postgres.conf`` * ``pg_hba.conf`` #. Log in as the ``postgres`` user using the following command: .. code-block:: bash sudo su - postgres #. Get your database name, using the following commands to enter PostgreSQL and then list the databases: .. code-block:: bash psql \l #. To exit PostgreSQL and log out as the ``postgres`` user, press Ctrl+D and then run the following command: .. code-block:: bash exit #. Copy database contents to a file. The following command gives an example: .. code-block:: bash pg_dump -U salt_eapi raas_db_name > postgres_raas_backup_$(date +%Y-%m-%d).sql Your database files are now backed up. For the latest information about performing backups, see `PostgreSQL database backups `_. Upgrade PostgreSQL database =========================== .. include:: _includes/latest-postgres-version.rst Upgrading to the latest version of PostgreSQL is not required. However, upgrading PostgreSQL can possibly improve performance. For instructions on upgrading to the latest version of PostgreSQL, see `PostgreSQL upgrade `_. Upgrade Redis database ====================== .. include:: _includes/latest-redis-version.rst Upgrading to the latest version of Redis 5.x is not required. However, upgrading Redis can possibly improve performance. For instructions on upgrading Redis, see `Redis Administration `_. Upgrade Salt ============ For best performance, ensure your Salt components are running on the latest major version of Salt. For instructions on upgrading your |masters| and other Salt components, see `Upgrading Your Salt Infrastructure `_. For instructions on upgrading the Salt Crystal package, see `How to Upgrade Salt Crystal `_. Download upgrade files ====================== To download the files for upgrading: #. On the :ref:`downloads` page, go to the :ref:`sse-upgrade-downloads` section. #. Download all files listed needed for upgrading. #. Verify the files. For specific instructions, see :ref:`verify-files-downloads`. After you've downloaded the files, proceed to the next section. .. _upgrade-raas: Upgrade the |raas| node ======================= This section explains how to upgrade the |raas| node from |last_major_release| to version |stable|. Be aware that database upgrades require re-indexing data. If your data is complex, a database upgrade could potentially take several hours. For a discussion of when to plan an upgrade and other tips, see `Best practices when upgrading`_. .. Warning:: Before upgrading your |raas| node, you **must back up your system data** to avoid data loss. For an explanation of which files and directories must be backed up before upgrading, see `Back up your data`_. To upgrade the |raas| node: #. Save any changes you made to the default file system, pillar data, and jobs as new files or jobs. #. Note any pillar assignments that are made to the default targets. These need to be re-assigned after upgrade. #. Stop the |raas-service| using the following command: .. code-block:: bash sudo systemctl stop raas #. Remove the log file(s) in the ``/var/log/raas`` directory. Clearing the log files provides a clean log file if troubleshooting is necessary. #. Remove the currently installed version of the |api| with the following command: .. code-block:: bash sudo yum remove raas #. Upgrade the |raas| node by installing the latest RPM. Use the following example command, replacing the exact file name of the RPM: .. code-block:: bash sudo yum install raas-rpm-file-name.rpm #. **IMPORTANT:** Restore the backup of the following files: * ``/etc/raas/raas`` * ``/etc/raas/raas.secconf`` * ``/etc/raas/pki/`` #. Update permissions for the ``raas`` user with the following command: .. code-block:: bash sudo chown -R raas:raas /etc/pki/raas/certs #. OPTIONAL: If you have a |secops| license and want to add the |comply| library, add the following new section to the ``/etc/raas/raas`` file: .. code-block:: yaml sec: ingest_override: true locke_dir: locke post_ingest_cleanup: true username: 'secops' content_url: 'https://enterprise.saltstack.com/secops_downloads' download_enabled: true download_frequency: 86400 stats_snapshot_interval: 3600 compile_stats_interval: 10 ingest_on_boot: True content_lock_timeout: 60 content_lock_block_timeout: 120 .. Note:: This step is optional and only applies to organizations that have a valid |secops| license. This add-on module is available for |sse| versions 6.0 and later. The previous configuration options in the ``/etc/raas/raas`` configuration file are specific to these add-on modules. #. OPTIONAL: If you have a |secops| license and want to add the |protect| library, add a new section to the ``/etc/raas/raas`` file: .. include:: _includes/ssprotect-config-options.rst .. Note:: This step is optional and only applies to organizations that have a valid |secops| license. This add-on module is available for |sse| versions 6.0 and later. The previous configuration options in the ``/etc/raas/raas`` configuration file are specific to these add-on modules. #. The |raas| currently has a known issue related to stale jobs. (See :ref:`known-issues` for more information.) When upgrading, some users might notice a queue of stale jobs that are stuck in a pending state. Upgrading the |raas| node can cause these jobs to run unless they are first cleared out. To prevent this from happening, first check whether any old commands are stored in the database. On your PostgreSQL node, check for any pending jobs using the following command: .. code-block:: postgresql select count(1) from commands where state = 'new'; The result is the number of pending jobs. If the number of jobs is ``0``, proceed with the rest of the upgrade process. If the number of jobs is greater than ``0``, :ref:`contact-support` for a workaround. #. Upgrade the |raas-service| database using the following command: .. code-block:: bash sudo su - raas raas upgrade .. Note:: Depending on the size of your database, the upgrade can take anywhere from several minutes to over an hour. If you encounter errors, check the ``/var/log/raas/raas`` logfile for more information. #. After the upgrade, exit the session for the ``raas`` user with the following command: .. code-block:: bash exit #. Start the |raas-service| using the following command: .. code-block:: bash sudo systemctl enable raas sudo systemctl start raas Verify that |sse| is functioning correctly and is running the latest version. Proceed to the next section. Upgrade |masters| with the |plugin| ====================================== After you have successfully upgraded the |raas| node, you can then upgrade any |masters| that use the |plugin| to connect to |sse|. .. Note:: Before you upgrade the |masters|, ensure that the pip3 application is installed on the |masters|. If you are upgrading from the latest version of the |plugin|, this application is already installed. To upgrade the |plugin| on a |master|: #. Stop the ``salt-master`` service using the following command: .. code-block:: bash sudo systemctl stop salt-master #. Check which version of Python is running on the |master|. If it is running Python 3.6 or higher, no changes are needed. Otherwise, delete the prior version of the SSEAPE module. (The SSEAPE is the |sse| plugin for the |master|). For example: .. tabs:: .. tab:: RHEL/CentOS .. code-block:: bash sudo rm -rf /usr/lib/python3.6/site-packages/SSEAPE* .. tab:: Ubuntu .. code-block:: bash sudo rm /usr/lib/python3.6/dist-packages/SSEAPE* #. Upgrade the |plugin| by manually installing the updated Python wheel. Use the following example commands, replacing the exact name of the wheel file: .. include:: _includes/plugin-install-script.rst #. Update the |api| module paths by editing the ``/etc/salt/master.d/eAPIMasterPaths.conf`` file to reference the paths to the various modules. For example, you might change all ``python2.7`` references in this file to ``python3.6``. .. Note:: Depending on how you initially installed |sse|, the eAPI |master| paths could instead be in the ``/etc/salt/master.d/raas.conf`` file instead. #. Check the ``engines`` section in ``/etc/salt/master.d/raas.conf`` to confirm that it matches the following: .. code-block:: yaml engines: - sseapi: {} - eventqueue: {} - rpcqueue: {} - jobcompletion: {} .. Note:: If a problem occurred, you may need to restore your backups of the ``/etc/salt/master.d/raas.conf`` and ``/etc/salt/master.d/eAPIMasterPaths.conf`` files. #. Start the ``salt-master`` service with the following command: .. code-block:: bash sudo systemctl start salt-master The upgrade process is now complete. If you encounter any other errors, refer to the :ref:`troubleshooting` page or :ref:`contact-support`.