NetBackup Upgrade Procedure

DISCLAIMER: This post is just to give quicker picture of overall key things to consider for upgrading NetBackup servers. Before doing actual upgrade please refer official NetBackup guides. The below information is tried & tested for all versions between 8.0 till 10.0

[1] Prerequisites / Planning:
=======================

1) For Windows, Make sure that all the Microsoft Windows OS and security updates are installed.

2) create a non-root SERVICE USER before upgarde.

3) MSDP rolling conversion

- 7.7.x/8.0 TO 8.1 UPGRADE includes this.

- It works in background to convert all existing data containers to AES encryption & SHA-2 Fingerprint algorithm.

- No manual intervention is required.

- After upgrade, it begins to work in background.

4) If the NetBackup PBX is running in secure mode, please add the web service user as authorized user in PBX.

https://www.veritas.com/content/support/en_US/article.100032858

5) Review & confirm all OS requirements for Linux & Windows.

Unix/Linux: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v112189449-147534104

Windows: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v112189469-147534104

6) Confirm Web server user & group are created & enabled.

7) (Optional) With a NetBackup 9.1 upgrade, new root CA with 2048 bit key strength is deployed. To use a certificate key size larger than 2048 bits, set the NB_KEYSIZE environment variable on the primary server before you start the installation.

For example:   NB_KEYSIZE = 4096

The NB_KEYSIZE can only have the following values: 2048, 4096, 8192, and 16384.

# NOTE: Known SuSE Linux Primary server upgrade issue:

- Might give error related to webservice user/group.

- Solution: Set ENVIRONMENT variable "NBPREINSTALL_CRITICAL_OVERRIDE=YES"


[2] Upgrade Tools:
===============

1) SORT - Helps with Install/Upgrade checklist, Hot fix/EEB auditor, feature plans.

2) NB Preinstall checker


[3] Begin Actual Upgrade:
=====================

Sequence:
        =========

Primary => Media => Clients

- Upgrade NB Master/Primary server before you upgrade NB on any other hosts.

- Once primary server upgrade is finished then you can upgarde Media servers

- and then clients.


    Master server Upgrade:
    ====================

  - PRE-UPGRADE:
         ================

- Remember to upgrade OpsCenter first before primary server upgrade.

- perform environment check with SORT tool.

- download customer registeration key.

- (Conditional) If NB db files reside on btrfs FS then move it to supported FS such as xfs,ext4.

- Perform pre-upgrade tasks:

- Run a hot catalog backup

- Disable all SLPs

        - Deactivate all NB Policies

- Disable OpsCenter

- Stop any apps on system that interact with NB. E.g DB or system components being backed up. For Oracle stop RMAN processes.

- Stop all NB services.


- UPGRADE:
        =========

- Windows: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v85746556-147534104 (It's easy/similar to new install, Nothing fancy, just follow UI steps).

- Linux: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v85637145-147534104

- POST-UPGRADE: (This steps are similar to All NB server Upgrades for all OSes (Linux/windows).
        ===============

- (post-install step) set passphrase for DR. otherwise catalog backups will fail.

- (Optional) designate a security administartor role if you plan RBAC.

- (Conditional) If primary was configured with ECA, configure ECA now. (set bp.conf entries & nbcertcmd -enrollCertificate)

- If you have any media servers that you intend to upgrade, you may upgrade them now.

- Reactivate the following in the order as shown:

- All disk staging storage units.

- All NetBackup policies.

- All storage lifecycle policies (SLPs).

- OpsCenter data collection for this primary server.

- (Conditional) If your environment uses cloud storage, you need to update the read and write buffer sizes. More information is available. See Post upgrade procedures for Amazon cloud storage servers.

- (Conditional) For the cloud and the storage servers with SSL enabled, the CRL validation is enabled by default. Verify if the storage servers are running and the CRL functionality works correctly.

- Monitor your backup environment to verify that normal NetBackup operation has resumed.

- Upgrade any media servers and clients not already upgraded as time and backup windows permit. Be sure to upgrade the media servers before you upgrade the clients.

- Update Storage Servers. (updatests & updatedp).

- If NBAC or Enhanced auditing was enabled then run the bpnbat -login


Media server upgrade:
===================

3 Upgrade Methods:
==============

1) NB Upgrade script

2) Unix & Linux installers

3) VxUpdate

- NB upgrade script is standard & recommended method.

- The native Unix & Linux installers are potentially more difficult & require additional steps.

- VxUpdate provides remote installation capabilities & ability to upgrade on a user-defined schedule.


Actual upgrade/Migration procedure:
==========================

1) Deactivate the media server.

2) stop all NB services on media you're upgrading.

3) Upgrading NB binaries.

- Windows: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v85746556-147534104 (It's easy/similar to new install, Nothing fancy, just follow UI steps).

- Unix/Linux: https://www.veritas.com/content/support/en_US/doc/125240132-147534104-0/v85637145-147534104

4) (Conditional) If you did not get a security certificate, generate the certificate.

5) Reactivate the media server.


MSDP upgrade: (consider this only if upgrading from NB 8.0 or older versions)
===================================================

1) MSDP upgrade considerations for NetBackup 8.1

- Because of the changes in the fingerprint algorithm for MSDP in NetBackup 8.1, consider your MSDP environment as you plan your upgrade path. Any NetBackup 8.0 and older host cannot access the NetBackup 8.1 MSDP because of the new fingerprint algorithm.

- You may observe job failure due to above.

- If there are multiple media servers under MSDP:

- Upgrade all media servers.

- Upgrade media & clients using client direct as your env allows.


2) About MSDP rolling data conversion (No manual intervention required, things will happen in background post upgrade).

- NetBackup 8.0 introduced the AES encryption algorithm to replace the existing Blowfish algorithm.

- NetBackup 8.1 introduces the SHA-2 fingerprint algorithm to replace the existing MD5-like algorithm.

- The upgrades to both the encryption and the fingerprint algorithms are designed to enhance data security.

- The environments that are upgraded to NetBackup 8.1 may include Blowfish encrypted data and the MD5-like fingerprints that need to be converted to the new format.

- To handle the conversion and secure the data, a new internal task converts the current data container to the AES encryption and the SHA-2 fingerprint algorithm.

- This new task is referred to as the rolling data conversion.

- You can manage & monitor this using "crcontrol" command.


3) About MSDP fingerprinting algorithm changes (No manual intervention required, things will happen in background post upgrade).

- With NetBackup 8.1, Media Server Deduplication Pool (MSDP) introduces a more secure fingerprint algorithm.

- The SHA-2 algorithm replaces the existing MD5-like algorithm.

- NetBackup 8 .1 can handle both fingerprint types, and the new server is compatible with old clients and old servers.

- Conversion happens during the interaction between old clients and old servers and the new server.

- The fingerprint conversion requires additional computation time.

- The interaction between old clients and old servers and new server is slower than if both the client and the server are new.


CP (Cloud Point) upgrade:
======================

Supported Upgrade path: (Direct upgrades are supported).
========================================

8.3 =>   9.0, 9.1, 9.1.0.1, 9.1.2

9.0 =>    9.1, 9.1.0.1, 9.1.2

9.1 =>    9.1.2


Pre-Upgrade procedure:
================

1) Ensure CP server meets the requirements of CP version you're upgrading to.

2) You can take manual backup of /cloudpoint volume. However, In upgrade procedure also you get chance to backup, you will be prompted.

3) Ensure that no jobs are running on CP you're upgrading to. If running, then you can wait/cancel them using SLP.


Upgrade procedure:
=================

1) Disable CP server.

2) Upgrade CP server.

3) Then enable CP server.


Upgrading CP in Docker environment:
================================

1) Download the CP version to which you want to upgrade on CP host.

2) Load the CP image using: "docker load -i <image_name>"

3) Make note of current CP build version, you will require in next step.

4) Stop CP using command:

"docker run -it --rm -v /cloudpoint:/cloudpoint -v /var/run/docker.sock:/var/run/docker.sock veritas/flexsnap-cloudpoint:<current_version> stop"

5) Upgrade CP using below command:

sudo docker run -it --rm -v /cloudpoint:/cloudpoint -v /var/run/docker.sock:/var/run/docker.sock veritas/flexsnap-cloudpoint:<new_version> install

6) The new CloudPoint installer detects the existing CloudPoint containers that are running and asks for a confirmation for removing them.

Press Y to confirm the removal of the old CloudPoint containers.

7) (Optional) Run the following command to remove the previous version images.

# docker rmi -f <imagename>:<oldimage_tagid>

8) verify new CP is installed successfully. That's it.


Post-Upgrade:
============

1) upgrade CP Agents (on Linux & windows VM) if any.

2) Enable CP.

3) After upgrade if you want to use BFS, you must re-edit CP by providing token so that NB certs are generated on CP.


Upgrading UNIX and Linux NB server software:
======================================

   1) Log on as the root user on the server.

   2) If the NetBackup Administration Console is open, you must close it now.

   3) Save any files that you have modified and want to keep.

   4) ./install

   5) Follow the prompts in the installation script to install the NetBackup server binaries.

   6) (Conditional: For primary servers only) If prompted, reply to the question regarding the infinite expiration conversion.

   7) (Conditional: For primary server only) If prompted, provide the name of the service user account you want to use to start most of the daemons.

   8) (Conditional: For media servers only) If your environment uses an external certificate authority, provide the ECA information at the prompts.

   9) (Conditional: For media servers only) When prompted, provide the required information for the CRL configuration.

   10) (Conditional: For media servers only) If you specified Use the CRL from a file path, you must enter the path to the CRL location.

   11) When the script finishes, the binaries are successfully installed.

   12) Perform the Post-Upgrade procedures.

Comments

Post a Comment

Popular posts from this blog

KMP Algorithm: Pattern Searching in Text

Agenda: Quick Intro Input Output Complexity Algo Logic Code Real-Life Use Cases Conclusion Sources for further reading 1. Quick Intro The Knuth-Morris-Pratt (KMP) algorithm is a classic string-searching algorithm used to find occurrences of a "pattern" string within a "text" string.  It was devised by Donald Knuth , James H. Morris , and Vaughan Pratt , hence the name KMP .  The key feature of the KMP algorithm is its ability to perform the search efficiently by avoiding unnecessary comparisons . 2. Input The KMP algorithm takes two inputs: Text (T) : The string in which we are searching for a pattern. Pattern (P) : The string we are searching for within the text. 3. Output Indices (I): Starting indices of all occurrences of the pattern in the text. If the pattern is not found, it returns an empty list. 4. Complexity Time Complexity :  O(n + m)   n ---> Length of the text. m ---> Length of the pattern. Sp...
Read more

Z-Function Algorithm: Substring Search

Agenda: Quick Intro Input Output Complexity Algo Logic C++ Code Real-Life use cases Practice Problem Conclusion Sources for further reading Quick Intro The Z-function is a powerful string-matching algorithm used for pattern matching and searching within a given string. Its primary purpose is to find all occurrences of a specified pattern in the text. Definition: Given a string s of length n, the Z-function produces an array Z where Z[i] represents the length of the longest substring starting from s[i] that is also a prefix of s. Input Text(T): The input to the Z-function algorithm is a single string T of length n . Output Z-Array(z): The output of the Z-function algorithm is an array Z of length n where each element Z[i] represents the length of the longest substring starting from the position i in T that matches a prefix of T . Complexity Time: O(n) Space: O(n) Where, n ---> length of input Text....
Read more

Back of the envelope estimations

# Agenda: Overview Common inputs in a requests-per-second (RPS) calculation Example of RPS / QPS calculation Techniques to simplify the calculations Handy conversion Table Capacity or Storage Planning Conclusion # Overview: Back-of-the-envelope math is a very  useful tool in our system design toolbox. In this article, we will go over how and when to use  it, and share some tips on using it effectively. Let’s dive right in. Experienced developers use back-of-the-envelope  math to quickly sanity-check a design. In these cases, absolute accuracy is not that important . Usually, it is good enough to get  within an order of magnitude or two of the actual numbers we are looking for. For example, if the math says at our scale our web  service needs to handle 1M requests per second, and each web server could only handle about 10K  requests per second, we learn two things quickly:     1) W e learn that we will need to cluster of web  servers, with a l...
Read more