Prerequisites
Before performing the uninstall, ensure you have:
kubectl
installed, and configured to connect to your cluster.jq
installed, and available in your Linux PATH.
Uninstall Kaptain
Uninstalling Kaptain requires the execution of several manual steps.
-
Delete all profiles configured with Kaptain. Profiles have dependencies (such as finalizers) associated with Kaptain, so they must be removed before uninstalling Kaptain. Attempting to uninstall Kaptain without removing the profiles will fail and leave many resources in a broken state.
The following command deletes all notebooks, pipelines and deployed models permanently. Notebook volumes created when deploying notebooks are cleaned up and deleted.
DO NOT proceed until all profiles are removed. If a profile cannot be removed, review the troubleshooting section for instructions to [forcibly delete a profile][#cannot-finalize-removing-a-profile].
-
Uninstall Kaptain using KUDO. This command deletes the Kaptain Percona DB store, all deployments, statefulsets, jobs, and volumes associated with the kubeflow namespace.
-
Ensure that the
kubeflow
namespace is removed: -
Clean up any additional configuration resources left behind by KUDO-managed Kaptain, so you can install Helm-managed Kaptain correctly. This command selects various resource types with the label
kudo.dev/instance
of either"kaptain"
or beginning with"kaptain-"
.Kaptain is now fully uninstalled.
-
Remove Kudo:
Troubleshooting and overcoming failures
Sometimes, the uninstall steps will fail or hang because a cleanup task gets stuck. In that case, you will need to force Kubernetes to uninstall the resources. In such cases, you may need to clean up things manually. Follow the steps below as necessary.
Cannot finalize removing a profile
-
Skip the finalizer and manually clean up the namespace and related persistent volumes.
-
Try to delete the profile again:
-
Find, and make a record of all persistent volumes IDs associated with this profile. You will delete these volumes later.
-
Delete the namespace associated with the profile.
-
Delete any persistent volumes that were associated with this profile.
KUDO Kaptain uninstall fails
If KUDO Kaptain uninstall hangs and fails to properly uninstall, force the removal of the Kaptain instance, and manually clean up volumes and namespaces.
-
Find the KUDO instance which is stuck:
-
Patch it, and remove it:
-
Repeat the above until
kubectl get instances -n kubeflow
returns an empty list. -
Find, and make a record of all persistent volumes IDs associated with Kubeflow. Look at the CLAIM column and look for values such as
kubeflow/datadir-kaptain-mysql-store-pxc-0
. You will delete these volumes later. -
Remove the kubeflow namespace:
-
Remove all dangling persistent volumes: