[Solved] Deleting Argocd Config makes the resource invisible but not deleting it completely and hangs in Argocd

 Error:-

Recently while depreciating the resources from the Argocd specifically the ingress , removed the application set from the Argocd after which Argocd tried to remove the resources. But while setting up Ingress we have configured delete_protection.enabled=true which was not removed. So  At this point the Argocd was not able to remove the resource due to the delete protection enabled rather start showing resource as Missing in Argocd Dashboard while multiple events showed resource not able to delete due to protection enabled. 

Now because we were using the Keda, the Keda validation Error also go high while the ingress getting struck to delete and keda trying to create the resource but was not able to either.

Cause:-

The obvious cause of the error was that the delete_protection.enabled should be set to the false first before removing the Appset so it can delete the resource successfully. However since it was not done, Argocd hanged in deleting state but cannot delete due to the flag.

At this point tried to disable the flag from the AWS Dasahboard on ALB Ingress which was successful however it did cause any change since Argocd already had a config and would try to sync it back and since multiple micro services calling same ingress it cant be changed manually at same time across all services.

Even if you somehow try to sync it back than also Argocd will revert back the config as it will sync back any changes to maintain the state. So only option for you to rollback the changes and set the delete_protection.enabled to false and than delete again. But this was also not possible because when argocd is trying to delete and struck it cannot sync in new changes. So you need to complete the previous sync than only new sync will happen.

So the last viable option is to delete the ingress using the kubectl, however in this case if you try to delete it will hang up and delete will not complete. So i tried to delete it forcefully and grace period 0 like 

--force --grace-period=0

However even this would not get finish and sort out the problem

Solution :-

In order to resolve the above issue only solution that you can apply to delete the resource stuck in kubernetes is by patching the resource and setting finalizers field to null i.e. 

kubectl patch ingress abc-ingress -n namespace -p '{"metadata":{"finalizers":null}}' --type=merge

Explanation of above command:-

kubectl patch: This is the base command for patching a Kubernetes resource.

ingress abc-ingress: Specifies the type of resource (ingress) and the name of the resource (abc-ingress) you want to patch.

-n namespace: Specifies the namespace where the ingress resource is located.

-p '{"metadata":{"finalizers":null}}': Provides the patch data in JSON format. This specific patch is setting the finalizers field in the metadata to null.

--type=merge: Specifies the patch type. In this case, merge indicates that the patch should be applied as a strategic merge patch, which is a type of patch that Kubernetes uses to intelligently combine the patch with the existing resource configuration.

Finalizers are special metadata fields in Kubernetes resources that ensure certain cleanup tasks are completed before the resource is fully deleted. By setting the finalizers field to null, this command effectively removes any finalizers associated with the ingress resource.

This is often necessary if a resource is stuck in a terminating state because it can't complete its finalizer tasks.

So next time if you are struck in deleting a resource in kubernetes you can use the patch resource to remove it from the kubernetes cluster.