Your AI Model Is Not Your AI Application
- 4 days ago
- 5 min read
Protecting the full generative AI pipeline on Kubernetes and OpenShift with RackWare SWIFT
If recovery brings back the model but loses the knowledge and state around it, the AI application is still down.
A usable AI service is more than a model
Generative AI gets attention at the model layer, but users experience an application. That application may combine an interface or API, enterprise content, retrieval, model serving, persistent data, security, networking, and Kubernetes services. Lose the connections between those parts and a healthy model cannot answer a business question.
Kubernetes and OpenShift make these components easier to deploy and scale. They also create a recovery surface that spans namespaces, custom resources, controllers, certificates, services, storage, and external repositories. Reapplying manifests can recreate objects, but it does not automatically restore the data, mappings, dependencies, and startup order that made the service useful.
RAG in 60 seconds
Retrieval-augmented generation (RAG). The application finds relevant information before asking the model to answer, much like giving it an open book for the question.
Embeddings. Text is converted into numbers that represent meaning, so related content can be found even when the wording differs.
Vector database. Those representations are stored as a searchable knowledge index that returns the most relevant context.
Why that changes recovery
The model may be replaceable from a repository. The costly or unique parts are often the knowledge index, ingested content, conversation or workflow state, application configuration, and the Kubernetes platform services around them. Protect those pieces separately, or skip one, and the cluster may look healthy while the AI service remains incomplete.
THE SWIFT APPROACH RackWare SWIFT protects the selected Kubernetes or OpenShift application boundary, including its objects and eligible persistent-volume state, then applies target-specific mappings and recovery order. The pattern is not tied to one model, vector database, framework, or user interface.
Meet KubeChat: small enough to understand, complete enough to matter
To make AI resilience tangible, we used KubeChat, a conversational application that answers Kubernetes questions from Kubernetes blog and documentation content. It is a compact example, but its recovery surface looks familiar: a user experience, retrieval, embeddings, a vector knowledge store, model serving, persistent user state, and platform dependencies.
A question enters KubeChat. The pipeline finds relevant context in Qdrant (RAG Database), sends a grounded prompt to a Qwen model served through KServe and vLLM stack, and saves the conversation. That creates a meaningful recovery test: can the service return with both its knowledge and the state users already created?
Start with the complete recovery boundary
The source OKE cluster shows why a namespace-only view is not enough. KubeChat, Qdrant, the embedding service, and the Qwen predictor run in ai-demo, while cert-manager and KServe provide prerequisites outside that namespace. Services expose the application and connect its internal components.
Knowledge and user state. The qdrant-data volume held the vector index; kubechat-data held the SQLite conversation database.
Application objects. The ai-demo scope included KubeChat, Qdrant, TEI, model-serving resources, Services, configuration, and persistent volume claims.
Platform and target needs. cert-manager and KServe had to recover first, while GKE storage, networking, security, images, and external model access had to be mapped and validated.
Put SWIFT around that boundary
In this lab, SWIFT ran outside the managed clusters and orchestrated protection from OKE to a pre-provisioned GKE target. It is shown target-side in the topology for readability, but remains external to both clusters.
One policy restores the platform in the right order
The hourly Stage 1+2 DR policy grouped three operations so prerequisites could return before the application that depended on them:
Level 0: cert-manager resources, including required custom resources and webhooks.
Level 1: the KServe namespace and serving control plane.
Level 2: the ai-demo application, model-serving resources, and application PVCs.
See the complete protection flow
When the policy ran, the SWIFT replication view showed Stage 1 capture and Stage 2 recovery for all three scopes, including job status, timing, policy association, and warnings for operator review. The order matters because an application restored before its controllers, certificates, or services may still be unavailable to users.
Protect the state that gives the application value
The ai-demo operation included both stateful parts of KubeChat. SWIFT reported kubechat-data and qdrant-data at 100 percent. Leaving out either volume would create an incomplete recovery: users would lose their conversations, or the service would lose the retrieval knowledge that grounds its answers.
Recover into a different cluster
On GKE, the restored namespaces, running pods, and services show the same application and prerequisite layers in a different cloud environment. The new LoadBalancer address is expected; service exposure is a target-specific mapping, not a reason to leave the application unprotected.
Recovery is proven where the user returns
After the ordered recovery completed, KubeChat opened on GKE with the previous chat list and conversation state visible. The interface also reported the Qwen, KServe, TEI, Qdrant, and retrieval-index components available. The test moved beyond recovered objects to a working user journey.
THE MEASURE OF RECOVERY A completed job proves that a workflow ran. A usable service with its expected data and dependencies proves that the application recovered.
What this example proves
Cross-cloud application recovery. The protected application returned on GKE with target-specific mappings.
Ordered dependency recovery. cert-manager and KServe returned before ai-demo.
Stateful continuity. Both application volumes reached 100 percent and prior chat state was visible.
Keep consistency in the application design
Qdrant and SQLite are independent data stores. Application owners must decide whether recovery requires a lock, flush, quiesce step, or application-native snapshot. SWIFT pre/post scripts and YAML hooks can coordinate workload-specific actions, but those actions must be tested.
Use the same pattern across your AI estate
KubeChat is one example. The same application-level method can cover RAG indexes and ingestion state, training checkpoints and artifacts, or agentic memory, tools, policies, and workflow state.
Four questions before the first drill
What is irreplaceable? Identify mutable data, indexes, checkpoints, histories, and metadata.
What must recover together? Map dependencies, order, and any consistency actions.
What changes at the target? Document storage, networking, identity, images, and model access.
What proves recovery? Test the user journey, data integrity, new writes, and recovery objectives.
From backup status to recovery confidence
AI resilience starts by treating the pipeline as an application, not a collection of replaceable containers. RackWare SWIFT helps protect that boundary, map it to another environment, restore dependencies in order, and exercise the result through drills. Start with one AI service that matters and prove users can return to a working experience.
READY FOR A PRACTICAL NEXT STEP? See how this approach can fit your Kubernetes or OpenShift environment: request a live RackWare demo or contact the RackWare team.
About the author
Aniket H. Kulkarni is Vice President of Engineering at RackWare and leads engineering for RackWare SWIFT across Kubernetes application mobility, backup, disaster recovery, and resilient cloud infrastructure.
Comments