Switching AI providers without starting from scratch on compliance

Martin Bastius

25.08.2026

5

min.

Use AI to summarize this article

How do you switch AI providers without starting your data protection and security approval process from scratch?

The pace of development in artificial intelligence is immense. What was the standard yesterday (like OpenAI's ChatGPT) is being challenged today by alternatives such as Claude (Anthropic), Google Gemini, or specialized open-source models. Companies want and need to remain technologically flexible to ensure they are always using the most powerful, secure, or cost-efficient model.

However, in many organizations, the thought of switching AI providers causes uncertainty: Do we have to go through the entire legal, data protection, and security approval process all over again?

The short answer is: No. If you build your AI governance properly, you are by no means starting from scratch. A sustainable compliance structure does not depend on a single provider or tool, but rather forms the foundation for your entire AI stack.

The basic principle: The use case is what matters, not the provider

Anyone who views compliance for every new AI tool in isolation as a one-off project is building a house of cards. If the provider changes, the governance collapses.

Forward-thinking companies therefore view compliance as long-term infrastructure. The legal and organizational framework is built primarily around the use case and the data being processed – not around the specific software product.

For example, if a marketing team switches from provider A to provider B for creating blog posts, neither the business purpose nor the sensitivity of the data used changes. The existing governance foundation remains intact; only the specific parameters of the new service provider need to be reviewed.

What can you reuse – and what needs to be re-evaluated?

The following overview provides a transparent look at which compliance components remain in place when switching providers and where a targeted review is required:

Compliance Area	Reusable (Remains Valid)	New or Needs to Be Reassessed
AI Policy & Guidelines	Internal rules of conduct, restrictions (e.g. entering confidential data), ethical guidelines.	Specific tool approvals in the appendices to the policy.
Use Case Assessment	Purpose limitation, process description, intended use of outputs, business case.	Technical model characteristics, hallucination risk, specific bias risks.
Human Oversight & Processes	Four-eyes principle, approval processes for AI outputs, roles & responsibilities.	Interface integration, role concepts and access rights management in the new tool.
Training & AI Literacy	Basic training on risks, prompting guidelines and data protection fundamentals.	Tool-specific user training and onboarding.
Data Protection & Legal (GDPR)	Legal basis (Art. 6 GDPR), basic concept for handling data subject rights.	DPA / data processing agreement, international data transfers (SCCs/DPF), retention periods, opt-out from model training.
IT Security & Vendors	General security policies, TISAX/ISO minimum requirements.	Provider SOC 2 / ISO certifications, subprocessors, encryption standards.
EU AI Act Documentation	System classification based on the use case, basic risk management.	Technical model documentation, provider compliance evidence.

‍

Practical example: Switching in the marketing team

Let’s imagine a concrete scenario: A medium-sized company has been using Provider A to create social media copy and customer correspondence. Now, they want to switch to Provider B because it offers better German language models and more favorable API terms.

What remains the same?

Use case approval: The risk analysis for drafting marketing copy is already complete.
The policy: Employees are still aware that no unencrypted personal customer data may be processed.
Human-in-the-loop: The requirement that no text be published without human review remains unchanged.

What specifically needs to be updated?

Data Processing Agreement (DPA): A legally compliant DPA must be signed with Provider B.
Review of model training: It must be contractually ensured that Provider B does not use input prompts for general model training (Data Retention & Training Opt-Out).
Record of Processing Activities (ROPA): The name of the data processor and, if applicable, the processing location (e.g., EU server vs. US transfer) must be updated in the ROPA.
Sub-processors & Security: The new provider's certifications (e.g., ISO 27001, SOC 2 Type II) are filed as part of the vendor risk management process.

Key AI Governance questions when switching providers

Does the AI risk assessment need to be redone when switching providers?

No, not from scratch. The fundamental risk assessment relates to the use case (e.g., automated resume analysis vs. marketing copy). You only need to check whether the new model introduces specific new risks—such as different privacy settings, changed hallucination rates, or different hosting locations.

Does a Data Protection Impact Assessment (DPIA) need to be updated?

If a DPIA has already been conducted for the use case, it must be reviewed and updated based on thresholds when switching AI providers. A completely new DPIA is usually not necessary. The update focuses on the changed technical and organizational measures (TOMs) as well as the new data processor.

Which documents specifically need to be updated when switching?

Record of Processing Activities (ROPA): Update service providers and subcontractors.
Privacy Policy (Transparency Obligations): If external parties are affected, update the service provider information.
Vendor Assessment File: Archive the new provider's security and compliance documentation.
Internal AI whitelist / tool registry: Remove the old tool or mark it as inactive, and approve the new tool.

‍

EU AI Act & GDPR: Regulatory considerations you need to know

When switching AI tools, two regulatory levels intersect:

The GDPR perspective: The focus here is on vendor management. Sign a DPA, verify third-country transfers (e.g., the Data Privacy Framework for US providers or Standard Contractual Clauses), and ensure that no data is used for training without a legal basis.
The EU AI Act perspective: Pay attention to the distribution of roles. In everyday language, we often refer to the "AI provider." However, under the EU AI Act, your company is usually the "deployer", while OpenAI, Anthropic, or Google are the "providers" . As long as the model is not fundamentally retrained or brought to market under your own name as a high-risk system, your obligations as a deployer remain largely the same when switching providers.

Conclusion: Flexibility through a solid compliance infrastructure

Switching AI providers will become routine in the coming years. If you reinvent the wheel every time you switch, you will significantly stifle innovation within your company and miss out on the opportunities offered by modern AI tools.

With a centralized compliance infrastructure, data protection and AI governance transform from a blocker into an enabler: You create a stable foundation that allows new AI models to be evaluated and integrated quickly, securely, and scalably.

Build your compliance correctly once – and stay flexible for the long term.

How heyData supports you

With the all-in-one compliance platform from heyData , you can manage AI governance, data protection, and vendor risk management in one central location. Instead of isolated silos, you create a reusable infrastructure that allows new AI tools to be vetted and approved for legal compliance in just a few steps.

FAQ

Can we test a new AI provider immediately if an AI policy exists in the company?

Yes, provided your AI policy includes basic rules for testing and evaluation phases (sandbox environments). Importantly, no sensitive customer data or trade secrets should be entered during testing phases until the Data Processing Agreement (DPA) and the provider's security review are fully finalized.

Does our role under the EU AI Act change if we switch from an API solution to a self-hosted open-source model?

That depends on the specific implementation: If you simply self-host an open-source model and use it as-is, you generally remain a deployer. However, if you make substantial model modifications (such as fine-tuning with your own datasets for specific high-risk purposes) or place the system on the market under your own name, you could legally be classified as a provider under the EU AI Act, which entails significantly more extensive obligations.

Do we need to retrain staff every time we switch AI tools?

No. Retraining on general fundamentals (AI literacy, data protection basics, confidentiality rules, best practices for prompting) is not necessary. A short, targeted user training session or documentation covering the specific features, interfaces, and approvals of the new tool is sufficient.

What happens to the data with the previous provider when we terminate the contract?

During the offboarding of an AI provider, you must ensure that all processed data, prompts, and temporary stores can be deleted or retrieved in compliance with the GDPR. Review the exit clauses (Data Retention & Deletion Policy) of the previous provider to confirm the complete deletion of all data from their servers and backups.

Published