Introduction

Over the past two years, artificial intelligence (AI) has been inseparably linked to the cloud for most mid-sized companies. Anyone wanting to use ChatGPT, DeepL, or Midjourney had to send data to a server — often outside the EU. 2026 now marks a radical turning point: AI is moving permanently into hardware.

So-called AI PCs and AI smartphones are no longer just marketing terms. They feature specialized processors called NPUs (Neural Processing Units), optimized to run complex neural networks directly on the local chip. For SMEs, this means shifting from a "cloud-first" to an "edge AI" strategy.

What looks like a simple hardware upgrade at first glance is, on closer inspection, a profound shift for corporate compliance. For the first time in years, companies have the chance to harness the power of AI without ceding sovereignty over their data to tech giants. But this gain in autonomy comes with a new burden of responsibility.

The Technical Deep Dive: What's Behind Local AI?

To understand the legal and strategic implications, it helps to look at the technology behind this "deep integration."

Traditional AI applications worked like a phone call: you ask a question, it's sent over the internet to a data center, processed there, and the answer comes back. With AI-integrated devices, this process happens internally instead.

  • NPUs (Neural Processing Units): These specialized cores are designed to perform the mathematical operations behind AI models (matrix multiplications) extremely fast and energy-efficiently, without burdening the main CPU.
  • Small Language Models (SLMs): While models like GPT-4 are massive, optimized models such as Llama 3, Mistral, or Microsoft's Phi-3 deliver impressive performance with far lower memory requirements. They fit directly into the RAM of a modern laptop.
  • Unified memory architecture: Modern chips (like Apple's M series or Intel's Core Ultra) let AI access data at lightning speed, enabling real-time features like live translation or automatic video analysis.

The GDPR Perspective: A Win for Data Protection?

For data protection officers and compliance leads, on-device AI is a double-edged sword.

The Opportunities: Privacy by Design

Under Art. 25 GDPR, companies are required to ensure data protection through technology design. Local AI is the ideal tool for this:

  • Data minimization: Since no personal data needs to be transmitted to external servers, the risk of data leaks at third-party providers is minimized.
  • No third-country transfers: The often problematic transfer of data to the US (keyword: Data Privacy Framework) is eliminated entirely for many processes.
  • Sovereignty: The company retains full control over who has access to the processed information. The AI provider "reading along" for training purposes is ruled out.

The Risks: The "Black Box" in the Operating System

The danger lies in a lack of transparency. Microsoft, Apple, and Google integrate AI features so deeply into their operating systems that it's often no longer clear to users when AI is actually active.

  • Built-in logging: Features like Microsoft's "Recall" (which records screen content for later search) create sensitive datasets on the device. If these aren't properly encrypted or protected by permission concepts, entirely new entry points open up for internal and external attackers.
  • Shadow AI 2.0: When the operating system offers AI tools out of the box, employees use them for sensitive tasks (e.g., analyzing personnel files or financial plans) without any prior review by the data protection officer.

The EU AI Act: Hardware Integration Under the Microscope

The new EU AI Act is written to be technology-neutral. That means it doesn't matter whether the AI runs in the cloud or locally on the sales director's laptop.

Classifying the Risks

SMEs need to assess which risk category their local AI use falls into:

  1. Minimal risk: Local spam filters or webcam image enhancement. There are hardly any requirements here.
  2. Transparency obligations: AI systems that interact with people or generate content (e.g., local chatbots for customer service) must be labeled as such.
  3. High-risk AI: This is where things get critical for SMEs. If a company uses locally installed AI software to evaluate job applicants, assess creditworthiness, or monitor employees, it's subject to strict requirements on risk management, data quality, and human oversight.

Documentation Is Mandatory

SMEs must maintain a record even for local systems. If AI features are used for business purposes, they must appear in the Record of Processing Activities (ROPA) and in the documentation required under the AI Act. "I didn't know my laptop did that automatically" won't hold up with regulators.

Cybersecurity and the Role of NPUs

This deep integration creates new attack vectors that go beyond classic viruses.

  • Local prompt injection: Attackers could try to manipulate local AI models through crafted emails or documents to extract protected information.
  • Model theft: If an SME distributes its own model, fine-tuned on company data, locally across devices, that model itself becomes a valuable asset that must be protected against theft.

Vulnerabilities in NPU firmware: Since NPUs are relatively new, their firmware is often not yet as mature and hardened as that of traditional CPUs. This gives hackers new opportunities for privilege escalation.

Management Liability: A Wake-Up Call

Similar to the NIS2 Directive, responsibility for the secure use of technology is moving directly into the spotlight for company leadership.

Duties of care

Managing directors must ensure that:

  1. AI literacy exists within the company. Leadership must understand the risks in order to make sound investment decisions.
  2. Monitoring mechanisms are established. You can't blindly trust the output of a local AI (hallucinations).
  3. Resources are allocated for securing these systems.

In cases of gross negligence — for instance, if it's known that employees are running highly sensitive customer data through unvetted local AI tools and damage results — personal liability looms. Implementing AI isn't purely an IT project; it's a strategic compliance task.

Governance Framework: How SMEs Can Safely Roll Out AI Hardware

An uncontrolled rollout of AI PCs inevitably leads to legal problems. A structured approach is therefore essential.

1. The "AI Acceptable Use Policy" (AUP)

Create a binding set of rules for all employees. It should clarify:

  • Which OS-level AI features are permitted, and which must be disabled?
  • Which categories of data may be processed with local AI?
  • How must AI outputs be labeled and verified?

2. Technical Mobile Device Management (MDM)

Use your IT systems to manage AI features centrally. For example, features that mirror data to the cloud without asking (like cloud syncing of AI analyses) can be blocked system-wide.

3. Updating Employment Contracts and Works Agreements

Since AI integration often changes how work can be monitored or evaluated, it's necessary to involve the works council early on (where one exists) and update internal policies on performance monitoring.

4. Vendor Risk Management (VRM)

Even when AI runs locally, the software still comes from the manufacturer (Microsoft, Apple, Dell, Lenovo). SMEs need to check what telemetry data these hardware makers collect despite local processing.

Conclusion: Don't Fear Hardware AI, But Respect Its Complexity

The growing spread of devices with deep AI integration is a historic opportunity for mid-sized businesses. It allows them to capture the efficiency gains of AI while avoiding the "data graveyard" of the cloud.

But autonomy requires discipline. SMEs that blindly jump on the AI PC bandwagon without adapting their compliance processes risk hefty fines under the EU AI Act and the GDPR. Those who combine hardware innovation with professional compliance, however, gain a genuine competitive advantage: digital resilience.