Compatibility & limitations
Requirements and feature support for on-device-llm 1.0.0. Android has its own package and capability table; the JavaScript core remains shared.
Requirements
| Requirement | Documented target |
|---|---|
| Apple on-device model | iOS / macOS 26.0 or newer; compatible native integration required |
| Example app Expo SDK | 57 |
| Example app React Native | 0.86.3 |
| React | Optional peer dependency; required for the React hooks |
| Runtime dependencies | None |
| Device | Apple Intelligence-eligible hardware, enabled feature, and downloaded model assets |
| Android on-device inference | Separate package: on-device-llm-android |
The native module links into iOS apps targeting 16.4 or newer. Below iOS 26, the Apple provider reports unsupportedPlatform, allowing a configured router to use a cloud provider. The example app versions above describe the release source; peer dependencies do not enforce an Expo or React Native version floor.
core and openai can run in Node or a modern browser without React Native. Importing the root is designed to be safe on unsupported platforms: the Apple provider resolves its native module lazily and reports unsupportedPlatform. This does not provide browser or Android on-device inference.
Provider features
| Feature | Apple provider | OpenAI-compatible provider |
|---|---|---|
| Streaming | Native token deltas | Streaming with a compatible injected fetch; otherwise one aggregated delta |
| Structured output | When guided generation is reported | response_format, subject to endpoint support |
| Tool calling | When the model and native module support it | Not supported; requests with tools fail as invalidRequest |
| Token counting | Exact from iOS 26.4; estimated on 26.0–26.3 | Estimated |
| Context window | Reported by the model: 4K or 8K by variant; may be UNKNOWN | Explicitly configured; UNKNOWN by default |
| Locales | Enumerated by the native model; package docs list 24 tags | UNKNOWN unless configured |
MockProvider supports scripted results, streams, and errors for tests. Its capabilities are configurable; it does not execute tool handlers. See custom providers.
What differs on iOS 26
The on-device model runs from iOS 26.0, not only 27.0, but the iOS 26
bridge has less to work with than iOS 27's. All gaps disappear at iOS 27; none of them
affect the openai provider or the router.
| Area | iOS 27+ | iOS 26.x |
|---|---|---|
contextOverflow error | Carries contextSize and tokenCount | Neither field is present |
decodingFailure error | Carries raw model output (rawContent) | No raw output |
unsupportedLocale error | Names the offending locale | No locale |
rateLimited error | Carries resetDate | No resetDate |
usage / finishReason | Real per-response usage | No usage; finishReason is always 'stop', even when truncated |
| Token counting | Always exact | exact from 26.4 onward, estimated on 26.0–26.3 (256-token safety margin) |
modelLabel | variant.displayName, queried live | Fixed string "Apple Foundation Model" |
supportsVision / supportsReasoning | Queried live | Fixed false |
supportsGuidedGeneration / supportsToolCalling | Queried live | Fixed true |
contextWindow | Device-reported | 4096 below iOS 26.4 |
Availability
| Reason | Meaning and suggested behavior |
|---|---|
deviceNotEligible | Hardware is not eligible, or the configured locale is unsupported. Offer another configured provider or hide the feature. |
notEnabled | Ask the user to enable Apple Intelligence if they want this feature. |
modelNotReady | Assets may still be preparing. Allow a later availability check. |
unsupportedPlatform | The native capability is unavailable on this platform, OS, or build. |
An available model can still fail at generation time. unsupportedLocale is a request error and a default fallback trigger, rather than an availability reason. See Troubleshooting.
Native build and streaming requirements
Use a development or production build containing the native module, not Expo Go. The native module requires a deployment target of at least 16.4, which the default Expo template already meets. The model's runtime requirement is separate: iOS 26 or newer.
For cloud token streaming in React Native, pass expo/fetch or another streaming fetch implementation. The Apple provider is unaffected. Follow Getting started and the streaming guide.
Data flow and fallback
Cloud requests transmit content to the selected endpoint. Cloud summarization and app-defined tools may also send data off-device. Use an appropriate backend or scoped authentication flow; do not embed long-lived provider secrets in a mobile app.
Default fallback covers unavailability, context overflow, network errors, rate limits, unsupported locales, and transient unknown failures. Guardrail fallback is off by default. Cancellation and invalid requests never fall back. The first event delivered to a stream consumer closes the fallback window, including a tool-call event. Read the routing guide.
When to choose something else
This package does not provide UI components, embeddings, speech, transcription, image generation, adapter loading, or retrieval. If you only call a cloud model, your provider's SDK may be sufficient, though the standalone context manager can still be useful.