Choose where it runs.
Use a common provider interface for Apple’s on-device model and OpenAI-compatible endpoints. Make routing decisions in your app.
On-device AI for your Apple app.
Connect your React Native app to Apple’s on-device model and your own cloud endpoint. Keep conversations within the model’s context window and control where each request runs.
v1.0.0 · Available on npm
WHERE A REQUEST GOESArchitecture overview
01Your appA message. A task. A conversation.
02Your routing policyAvailability, context, and your preferences.
ON DEVICE ↓
Apple modelNative inference on eligible Apple devices.
CONFIGURED FALLBACK ↓
Cloud providerAn OpenAI-compatible endpoint.
Keep context in reach.
Fit conversations to Apple’s 4–8K token window, depending on the model variant. Choose a sliding window or rolling summary, and render current app state into each request.
Build with React.
Connect a router to useChat for context management and streaming. Use useAvailability and useGenerate for model status and one-shot requests.
A LOOK AT THE APIA few lines
A few lines
to begin.
Check the model before you ask it to work. Handle unavailable devices and generation errors in your interface.
API example. Requires the native module in a compatible app build.
Read the setup guide →First generation
import { createAppleProvider } from '@taaltreelabs/on-device-llm/apple';
const model = createAppleProvider();
const availability = await model.availability();
if (availability.available) {
try {
const result = await model.generate({
messages: [{ role: 'user', content: 'Explain why leaves change color.' }],
});
console.log(result.text);
} catch (error) {
// Availability does not guarantee generation will succeed.
console.error('Generation failed', error);
}
} else {
console.log('Model unavailable:', availability.reason);
}
See where it fits.
Review the documented requirements and platform support, then follow a guide for your integration.
Compatibility & limitations →