Usage & routing

Complete native setup first. These examples use the Android provider with the shared router and React hooks.

Use the shared router

The Android provider implements the same LLMProvider interface as the Apple and HTTP providers. Imports are lazy, so listing both native providers is supported: the unavailable platform is skipped by the router.

llm.ts

import { createAppleProvider } from '@taaltreelabs/on-device-llm/apple';
import { createRouter } from '@taaltreelabs/on-device-llm/core';
import { createAndroidProvider } from '@taaltreelabs/on-device-llm-android';

export const llm = createRouter({
  providers: [createAppleProvider(), createAndroidProvider()],
});

This configuration has no cloud fallback. It does not disable ML Kit's own metrics collection. See Privacy & SDK terms.

Add cloud fallback deliberately

import { createRouter } from '@taaltreelabs/on-device-llm/core';
import { createOpenAIProvider } from '@taaltreelabs/on-device-llm/openai';
import { createAndroidProvider } from '@taaltreelabs/on-device-llm-android';
import { fetch as expoFetch } from 'expo/fetch';

export const llm = createRouter({
  providers: [
    createAndroidProvider(),
    createOpenAIProvider({
      baseUrl: 'https://your-endpoint.example.com/v1',
      model: 'your-model',
      // Use your authenticated backend; do not bundle a provider secret.
      fetch: expoFetch as unknown as typeof fetch,
      contextWindow: 128_000, // Replace with your endpoint's actual limit.
    }),
  ],
});

The cloud endpoint receives content when selected. Configure the routing policy for your app. The shared guardrail fallback setting cannot identify Android refusals that never map to that error code. Do not assume identical refusal handling across platforms.

Connect React

Pass either provider or router into the hooks from the main package:

import { useChat } from '@taaltreelabs/on-device-llm/react';
import { Button, Text, View } from 'react-native';
import { llm } from './llm';

export function Assistant() {
  const { streamingText, status, error, send, stop } = useChat({
    provider: llm,
    systemPrompt: 'Give a concise, useful answer.',
  });
  return (
    <View>
      {streamingText !== undefined && <Text>{streamingText}</Text>}
      {error && <Text>Unable to complete the request.</Text>}
      <Button
        title="Ask"
        disabled={status !== 'idle'}
        onPress={() => { void send('Explain how a rainbow forms.').catch(() => {}); }}
      />
      <Button title="Stop" onPress={stop} disabled={status === 'idle'} />
    </View>
  );
}

For a full conversation view, render messages as in the shared React quick start. useChat also handles context fitting and exposes completed and partial responses.

Shared guides, different capabilities

Use the context manager with the Android provider, and read the stream contract for cancellation and final-result handling. Provider-specific details in those guides are labeled Apple or HTTP; they do not imply Android feature parity.

Android does not support runtime schemas or tool calling. See its capability table before reusing an Apple example.