Four senses, one brain, on-device first.

1. The premise: intelligence without upload

Most apps that feel "smart" in 2026 are smart because a server somewhere is doing the work. The app captures an image, transcribes an audio clip, or logs a click, then ships the payload to an inference endpoint. The cleverness lives in the data center.

Cue is the opposite wager. Every feature that looks like the app is paying attention to you, the weather around the user, whether the photo being held up is sensitive, whether the hour is right to interrupt, was built by composing four on-device Apple frameworks. Those run entirely on your iPhone. Card reading goes one step further: the on-device pass happens first, then by default the photo is sent securely to our AI provider for a more accurate read, unless you turn on "Scan on-device only."

The payoff is both ethical and practical. Ethical: the sensing layer keeps your context on the iPhone, and you can keep card scanning fully on-device too. Practical: the app is fast, the local frameworks work on an airplane, and there is no data-broker business model behind any of it.

2. Foundation Models: on-device categorization

When a new card lands in the wallet, the design calls for Apple Intelligence's Foundation Models framework (import FoundationModels, iOS 18.1+) to categorize it. Is this a coffee shop card? A restaurant? A big-box retailer? A personal care brand?

The alternative would be a keyword map ("starbucks" → coffee, "target" → retail), which breaks on regional chains, misspellings, and anything the developer has never heard of. Foundation Models instead gets a one-shot prompt with the card's brand string and returns a structured category. The model is small, runs on the Neural Engine, and in our testing finished in under 200ms on an iPhone 15.

// Services/FoundationModelsCategorizer.swift
@available(iOS 18.1, *)
func categorize(brand: String) async -> CardCategory {
    guard let session = try? LanguageModelSession() else { return .other }
    let prompt = "Classify this retail brand into one CardCategory: \\(brand)"
    let reply = try? await session.respond(to: prompt)
    return CardCategory(fromModelReply: reply?.content) ?? .other
}

Until it ships, and wherever the framework is unavailable (older hardware, Apple Intelligence off), the scanner uses the keyword map. The feature degrades instead of failing. A user on an iPhone 13 gets the same wallet; they just don't get the "Food & Drink" pill auto-filled.

3. Vision: the dual-engine scan

Cue's camera pipeline is built on Apple's Vision family, and the on-device pass always runs before anything goes to the cloud:

Engine A, VNRecognizeTextRequest + VNDetectBarcodesRequest

The auto-fire scanner (detailed in the Scanner kit) runs these two Vision requests in parallel against a stabilized frame. Vision is on-device, free, fast (100-250ms), and private. It reads text, barcodes, and QR codes in one pass.

Engine B, DataScannerViewController (VisionKit), in development

For users who prefer to aim instead of photograph, a "Live Capture" mode built on DataScannerViewController with recognizedDataTypes: [.barcode(), .text()] is prototyped: the same machinery the Camera app uses for Live Text, repurposed as a barcode-forward scanner. Highlighted rectangles land on what the camera sees, one tap captures the card number, still on-device. It's an engineering spike today, not a v1 surface; the shipping scanner is Engine A.

By default, after Apple Vision reads what it can on-device, the scanned photo is sent securely to Anthropic's Claude API to extract the structured card details. You can disable the cloud path entirely with the "Scan on-device only" setting, in which case extraction stays on-device with Vision alone, at the cost of slightly less rich extraction for unusual cards. And the tension this page keeps circling has a planned resolution: on our roadmap, targeted mid-2027, is card reading that runs entirely on your device, no cloud pass at all.

4. The Share Extension: regex on device, no key, no upload

Vision is the first on-device parser in CardCue Pro. The iOS Share-from-Mail extension, built and shipping, is the second. Where Vision reads pixels, the Share Extension reads the text of the email a user just tapped Share on, and it does so with pure regex patterns running inside the extension process. Nothing is uploaded. The email body was already on the phone because Mail had already rendered it; the extension just structures what is there.

That matters for the on-device story because structured-text extraction from a gift-card email is a problem that does not need cloud intelligence. Regex handles email text on-device, full stop. Camera scans are the different case: Vision reads the photo on-device first, and by default the image then goes securely to Claude with Vision's read attached as a hint, because Claude is the engine that extracts the details most accurately. Turn on "Scan on-device only" in Settings and Vision works alone. The Share-from-Mail path never reaches the network at all.

The extension binary carries no Claude API key. The only binary with a key is the main app, because only the main app ever has a reason to call out. The Share Extension target has no key in its Info.plist, no key compiled in, and no network code path at all. It is, by construction, incapable of making a request to Anthropic. It reads the shared email text, runs regex over it, and writes the result into the shared App Group for the main app to pick up.

What the regex extracts: brand, card number, PIN, balance, expiration, and for certificates, the redeemable item. When a pattern hits, the field arrives in the Add Card form pre-filled. When a pattern misses, the field is left blank and the user fills it by eye. The app never shows a "we couldn't read your email" dead end; a partial parse is still a useful parse, and the user is never blocked.

The second parser "Vision reads cards in the world. Regex reads cards in the inbox. Neither one needs the cloud, and the extension that runs the second parser does not even have the option."

5. Sensitive Content Analysis: the polite refusal

The card scanner is a camera. Cameras sometimes see things they shouldn't, a photo library image the user accidentally picked, a frame that caught something private on a nearby screen. CardCue Pro treats that possibility as a first-class concern, not a content-moderation afterthought.

Before any photo enters the Vision pipeline, it goes through Apple's Sensitive Content Analysis framework (import SensitiveContentAnalysis, iOS 18+):

// Services/SensitiveContentGate.swift
static func analyze(_ image: UIImage) async -> SensitiveContentResult {
    guard #available(iOS 17.0, *) else { return .safe }
    let analyzer = SCSensitivityAnalyzer()
    guard analyzer.analysisPolicy != .disabled else { return .safe }
    do {
        let result = try await analyzer.analyzeImage(image.cgImage!)
        return result.isSensitive ? .sensitive : .safe
    } catch {
        return .error(error)
    }
}

If the analyzer flags the frame, the scanner stops. The user sees a single-line message, "We couldn't read that photo. Try a clearer shot of the card.", and is not told why the image was rejected. The whole check runs on-device in under 100ms.

That design choice has two effects. The obvious one: Cue does not host or transmit anything the framework flags. The less-obvious one: the user is protected from accidentally OCR-ing something private even if they had no intention of sharing it. It's the same posture a well-designed password manager takes when a shoulder-surf concern is raised.

6. WeatherKit: the ambient bias

The Notification Conductor (see the Ecosystem kit) gates every outbound alert through a sequence of checks. One of them, newly added, is weather.

When WeatherBiasService.currentBias() returns .suppress, the Conductor drops the CarPlay banner surface from a routing decision. Rationale: if it's pouring and the user is clearly not going to detour into a store, a "walk 3 minutes to Starbucks" banner in the car is noise. The user doesn't feel the logic; they feel that the app simply respected the moment.

// Services/WeatherBiasService.swift (extract)
@available(iOS 16.0, *)
private func classify(current: CurrentWeather) -> WeatherBias {
    let intensity = current.precipitationIntensity.value
    return intensity >= Self.heavyPrecipitationMMPerHour ? .suppress : .neutral
}

The service caches for 30 minutes because WeatherKit is metered (500k free calls/month, $0.50 per 1k after) and the conductor routes many events per day. The setting is off by default; users who opt in the Settings app get the quieter behavior. On phones without WeatherKit, the bias returns .neutral and nothing changes.

The ambient idea "The weather is around you anyway. The app should at least know what you know."

7. How they compose: one brain, four senses

None of these frameworks know about each other. Foundation Models doesn't care about weather; WeatherKit doesn't care about photos. What ties them together is the application layer: CardCue Pro treats each framework as a sensor and the Conductor as the brain that aggregates their readings.

Each sensor has a graceful-degradation path. If one is unavailable, the app still works; it just gets a little less specific in the way it helps. That's what "on-device AI" looks like when it's built to last longer than a single keynote cycle: the intelligence is everywhere, and yet the critical path never depends on it.

8. What the user actually feels

• A scanned card arrives with a category already filled in, named by the keyword map today, and by Foundation Models once that categorizer ships.
• A scanner that simply says "can't read this" when it was handed something it shouldn't have been analyzing, without telling them whether it was a cat photo or a driver's license.
• A quiet Thursday when the lock-screen banner for a coffee card doesn't fire because WeatherKit knows it's raining half an inch per hour and the user's phone charitably agreed that this was not the moment.
• An offline flight where the wallet still scans, still categorizes (via the keyword map), and still refuses to send anything.

The four features feel like one feature: "the app notices things". The architectural story, four frameworks, on-device first with a secure cloud read by default, graceful fallbacks, is the reason it can feel that way at all.

9. Appendix: frameworks and versions

All five paths are feature-gated behind #available and canImport checks. A user on iOS 15 gets a wallet; they don't get a crash. That's the whole bar.