Federated analytics

On this page 7

Federated analytics is a technique for computing aggregate statistics over data that stays on user devices: a server sends a query to phones, PCs or embedded systems, each device computes a small local result, and only those results, usually protected by secure aggregation and differential privacy, are combined into a population-level answer. In Edge AI it is the measurement counterpart of federated learning, because it answers questions about how models and features behave in the field without collecting the underlying records.

The approach matters wherever raw data is too sensitive, too large or too regulated to upload. Typical questions include how often a keyboard suggestion is accepted, which songs are recognized most often, or how accurate an on-device model is across different hardware, all answered while individual records never leave the device.

How a federated analytics query runs

A federated analytics query runs as a coordinated round between a server and a sample of devices, in which computation moves to the data instead of data moving to the computation. The server defines the query, devices that meet eligibility rules execute it locally, and an aggregation layer releases only the combined result.

Eligibility rules usually require that a device is idle, charging and on an unmetered network, so the query does not drain the battery or consume mobile data. Each participating device reads its local log or cache, computes a compact summary such as a count, a histogram bucket or a sum, and encrypts or masks that summary before upload.

The following table shows the main stages of a typical round and what each stage protects.

StageWhat happensPrivacy role
Query definitionAnalyst specifies a metric, bucket scheme and minimum cohortLimits what can be asked
Device selectionServer samples eligible devices from a large populationHides who participated
Local computationDevice computes a count, sum or sketch on its own dataRaw records stay local
Masking or noiseDevice adds secret-shared masks or local noiseHides single contributions
Secure aggregationServer combines masked values into one totalServer sees only the sum
ReleaseResult is published after thresholds and central noiseLimits re-identification
Stages of a federated analytics round

A key design choice is that the server should be able to decode only the aggregate. With secure aggregation protocols, individual uploads are cryptographically masked so that the masks cancel out only when enough devices contribute, which also makes the system tolerant to devices that drop out mid-round.

Privacy mechanisms and their guarantees

Federated analytics is only as private as the mechanisms layered on top of decentralization, because a simple per-device answer can still reveal sensitive facts. Keeping data on the device reduces exposure, but a careful system adds formal protections against both the server and other analysts.

The table below compares the mechanisms most often combined in production deployments.

MechanismWhat it protects againstMain cost
Secure aggregationServer inspecting individual uploadsExtra rounds and bandwidth
Local differential privacyAny observer of a single uploadHigh noise, needs large cohorts
Central differential privacyInference from the published resultRequires a trusted aggregator
Distributed differential privacyBoth server and output observersProtocol complexity
Minimum cohort thresholdsAnswers about tiny groupsRare events go unreported
Trusted execution environmentsOperator access to plaintext sumsHardware trust assumptions
Common privacy mechanisms in federated analytics

The strength of differential privacy is expressed by the privacy budget epsilon. Smaller values give stronger protection and noisier answers; production systems often report per-query epsilon values in the low single digits and track cumulative spending, although the right setting depends on the threat model and is still debated among practitioners.

Federated analytics differs from federated learning in its output: statistics rather than model weights. Both rely on the same device orchestration, but analytics typically needs only one lightweight round per query, while training requires many rounds of gradient exchange.

ApproachOutputData locationTypical use
Federated analyticsCounts, histograms, quantilesStays on devicesMeasuring usage and model quality
Cross-device federated learningUpdated global modelStays on devicesTraining keyboard or speech models
Centralized analyticsAny statisticUploaded to a warehouseBusiness intelligence on server logs
Split learningPartially trained modelSplit between client and serverCollaborative training with few parties
On-device personalizationLocally adapted modelStays on one devicePer-user tuning without sharing
Federated analytics and neighboring techniques

In practice, the two federated techniques are complementary parts of one pipeline: federated analytics finds which features or failure cases matter, federated averaging trains the model, and federated analytics again measures the deployed model across the fleet.

Typical queries and algorithms

Federated analytics supports a restricted but useful set of query types, chosen because they can be decomposed into per-device contributions that add up correctly. Arbitrary joins or free-form SQL are generally not available, which is a deliberate safety constraint.

  • Counts and sums, such as how many devices triggered a wake word falsely in a day.
  • Histograms over a predefined bucket scheme, for example inference latency per chipset.
  • Heavy hitters discovery, finding frequent items such as popular new words without a known list, often with trie-based interactive algorithms.
  • Quantiles and means for continuous metrics such as battery cost of a model.
  • Model evaluation metrics, such as accuracy against labels that exist only on the device.

Heavy hitters discovery illustrates the craft involved. Because the vocabulary is unknown, the server grows a prefix tree round by round, asking devices whether their items extend popular prefixes, so that only strings shared by many users ever become visible.

Design decisions and common pitfalls

Successful deployments depend on planning queries around statistical and privacy limits instead of treating the fleet like a database. The table below lists frequent problems, their causes and practical remedies.

ProblemCauseRemedy
Noisy or useless resultsCohort too small for the noise levelIncrease sample size or coarsen buckets
Biased estimatesOnly idle, charging, Wi-Fi devices participateReweight or report participation bias
Privacy budget exhaustedMany overlapping queries on the same usersCentral budget accounting and query review
Slow answersRounds wait for eligible devicesSchedule queries over hours or days
Inconsistent device dataDifferent app versions and log formatsVersion the query and local schema
Pitfalls in federated analytics deployments

The most underestimated issue is selection bias. Devices that are often idle on Wi-Fi tend to be newer, wealthier or used in particular ways, so results describe participating devices, not all users, unless the analysis corrects for it.

In everyday language

Federated analytics is a way to learn facts about a large group without collecting anyone’s personal data. Instead of asking every phone to send in its records, the company sends a small question to the phones. Each phone works out its own tiny answer and sends back only that number.

It is a bit like a secret ballot. Everyone drops a sealed vote in a box, and only the final tally is announced. Nobody sees an individual vote, yet everyone learns the overall result. Extra random noise is often added so that even the tally cannot reveal one person’s choice.

Seen in the real world

A team building a smartphone keyboard wants to know whether its new on-device next-word model performs equally well on older and newer phones. Typed text is far too sensitive to upload, so the team defines a federated analytics query: each phone records how often a suggestion was accepted over seven days and reports one bucketed acceptance rate together with its chipset class.

Over 48 hours, about 200,000 eligible phones answer while charging on Wi-Fi. Secure aggregation combines the masked values, and central noise is added before release, with groups smaller than 5,000 devices suppressed. The result shows that acceptance on older midrange chips is roughly four percentage points lower, traced to a latency timeout that dropped suggestions. The team shortens the model’s context window for those devices and confirms the fix with a second query, without reading a single typed word.