Polling is an integration technique in which one system repeatedly asks another system whether new or changed data exists, at a fixed or adaptive interval, and processes whatever it finds. In integration and automation, polling is the pull-based counterpart of webhooks: instead of waiting to be notified, the automation checks a source on its own schedule.
Polling is the default way to start automated workflows when the source system offers an API but no event notifications, which is still common in ERP systems, older SaaS products, databases, mailboxes and file servers. Its central trade-off is simple: shorter intervals reduce latency but multiply requests, most of which return nothing new.
How polling works in an integration
A polling integration works as a loop of request, comparison and processing that repeats at a set interval. Each cycle has to answer one question reliably: which records have appeared or changed since the last successful check?
- The poller wakes up at the configured interval, usually driven by a scheduler.
- It sends a request with a change marker, such as a timestamp, an ID or a cursor saved from the previous cycle.
- The source returns only matching records, or an empty result if nothing changed.
- The poller deduplicates and processes each new record, typically by starting one workflow run per item.
- It stores the new change marker, often called a high-water mark, only after processing succeeded.
The change marker is the heart of a correct poller. The table below compares the most common strategies for detecting what is new.
| Strategy | How it detects changes | Strength | Weakness |
|---|---|---|---|
| Timestamp filter | Requests records modified after the last check | Simple, widely supported | Clock skew and equal timestamps can skip records |
| Incrementing ID | Requests records with an ID above the last seen | Exact for new records | Misses updates to existing records |
| Cursor or sync token | Sends an opaque token issued by the source | Most reliable, handles updates and deletes | Only available if the API provides it |
| Full comparison | Fetches everything and compares with a stored copy | Works with any source | Expensive in requests and memory |
| Conditional request | Uses ETag or Last-Modified headers | Unchanged data returns 304 with no body | Says whether, not what, changed |
To avoid losing records at the boundary, robust pollers apply a small overlap window, for example re-reading the last 60 seconds, and then deduplicate by record ID. Accepting a few duplicate reads is far cheaper than silently missing an order or a ticket.
Polling interval, latency and request volume
The polling interval determines both how quickly changes are detected and how many requests the integration consumes. With a fixed interval, the worst-case detection delay equals the interval, and the average delay is about half the interval, assuming changes arrive at random times.
The following table shows how the interval translates into requests per day for a single polled endpoint.
| Interval | Requests per day | Average delay | Maximum delay |
|---|---|---|---|
| 30 seconds | 2,880 | 15 seconds | 30 seconds |
| 1 minute | 1,440 | 30 seconds | 1 minute |
| 5 minutes | 288 | 2.5 minutes | 5 minutes |
| 15 minutes | 96 | 7.5 minutes | 15 minutes |
| 1 hour | 24 | 30 minutes | 1 hour |
The numbers reveal the main inefficiency of polling. If a source produces 20 changes a day and is polled every minute, at least 1,420 of the 1,440 daily requests return nothing, so about 98.6 percent of the traffic is wasted. Multiplied by dozens of endpoints and customers, this is why API providers impose rate limits and why automation platforms often enforce minimum polling intervals on lower pricing tiers.
Variants reduce this cost. Long polling keeps the request open until data arrives or a timeout expires, which cuts empty responses and latency at the price of open connections. Adaptive polling shortens the interval when changes are frequent and lengthens it when the source is quiet.
Polling compared with webhooks and streaming
Polling differs from webhooks and streaming in who initiates the communication: the consumer pulls in polling, whereas the source pushes in webhooks and streaming. A scheduled trigger is related but distinct, because it starts a workflow at fixed times regardless of whether anything changed.
| Pattern | Direction | Latency | Best use |
|---|---|---|---|
| Polling | Consumer pulls at intervals | Up to one interval | Sources without events, firewalled consumers |
| Long polling | Consumer pulls, source holds the request | Near real time | Simple real-time updates over plain HTTP |
| Webhook | Source pushes an HTTP call | Seconds | Event-driven workflows between SaaS apps |
| Streaming | Persistent connection with pushed events | Sub-second | High-volume event feeds and live data |
| Scheduled trigger | Timer starts a job | Fixed by schedule | Reports, batch jobs and reconciliations |
Polling has real advantages that keep it in use. The consumer needs no public endpoint, so it works behind firewalls; it recovers naturally after downtime because the next cycle reads everything since the stored marker; and it does not depend on the source delivering notifications reliably. Many mature integrations therefore combine webhooks for speed with periodic polling as a safety net to catch missed events.
Common polling mistakes and how to avoid them
Most polling failures come from incorrect change markers, missing deduplication and careless request timing. They rarely produce errors; instead they cause missing or duplicate records that surface days later.
- Saving the marker before processing loses records when a run fails midway; save it only after all items succeeded.
- Using the poller’s own clock instead of the source’s timestamps creates gaps whenever the two clocks differ.
- Ignoring pagination processes only the first page when more changes arrived than one response returns.
- Polling in lockstep, where many clients fire at exactly the full minute, causes load spikes; adding random jitter spreads requests.
- Retrying immediately after HTTP 429 worsens rate limiting; honor the Retry-After header and use exponential backoff.
The short version
Polling is like checking your mailbox every few minutes instead of having the mail carrier ring the doorbell. Each time you look, you either find new letters or an empty box, and you handle whatever is there.
Checking often means you see new mail sooner but walk to the mailbox far more times for nothing. Checking rarely saves effort but leaves letters waiting. Polling works with almost any system, which is why it remains common even though notifications are more efficient.
Where it makes a difference
A manufacturing company uses an older on-premises ERP system that has a reporting API but cannot send webhooks. The sales team wants every new production order to appear in its CRM so that account managers can inform customers.
The integration polls the ERP every 2 minutes, which means 720 requests a day and an average delay of about one minute. Each request asks for orders whose modification time is later than the stored high-water mark minus a 60-second overlap, follows all result pages and skips order numbers already processed. The new marker is saved only after every order has been written to the CRM.
In the first week, the ERP was restarted twice for maintenance. Because the poller resumed from its stored marker, no orders were lost and none were duplicated, and customers received confirmation within minutes instead of the next business day.