Integration runtime

On this page 7

An integration runtime is the compute environment that executes integration flows: it runs the connectors, mappings, transformations and routing logic that a designer defines, and it supplies the network position, credentials and processing capacity those flows need. In automation and system integration, the runtime is the part that actually does the work, while the design tool, the flow definition and the management console only describe and supervise it.

The distinction matters because the same integration logic can behave very differently depending on where the runtime sits. A runtime hosted by a cloud provider reaches public SaaS APIs easily but cannot see a database behind a corporate firewall, whereas a self-hosted runtime installed inside the private network can. Choosing, sizing and securing the runtime is therefore an architectural decision, not a deployment detail.

How an integration runtime executes a flow

An integration runtime separates the control plane from the data plane. The control plane, usually a cloud management service, stores flow definitions, schedules, credentials references and monitoring data. The data plane is the runtime itself: it receives a deployed flow, pulls or receives the trigger, opens connections, processes records and reports status back.

A typical execution cycle follows a fixed sequence. The runtime registers with the control plane through an outbound, encrypted channel, so no inbound firewall port is usually required. When a trigger fires, whether a schedule, a webhook, a polling interval or a message on a queue, the runtime loads the flow version, resolves secrets from a vault, executes each step and writes execution logs and metrics that operators read in the console.

The following table summarizes the main components found in most runtime implementations and what each one contributes.

ComponentRole in executionTypical failure symptom
Execution engineInterprets the flow and runs steps in orderFlows stall or exceed time limits
Connector librarySpeaks each target protocol or APIAuthentication or schema errors
Agent or gatewayMaintains the channel to the control planeRuntime shown as offline
Secret resolverFetches credentials at run timeExpired or missing credentials
Buffer and retry storeHolds messages during outagesDuplicates or lost records
Telemetry moduleEmits logs, metrics and tracesBlind spots during incidents
Core components of an integration runtime

Types of integration runtime by hosting model

Integration platforms usually offer three hosting models, and many organizations run more than one at the same time. The choice depends on where the data lives, which compliance rules apply and who is expected to operate the infrastructure.

Hosting modelWhere it runsBest suited forMain trade-off
Cloud-managed runtimeVendor infrastructureSaaS-to-SaaS flows and public APIsNo direct reach into private networks
Self-hosted runtimeCustomer servers or virtual machinesOn-premises databases and file sharesCustomer patches, scales and monitors it
Hybrid runtimeCloud control plane with local workersMixed estates with sensitive dataTwo operational models to manage
Containerized runtimeKubernetes or a container serviceElastic workloads and DevOps teamsRequires container platform skills
Hosting models compared

Some platforms add specialized runtimes for particular workloads, for example a runtime dedicated to executing legacy SQL Server Integration Services packages, or one optimized for large data movement with parallel copy. These variants share the same principle: the runtime provides compute and network position, and the flow definition remains portable across them as far as the connectors allow.

The integration runtime is often confused with the platform, the connector or the middleware pattern it executes. The runtime is infrastructure, not architecture: an enterprise service bus, an iPaaS product or a point-to-point integration each need some runtime to execute, but none of them is identical to it.

ConceptWhat it isRelationship to the runtime
iPaaSA cloud integration platform delivered as a serviceProvides managed runtimes plus design and monitoring tools
Integration connectorA packaged adapter for one systemLoaded and executed by the runtime
Enterprise service busA centralized mediation architectureRuns on its own runtime nodes
Message queueDurable storage for asynchronous messagesFeeds triggers to the runtime or buffers its output
API orchestrationCoordinating several API calls into one processA workload the runtime executes
How the runtime relates to neighboring terms

A useful mental test is to ask what would break if the runtime were stopped. Flow definitions, mappings and connector configurations would survive in the platform repository, but no data would move. That asymmetry is why runtime availability is usually the first metric that integration operations teams watch.

Choosing and sizing an integration runtime

Runtime selection starts with data location and network reachability, followed by volume, latency and compliance requirements. A design that ignores network position tends to fail late, during testing, when a cloud runtime cannot resolve an internal host name.

SituationRecommended runtimeReason
Both endpoints are public SaaSCloud-managedNo infrastructure to operate
Source database behind a firewallSelf-hosted near the sourceOutbound-only connectivity, no open ports
Data must stay in one jurisdictionRuntime pinned to that regionKeeps processing within residency rules
Large nightly batch transfersScaled-out or multi-node runtimeParallel copy shortens the batch window
Spiky event-driven trafficContainerized with autoscalingCapacity follows demand
Decision guide for runtime placement

Sizing depends on concurrency, payload size and transformation complexity rather than on record counts alone. A flow that parses large XML documents or performs lookups for every record consumes far more memory and CPU than a simple file copy of the same volume. Practitioners load-test with realistic payloads and plan for at least two nodes when the integration supports a business-critical process, so that patching or a node failure does not stop the flow.

Limitations, risks and common mistakes

Most runtime incidents come from operational neglect rather than design flaws. Self-hosted runtimes in particular are easy to install and easy to forget, which leaves outdated versions, expiring certificates and single points of failure in production.

  • Single-node deployment for a critical flow, so one reboot interrupts the business process.
  • Version drift between runtime and control plane, which disables newer connector features or blocks deployment.
  • Over-privileged service accounts that give the runtime broader database or file access than the flows need.
  • Unmonitored queues and retry stores, where failed messages accumulate silently until a backlog surfaces downstream.
  • Shared runtimes without isolation, where one heavy batch flow starves latency-sensitive real-time flows.

Mitigations are well established: run high-availability node groups, apply least-privilege credentials stored in a vault, separate batch and real-time workloads onto different runtimes, and alert on runtime heartbeat, queue depth and error rate. Treating the runtime as production infrastructure, with change management and capacity reviews, prevents most surprises.

In simple terms

An integration runtime is the engine that does the actual moving of data between systems. Someone draws up the instructions, such as take new orders from the shop and put them into the accounting system, but the instructions alone do nothing. The runtime is the worker that carries them out, every time they are needed.

Where that worker stands matters. A worker based in the cloud can reach other cloud services easily, but cannot walk into a locked office building. If the data is inside the company network, the worker has to be placed inside too, which is what a self-hosted runtime does.

A practical example

A regional distributor keeps its inventory in an on-premises SQL database and sells through a cloud commerce platform. The integration team designs a flow that sends stock levels to the storefront every 15 minutes. A cloud-managed runtime cannot reach the internal database, so the team installs a self-hosted runtime on two virtual machines inside the data center, both registered to the same cloud control plane through outbound HTTPS.

The two nodes share the load and cover for each other during monthly patching. A separate cloud-managed runtime handles order notifications between the storefront and the shipping provider, because both are public services. After go-live, stock discrepancies on the storefront fell from several per day to almost none, and no inbound firewall port had to be opened.