Jenkins

On this page 7

Jenkins is an open-source automation server that runs build, test and deployment jobs, most often defined as pipelines as code in a Jenkinsfile stored with the application source. In DevOps, it is one of the longest-established engines for continuous integration and continuous delivery, and its large plugin ecosystem lets it connect to almost any version control system, build tool, artifact repository or Cloud platform.

Jenkins began in 2004 as the Hudson project and took its current name in 2011. It is written in Java, self-hosted by the organization that uses it, and governed today under the Continuous Delivery Foundation. Its main strength is flexibility; its main cost is the operational effort of running and securing it.

How Jenkins runs pipelines

Jenkins runs pipelines through a controller and agent architecture. The controller stores configuration, schedules work, serves the web interface and records results, while agents are separate machines or containers that actually execute the build steps. Each agent offers one or more executors, and each executor runs one job at a time.

A run is usually triggered by a webhook from the source repository, a schedule or an upstream job. The controller reads the Jenkinsfile, allocates an agent that matches the requested label, checks out the code into a workspace, and executes the stages in order, archiving artifacts, test reports and logs at the end. The table below names the core building blocks.

ComponentRoleTypical scale
ControllerScheduling, configuration, user interface, historyOne per instance, sometimes several per organization
AgentExecutes build and test stepsFrom a few to hundreds
ExecutorOne concurrent job slot on an agentOften one to four per agent
WorkspaceWorking directory with the checked-out codeOne per job per agent
PluginAdds integrations, steps and UI featuresDozens per typical instance
Core components of a Jenkins installation

Running builds on the controller itself is strongly discouraged, because build scripts would then have access to the controller’s files and credentials. Best practice sets the controller’s executor count to zero and sends all work to agents.

Declarative and scripted pipelines

Jenkins pipelines come in two syntaxes, both written in a Groovy-based domain-specific language. Declarative pipeline uses a fixed structure of pipeline, agent, stages and post blocks and is the recommended default; scripted pipeline exposes full Groovy control flow for unusual cases.

Job typeDefinitionStrengthWeakness
Declarative pipelineStructured JenkinsfileReadable, validated, easy to reviewLess flexible for complex logic
Scripted pipelineFree-form Groovy JenkinsfileFull programming controlHarder to read and maintain
Multibranch pipelineDiscovers branches and pull requestsAutomatic per-branch buildsMany jobs to manage at scale
Freestyle jobConfigured in the web interfaceQuick for simple tasksNot versioned with the code
Pipeline syntaxes and older job types

Shared libraries let organizations keep reusable pipeline steps in a separate repository, so dozens of Jenkinsfile definitions can call a common buildAndDeploy() function. This is how large teams enforce consistent build pipeline standards without copying code between projects.

Jenkins compared with hosted CI services

Jenkins differs from services such as GitHub Actions and GitLab CI/CD mainly in who operates the platform. With Jenkins, the organization runs the controller, agents, upgrades and backups; with integrated services, the provider runs most of that and the pipeline lives inside the code hosting product.

AspectJenkinsGitHub ActionsGitLab CI/CD
HostingSelf-hostedHosted or self-hosted runnersHosted or self-managed
Pipeline formatGroovy JenkinsfileYAML workflowsYAML .gitlab-ci.yml
ExtensibilityVery large plugin catalogMarketplace actionsBuilt-in features and templates
Source control couplingWorks with any systemTied to GitHubTied to GitLab
Operational burdenHighLow to moderateLow to moderate
Jenkins and two integrated CI services

Teams keep choosing Jenkins when they need on-premises builds, unusual hardware such as embedded test rigs, complex cross-repository orchestration, or independence from any single code hosting vendor. Teams starting fresh in the cloud often prefer an integrated service to avoid platform maintenance.

Scaling and operating Jenkins

Scaling Jenkins means scaling agents, not the controller. Modern installations use ephemeral agents: the Kubernetes plugin starts a Pod for each build and removes it afterwards, and cloud plugins do the same with virtual machines. Clean agents give reproducible builds and remove leftover workspace state.

  • Configuration as Code (the JCasC plugin) defines the controller’s settings in YAML, so an instance can be rebuilt from version control.
  • Backups of the Jenkins home directory protect job history, credentials and configuration.
  • Splitting very large instances into several controllers by department limits the impact of a single failure.
  • Pinning plugin versions and testing upgrades on a staging environment avoids broken pipelines after updates.

The controller is a single point of failure in the open-source edition. Restarting it interrupts running builds, although durable pipelines can resume some steps, so maintenance windows and fast restore procedures matter.

Security risks and common mistakes

A Jenkins controller is a high-value target because it holds credentials for repositories, registries and production environments. The table lists frequent weaknesses and their remedies.

ProblemCauseFix
Exposed controllerWeb interface reachable from the internetRestrict access behind a VPN or single sign-on
Outdated pluginsUpgrades postponedRegular patch cycle and security advisories review
Secrets in logsCredentials echoed by scriptsUse the Credentials plugin with masking
Builds on the controllerDefault executors left enabledSet controller executors to zero
Plugin sprawlPlugins added without reviewApproved plugin list and periodic cleanup
Frequent Jenkins problems and fixes

Pull requests from forks deserve special caution, since a malicious Jenkinsfile could read credentials. Trusted-branch rules and approval steps prevent untrusted code from running with privileged credentials.

Put simply

Jenkins is like a tireless factory supervisor for software. Every time a developer saves a change, Jenkins picks it up, builds the program, runs the tests and, if everything passes, can ship it to servers.

The written recipe it follows lives next to the code, so everyone can see and review the steps. The catch is that the company must look after the supervisor itself: keep it updated, protected and backed up.

In practice

A manufacturer of medical devices builds firmware that must be tested on physical hardware in its own lab, which rules out purely hosted build services. Its platform team runs one Jenkins controller in the company data center and about 25 agents, some of them connected to test benches with real circuit boards.

Each pull request triggers a multibranch pipeline: compile the firmware in a container agent, run unit tests, flash the image onto a bench device and run a 40-minute hardware test suite. Release builds add signing with a key stored in the Credentials plugin. Build time per change fell from a manual half day to under an hour, and every release now has a complete, auditable record of the tests it passed.