Semantic Kernel

On this page 8

Semantic Kernel is an open-source software development kit from Microsoft that lets developers combine large language models with conventional application code in C#, Python and Java. Among AI frameworks & tools, it occupies the orchestration layer: it does not train models, but it decides which model to call, which functions the model may invoke and how results flow back into the program.

The project is released under the MIT license and was designed for enterprise integration, with dependency injection, telemetry and strict typing as first-class concerns. In artificial intelligence projects it is often compared with LangChain, because both frameworks solve the same problem of turning a model call into a reliable, testable software component.

How Semantic Kernel orchestrates model calls

Semantic Kernel works by placing a central object, the kernel, between the application and every AI service it uses. The kernel acts as a dependency injection container: it holds registered model connectors, plugins, filters and logging services, and every function invocation passes through it.

When the application sends a request, the kernel renders a prompt template, attaches the descriptions of available functions and forwards everything to a chat completion service. If the model responds with a tool call, the kernel executes the matching function, returns its result to the model and repeats the loop until the model produces a final answer. This cycle is called automatic function calling.

The following table lists the main building blocks and the responsibility of each one.

ComponentResponsibilityTypical example
KernelHolds services and plugins, runs functionsOne kernel per request scope
AI service connectorWraps a model provider APIAzure OpenAI, OpenAI, Ollama
PluginGroups related functions for the modelCalendar or order lookup plugin
Prompt templateDefines a reusable prompt with variablesSummarize ticket text
FilterIntercepts rendering and invocationBlock a function without approval
Vector store connectorStores and searches embeddingsAzure AI Search, Qdrant, Redis
Core components of Semantic Kernel

Plugins, functions and prompt templates

A plugin in Semantic Kernel is a named collection of functions that the model can discover and call. Each function carries a name, a natural-language description and typed parameters, and the kernel converts this metadata into a JSON schema that the model reads during function calling.

There are two kinds of functions. Native functions are ordinary methods in C#, Python or Java, marked with the KernelFunction attribute or decorator, which run deterministic code such as a database query. Prompt functions, historically called semantic functions, are prompt templates that call a model and return its text. Templates use the default syntax with {{$input}} variables or the Handlebars and Liquid formats.

The table below compares the two function types, because choosing the right one is the most frequent design decision.

PropertyNative functionPrompt function
ExecutionRuns application codeCalls a language model
OutputDeterministic and typedProbabilistic text
Cost per callNegligibleTokens billed by provider
Best useData access, calculations, actionsSummaries, classification, drafting
TestingStandard unit testsEvaluation sets and review
Native and prompt functions compared

Plugins can also be imported from an OpenAPI specification or from a Model Context Protocol server, which lets an existing REST service or tool server become callable by the model without new wrapper code.

From planners to agents and processes

Semantic Kernel moved from dedicated planners to native function calling as models became better at selecting tools. Early versions shipped the Sequential, Stepwise and Handlebars planners, which asked the model to write a complete plan before execution; these are now deprecated because built-in tool calling is cheaper and more reliable.

The Agent Framework inside Semantic Kernel adds agent abstractions such as ChatCompletionAgent and agents backed by hosted assistant services, plus group chat orchestration for several cooperating agents. The separate Process Framework models long-running business workflows as steps and events. In October 2025 Microsoft introduced the Microsoft Agent Framework, which unifies ideas from Semantic Kernel and AutoGen and is positioned as the recommended path for new agent projects, while Semantic Kernel itself remains supported.

Semantic Kernel differs from neighboring frameworks mainly in its language coverage and its enterprise orientation. LangChain and LlamaIndex grew from the Python data science community, whereas Semantic Kernel treats .NET as a primary platform and follows its conventions for configuration and logging.

The next table summarizes the practical differences that usually decide the choice.

FrameworkPrimary languagesMain strengthTypical fit
Semantic KernelC#, Python, JavaEnterprise integration and typing.NET and Azure applications
LangChainPython, JavaScriptLarge integration catalogRapid prototyping
LlamaIndexPython, TypeScriptDocument indexing and retrievalRetrieval-heavy assistants
AutoGenPython, .NETMulti-agent conversation researchExperimental agent systems
DSPyPythonProgrammatic prompt optimizationMeasured pipeline tuning
Semantic Kernel and comparable orchestration frameworks

Configuring Semantic Kernel for production

A production deployment of Semantic Kernel depends on filters, telemetry and careful function exposure more than on the prompt itself. Filters run before and after prompt rendering and function invocation, so they are the natural place for human approval, input validation and redaction of personal data.

The kernel emits OpenTelemetry traces, metrics and logs, which record token usage, latency and each function call. Teams should expose only the functions a scenario needs, because every additional function description consumes context tokens and widens the attack surface.

The decision table below maps common situations to recommended settings.

SituationRecommended choiceReason
Function changes dataApproval filter before invocationPrevents unintended actions
Many plugins availableExpose a small, relevant subsetImproves tool selection accuracy
Several model providersRegister services with service IDsAllows routing per function
Need for audit trailEnable OpenTelemetry exportRecords prompts and tool calls
Grounded answers requiredAdd a vector store search functionSupplies current internal data
Recommended configuration choices

Limitations and common mistakes

Semantic Kernel reduces integration work but does not remove the inherent unreliability of model output. Most failures come from design choices rather than from the framework itself.

  • Vague function descriptions cause the model to choose the wrong function or to skip a needed one.
  • Unrestricted automatic function calling on write operations can trigger irreversible actions without review.
  • Fast API evolution means older samples using planners or memory classes no longer match current packages.
  • Feature parity differs between the C#, Python and Java versions, with Java usually behind.
  • Prompt injection through retrieved documents or tool results can steer the model unless outputs are validated.

The simple explanation

Semantic Kernel is a toolkit that connects software and AI: software can talk to the model, and the model can ask the software for help. The program tells the model which actions are available, such as checking an order or reading a calendar. When the model needs one of those actions, Semantic Kernel runs it and passes the result back.

Think of it as a switchboard operator between the model and your code. The model is good with language. The code is good with facts and actions. The switchboard sends each request to the right place. It also keeps a record of every call.

How it is used in practice

A logistics company runs its customer portal on .NET and wants an assistant that answers shipment questions. The development team registers an Azure OpenAI chat model in a Semantic Kernel kernel and writes three native functions: one reads shipment status from the tracking database, one calculates a revised delivery date and one creates a support ticket.

An approval filter blocks ticket creation until the customer confirms it in the chat window, and OpenTelemetry traces show each function call with its latency. During a four-week pilot, the assistant resolved most routine status questions without staff involvement, while every ticket it created had an explicit customer confirmation recorded in the audit log.