Code-switching

On this page 8

Code-switching is the alternation between two or more languages or language varieties within a single conversation, turn or sentence, as when a speaker in Mumbai mixes Hindi and English in one utterance. In Speech & language AI, the term names both the linguistic phenomenon and the engineering problem it creates: models trained mostly on monolingual data must recognize, understand and generate text or speech in which the language changes without warning.

Code-switching is common rather than exotic. Hundreds of millions of bilingual speakers switch daily, in Spanish-English communities in the United States, in Arabic-French usage across North Africa, in Mandarin-English speech in Singapore and Malaysia, and in many Indian and African language pairs. Systems for speech-to-text, translation, sentiment analysis and chatbots that ignore it fail precisely for these users.

How code-switching works in language and speech

Code-switching follows structural and social regularities rather than random mixing. Linguists distinguish intersentential switching, where the language changes at a sentence boundary, from intrasentential switching, where it changes inside a sentence, and from tag switching, where a short tag or filler from another language is inserted.

Related phenomena are often grouped under the same label. Code-mixing usually refers to dense switching inside words or phrases, and borrowing refers to foreign words that have been absorbed into the host language. The boundary between a borrowing and a genuine switch is disputed and gradual, which matters when annotators label training data.

The following table summarizes the main switching types and the difficulty each creates for automatic processing.

TypeWhere the switch happensExample patternDifficulty for models
IntersententialBetween sentencesOne sentence in Spanish, next in EnglishModerate
IntrasententialInside a sentence or clauseEnglish noun phrase inside a Hindi clauseHigh
Tag switchingFillers and discourse markersAdding a foreign tag questionLow to moderate
Intraword mixingInside a single wordEnglish stem with a native suffixVery high
Main types of code-switching

In speech, switching also changes phonetics and prosody. Speakers often pronounce embedded words with the accent of their dominant language, so an English word inside Cantonese speech may not match the acoustic patterns an English-trained acoustic model expects.

Why code-switching is hard for AI models

Code-switching is hard mainly because of data scarcity. Large text and speech corpora are collected and filtered per language, and language filters often discard mixed content as noise. Code-switched data therefore remains a low-resource setting even when each individual language is well resourced.

Further obstacles come from script and spelling variation. Hindi-English is frequently written in romanized Hindi without standard spelling, Arabic-French may combine Arabic script with Latin letters and digits, and tokenizers trained on clean monolingual text fragment such words into many subword pieces.

The table below maps the main technical challenges to the components they affect.

ChallengeAffected componentTypical effect
Sparse mixed training dataAll modelsPoor generalization to real users
Accent transfer on embedded wordsAcoustic modelSubstitutions and deletions in transcripts
Romanized or nonstandard spellingTokenizer and text modelsLong token sequences and lost meaning
Monolingual language model priorsSpeech-to-text decoderEmbedded words forced into the wrong language
Switch-point ambiguityLanguage identificationWrong language label per token
Technical challenges and affected components

A frequent failure in speech recognition is language collapse: the decoder favors the dominant language and transcribes an embedded English phrase as phonetically similar words in the matrix language. Evaluation must account for this, because a single word error rate figure can hide poor accuracy on the switched segments.

Code-switching is a property of the input, not a task in itself, but several neighboring tasks in Speech & language AI depend on it. The comparison below clarifies where the concepts overlap and where they differ.

ConceptMain questionRelation to code-switching
Language identificationWhich language is this?Must work per token or per segment, not per document
TranslationWhat does this mean in another language?Mixed source text needs normalization first
Low-resource languageHow to model languages with little data?Mixed pairs behave like a low-resource language
Text normalizationHow to map raw text to a standard form?Must handle transliteration and mixed scripts
Multilingual modelCan one model serve many languages?Necessary but not sufficient for mixed input
Code-switching and neighboring concepts

A common misconception is that a multilingual model automatically handles code-switching. Multilingual training teaches each language separately; without mixed examples, such a model may still expect one language per utterance and degrade sharply at switch points.

Methods for handling code-switched input

Practitioners combine several methods, and the right mix depends on the language pair, the available data and the latency budget. For speech, end-to-end multilingual models with a shared vocabulary across languages now outperform pipelines that run a separate recognizer for each language.

  • Data augmentation: synthesizing mixed sentences by replacing phrases with aligned translations, constrained by linguistic theories such as the equivalence constraint or the matrix language frame model.
  • Token-level language identification: tagging each word with its language so that downstream components can apply the right rules.
  • Mixed-language subword vocabularies: building tokenizers on code-switched text so that romanized words are not split excessively.
  • Fine-tuning on in-domain mixed data: even a few hours of transcribed mixed speech often yields large gains.
  • Transliteration and normalization: converting romanized text to native script, or the reverse, before analysis.

The decision table below shows typical starting points for common situations.

SituationRecommended approachReason
Speech assistant for a bilingual marketFine-tune a multilingual speech modelShared vocabulary handles switch points
Social media sentiment analysisNormalize then classifySpelling variation dominates errors
Little or no mixed dataSynthetic augmentationCreates switch patterns cheaply
Routing to language-specific servicesToken-level language identificationSegments are labeled before routing
Choosing an approach

Evaluating code-switching performance

Reliable evaluation requires test sets with genuine switching and metrics that expose errors at switch points. Researchers measure switching density with the code-mixing index, which estimates how evenly the languages are mixed, and report error rates separately for each language and for switched segments.

For speech recognition, mixed error rate combines a word error rate for alphabetic languages with a character error rate for languages such as Mandarin, where word boundaries are not marked. Without such metrics, a model may look accurate overall while systematically mangling the embedded language.

Limitations and common mistakes

The most damaging mistake is filtering mixed content out of training data as noise, which removes exactly the patterns real users produce. Other frequent errors are treating romanized text as a separate language and assuming one language per document in analytics pipelines.

There are also social and fairness implications. Code-switching is often stigmatized, and systems that fail on it effectively serve bilingual speakers worse. Teams should also avoid forcing users to choose a single language, because switching is often unconscious and context dependent.

In plain words

Many people who speak two languages move between them naturally, sometimes in the middle of a sentence. A person might start a sentence in Spanish, add an English phrase and finish in Spanish again. This is code-switching, and it is completely normal.

Computers find it difficult because most of them learned each language separately, like a student who studied two textbooks but never heard a real bilingual conversation. The key point is simple: AI understands mixed speech only if it has learned from mixed speech.

A real-world example

A bank in Southeast Asia runs a voice assistant for customer calls. Many callers speak Malay with English banking terms such as account balance, transfer limit and credit card, often inside the same sentence. The first version used a Malay recognizer and transcribed the English terms as unrelated Malay words, so intent recognition failed on about one call in four.

The team collected around 50 hours of transcribed mixed calls, added synthetic mixed sentences built from banking phrases and fine-tuned a multilingual speech model with a shared vocabulary. They tracked error rates separately on switched segments. Errors on English banking terms fell sharply, and the share of calls routed correctly without a human agent rose noticeably, which shortened waiting times for customers.