Code-switching is the alternation between two or more languages or language varieties within a single conversation, turn or sentence, as when a speaker in Mumbai mixes Hindi and English in one utterance. In Speech & language AI, the term names both the linguistic phenomenon and the engineering problem it creates: models trained mostly on monolingual data must recognize, understand and generate text or speech in which the language changes without warning.
Code-switching is common rather than exotic. Hundreds of millions of bilingual speakers switch daily, in Spanish-English communities in the United States, in Arabic-French usage across North Africa, in Mandarin-English speech in Singapore and Malaysia, and in many Indian and African language pairs. Systems for speech-to-text, translation, sentiment analysis and chatbots that ignore it fail precisely for these users.
How code-switching works in language and speech
Code-switching follows structural and social regularities rather than random mixing. Linguists distinguish intersentential switching, where the language changes at a sentence boundary, from intrasentential switching, where it changes inside a sentence, and from tag switching, where a short tag or filler from another language is inserted.
Related phenomena are often grouped under the same label. Code-mixing usually refers to dense switching inside words or phrases, and borrowing refers to foreign words that have been absorbed into the host language. The boundary between a borrowing and a genuine switch is disputed and gradual, which matters when annotators label training data.
The following table summarizes the main switching types and the difficulty each creates for automatic processing.
| Type | Where the switch happens | Example pattern | Difficulty for models |
|---|---|---|---|
| Intersentential | Between sentences | One sentence in Spanish, next in English | Moderate |
| Intrasentential | Inside a sentence or clause | English noun phrase inside a Hindi clause | High |
| Tag switching | Fillers and discourse markers | Adding a foreign tag question | Low to moderate |
| Intraword mixing | Inside a single word | English stem with a native suffix | Very high |
In speech, switching also changes phonetics and prosody. Speakers often pronounce embedded words with the accent of their dominant language, so an English word inside Cantonese speech may not match the acoustic patterns an English-trained acoustic model expects.
Why code-switching is hard for AI models
Code-switching is hard mainly because of data scarcity. Large text and speech corpora are collected and filtered per language, and language filters often discard mixed content as noise. Code-switched data therefore remains a low-resource setting even when each individual language is well resourced.
Further obstacles come from script and spelling variation. Hindi-English is frequently written in romanized Hindi without standard spelling, Arabic-French may combine Arabic script with Latin letters and digits, and tokenizers trained on clean monolingual text fragment such words into many subword pieces.
The table below maps the main technical challenges to the components they affect.
| Challenge | Affected component | Typical effect |
|---|---|---|
| Sparse mixed training data | All models | Poor generalization to real users |
| Accent transfer on embedded words | Acoustic model | Substitutions and deletions in transcripts |
| Romanized or nonstandard spelling | Tokenizer and text models | Long token sequences and lost meaning |
| Monolingual language model priors | Speech-to-text decoder | Embedded words forced into the wrong language |
| Switch-point ambiguity | Language identification | Wrong language label per token |
A frequent failure in speech recognition is language collapse: the decoder favors the dominant language and transcribes an embedded English phrase as phonetically similar words in the matrix language. Evaluation must account for this, because a single word error rate figure can hide poor accuracy on the switched segments.
Code-switching compared with related tasks
Code-switching is a property of the input, not a task in itself, but several neighboring tasks in Speech & language AI depend on it. The comparison below clarifies where the concepts overlap and where they differ.
| Concept | Main question | Relation to code-switching |
|---|---|---|
| Language identification | Which language is this? | Must work per token or per segment, not per document |
| Translation | What does this mean in another language? | Mixed source text needs normalization first |
| Low-resource language | How to model languages with little data? | Mixed pairs behave like a low-resource language |
| Text normalization | How to map raw text to a standard form? | Must handle transliteration and mixed scripts |
| Multilingual model | Can one model serve many languages? | Necessary but not sufficient for mixed input |
A common misconception is that a multilingual model automatically handles code-switching. Multilingual training teaches each language separately; without mixed examples, such a model may still expect one language per utterance and degrade sharply at switch points.
Methods for handling code-switched input
Practitioners combine several methods, and the right mix depends on the language pair, the available data and the latency budget. For speech, end-to-end multilingual models with a shared vocabulary across languages now outperform pipelines that run a separate recognizer for each language.
- Data augmentation: synthesizing mixed sentences by replacing phrases with aligned translations, constrained by linguistic theories such as the equivalence constraint or the matrix language frame model.
- Token-level language identification: tagging each word with its language so that downstream components can apply the right rules.
- Mixed-language subword vocabularies: building tokenizers on code-switched text so that romanized words are not split excessively.
- Fine-tuning on in-domain mixed data: even a few hours of transcribed mixed speech often yields large gains.
- Transliteration and normalization: converting romanized text to native script, or the reverse, before analysis.
The decision table below shows typical starting points for common situations.
| Situation | Recommended approach | Reason |
|---|---|---|
| Speech assistant for a bilingual market | Fine-tune a multilingual speech model | Shared vocabulary handles switch points |
| Social media sentiment analysis | Normalize then classify | Spelling variation dominates errors |
| Little or no mixed data | Synthetic augmentation | Creates switch patterns cheaply |
| Routing to language-specific services | Token-level language identification | Segments are labeled before routing |
Evaluating code-switching performance
Reliable evaluation requires test sets with genuine switching and metrics that expose errors at switch points. Researchers measure switching density with the code-mixing index, which estimates how evenly the languages are mixed, and report error rates separately for each language and for switched segments.
For speech recognition, mixed error rate combines a word error rate for alphabetic languages with a character error rate for languages such as Mandarin, where word boundaries are not marked. Without such metrics, a model may look accurate overall while systematically mangling the embedded language.
Limitations and common mistakes
The most damaging mistake is filtering mixed content out of training data as noise, which removes exactly the patterns real users produce. Other frequent errors are treating romanized text as a separate language and assuming one language per document in analytics pipelines.
There are also social and fairness implications. Code-switching is often stigmatized, and systems that fail on it effectively serve bilingual speakers worse. Teams should also avoid forcing users to choose a single language, because switching is often unconscious and context dependent.
In plain words
Many people who speak two languages move between them naturally, sometimes in the middle of a sentence. A person might start a sentence in Spanish, add an English phrase and finish in Spanish again. This is code-switching, and it is completely normal.
Computers find it difficult because most of them learned each language separately, like a student who studied two textbooks but never heard a real bilingual conversation. The key point is simple: AI understands mixed speech only if it has learned from mixed speech.
A real-world example
A bank in Southeast Asia runs a voice assistant for customer calls. Many callers speak Malay with English banking terms such as account balance, transfer limit and credit card, often inside the same sentence. The first version used a Malay recognizer and transcribed the English terms as unrelated Malay words, so intent recognition failed on about one call in four.
The team collected around 50 hours of transcribed mixed calls, added synthetic mixed sentences built from banking phrases and fine-tuned a multilingual speech model with a shared vocabulary. They tracked error rates separately on switched segments. Errors on English banking terms fell sharply, and the share of calls routed correctly without a human agent rose noticeably, which shortened waiting times for customers.