Data import

On this page 7

Data import is the process of reading data from an external source, such as a file, a database export or another application, and converting it into the internal structures of a target software system so that the data can be stored, searched and edited there. In interoperability, data import is the receiving half of every exchange between programs: it decides whether information created elsewhere arrives complete, correctly interpreted and usable.

A reliable import is much more than opening a file. It must recognize the file format, decode the character encoding, map source fields to target fields, validate values and handle conflicts with data that already exists.

How a data import pipeline works

A data import usually runs through a sequence of five stages: detection, parsing, mapping, validation and loading. Each stage can fail independently, which is why mature applications report errors per stage rather than with a single generic message.

StageWhat happensTypical failure
DetectionFormat and encoding are identifiedUTF-8 file read as Windows-1252
ParsingBytes become records and fieldsUnescaped delimiter splits a field
MappingSource fields matched to target fieldsPhone number lands in a name column
ValidationTypes, ranges and rules are checkedDate 31/02 accepted as valid
LoadingRecords written to storageDuplicate keys or partial writes
Stages of a typical import

Many systems add a preview or dry-run step between validation and loading. It shows the first rows as they will be interpreted, so a person can correct the mapping before any record is changed.

Common formats and what they preserve

The source format determines how much meaning survives an import. Simple tabular formats carry values but little structure, while richer formats can keep data types, nesting and relationships. Formats based on an open standard are generally easier to import correctly than undocumented proprietary formats.

FormatStructure keptMain import risk
CSVFlat rows and columns onlyDelimiter, quoting and encoding ambiguity
JSONNested objects and basic typesInconsistent schemas between records
XMLHierarchy, attributes, schema validationNamespaces and verbose edge cases
XLSXSheets, cell types, formulasDates and numbers stored as display text
SQL dumpTables, keys and constraintsDialect differences between databases
vCard or iCalendarContacts or events with fieldsVendor-specific extensions ignored
Import formats compared

CSV deserves particular caution. Although the format is described in RFC 4180, real files vary in separator (comma or semicolon), decimal mark, line endings and quoting. A file exported in one locale can therefore shift every column when imported in another.

Data import is often mentioned together with neighboring ideas that describe different parts of the same exchange. Separating them clarifies where a problem lies when data does not arrive as expected.

ConceptFocusRelation to data import
Data exportWriting data out of a systemProduces the file that import reads
Format conversionChanging one format into anotherOften a step inside the import
Data portabilityThe right and ability to move dataDepends on working export and import
Data migrationMoving a whole system’s data onceUses large, carefully tested imports
ETLExtract, transform, load at scaleAutomated, recurring form of import
Data import and neighboring terms

Import quality also shapes vendor lock-in. When a competing product cannot import a system’s data faithfully, users stay because leaving would mean losing history, not because the original product is better.

Designing and running a reliable import

A reliable import starts with an explicit field mapping and a known encoding, ideally UTF-8, rather than relying on automatic guesses. Practitioners run the import first against a copy or staging environment, compare record counts and spot-check values before touching production data.

  • Define a unique key for each record so that repeated imports update rather than duplicate.
  • Choose a conflict policy: skip, overwrite or merge when a record already exists.
  • Make imports idempotent, so running the same file twice yields the same result.
  • Wrap large loads in transactions or batches that can be rolled back on failure.
  • Keep an error report listing rejected rows and the reason for each rejection.

For recurring transfers, a scheduled import through an application programming interface (API) is usually more robust than manual file uploads, because the schema is versioned and errors are returned in a machine-readable form.

Typical import errors and how to fix them

Most failed imports trace back to a small set of recurring causes, and many of them corrupt data quietly instead of stopping with an error. The table below links visible symptoms to their usual cause and remedy.

SymptomLikely causeFix
Accented letters shown as odd symbolsEncoding mismatchDeclare UTF-8 on export and import
All data in a single columnWrong delimiterSet comma or semicolon explicitly
Leading zeros missing in codesType inference treats text as numberImport the column as text
Day and month swappedAmbiguous date formatUse ISO 8601 dates such as 2026-03-04
Records doubled after rerunNo unique keyMatch on a stable identifier
Symptoms, causes and fixes

The most dangerous outcome is a silent partial import, in which the process reports success while some rows were skipped or truncated. Comparing source and target record counts after every run is the simplest safeguard.

Understanding it simply

Data import means bringing information from somewhere else into a program you use. For example, you might move your contacts from an old phone app into a new one.

Think of it like moving into a new home. Your belongings arrive in boxes, and each item must go to the right room. If a box is labeled badly, the plates may end up in the bathroom. A good import puts every piece of data in the right place. It also tells you clearly if something did not fit.

Applied in practice

A small accounting firm replaces its old customer database with a new practice management application. The old system can only export a CSV file of about 3,800 clients, saved with semicolons and a regional Windows encoding.

The office manager first imports the file into a test account. The preview shows broken accented names and tax numbers that have lost their leading zeros. She re-exports the file as UTF-8, sets the delimiter to semicolon, marks the tax number column as text and maps the old client number as the unique key. The second test imports 3,791 records, and the error report lists nine rows with invalid email addresses.

After correcting those nine rows, the final import into the live account completes with all 3,800 clients matching the source count. Because the client number is the key, a later rerun to add late changes updates existing records instead of creating duplicates.