Raw data is data in the form in which it was originally captured from its source, before any cleaning, correction, transformation, aggregation or interpretation has been applied. Sensor readings, server log lines, survey responses as submitted and transaction records exported from an operational system are all raw data. Among the data fundamentals, it is the starting point from which every dataset, metric and model is derived.
The term is relative rather than absolute. A file can be raw for one team and already processed for another, because almost every capture involves some processing. What matters is that raw data is the earliest preserved version available to its users and that it is kept unaltered, so later results can always be traced and recomputed.
What makes data raw
Data counts as raw when it has not been modified after capture in ways that change its meaning: no records removed, no values corrected or imputed, no units converted and no rows combined. It may still be in a technical encoding, such as a binary sensor format or compressed log archive, because encoding alone does not alter the content.
Even raw data carries the fingerprints of how it was captured. A thermometer rounds to its resolution, a web form trims input to a maximum length, and an application logs only the events its developers chose to record. Understanding these capture-time decisions is part of using raw data correctly, because they cannot be undone later.
The following table lists typical sources of raw data and the form in which it usually arrives.
| Source | Typical raw form | Common quirks |
|---|---|---|
| Sensors and devices | Timestamped numeric readings | Drift, gaps, device-specific units |
| Application logs | Text or JSON lines | Inconsistent fields across versions |
| Transactional systems | Table exports or change streams | Codes that need lookup tables |
| Surveys and forms | Responses as submitted | Free text, typos, partial answers |
| Web and API capture | HTML pages or API payloads | Layout changes, rate-limit gaps |
| Scientific instruments | Binary files with headers | Calibration needed before use |
Raw data compared with processed data
Raw data differs from processed data in purpose: raw data preserves what was observed, while processed data presents what is believed to be true and useful after rules have been applied. Both are necessary, and confusing them is a common source of analytical errors.
| Property | Raw data | Processed data |
|---|---|---|
| Fidelity to source | Complete, as captured | Filtered and corrected |
| Structure | Often irregular or nested | Consistent schema |
| Ready for analysis | Rarely | Usually |
| Reproducibility role | Basis for recomputation | Result of a specific pipeline version |
| Typical volume | Larger | Smaller after aggregation |
| Mutability | Should be immutable | Rebuilt when rules change |
Raw data is also distinct from unstructured data. A neat table export can be raw, and a free-text document can be thoroughly processed. Rawness describes the processing history, while structure describes the shape of the content, so the two classifications are independent.
Stages from raw to analysis-ready data
Raw data becomes useful through a sequence of processing stages, each of which should be recorded so that the path back to the original is never lost. Many data platforms formalize these stages as storage zones or layers.
| Stage | What happens | Output |
|---|---|---|
| Capture | Data is recorded at the source | Raw files or streams |
| Landing | Raw data is stored unchanged with metadata | Immutable raw zone |
| Validation and cleaning | Types checked, duplicates and errors handled | Cleaned records |
| Transformation | Joins, unit conversion, standard codes | Conformed tables |
| Aggregation | Summaries for reporting or features | Metrics and features |
At the landing stage, the most valuable addition is metadata: when and how the data was captured, by which system and software version, and under what schema. Without this context, raw files become hard to interpret months later, even if every byte is preserved.
Why keeping raw data matters
Preserving raw data allows an organization to reprocess history whenever cleaning rules, definitions or models change. If only processed data is kept, a bug in an old transformation permanently corrupts the record, while with raw data the fix can be applied retroactively.
- Reproducibility: analyses and models can be rebuilt from the same inputs and audited step by step.
- Data lineage: every published figure can be traced back to the specific source records behind it.
- New questions: details discarded by earlier aggregation may be exactly what a later analysis needs.
- Error recovery: faulty pipelines can be corrected and rerun instead of patched manually.
Keeping raw data has costs as well. Storage grows continuously, and raw records often contain personal or sensitive information that later stages remove. Retention therefore needs a policy that balances reprocessing value against privacy obligations and storage budgets.
Common pitfalls with raw data
The typical mistakes with raw data either alter it too early or trust it too much. Both lead to results that look precise but rest on an unverified foundation.
| Pitfall | Consequence | Remedy |
|---|---|---|
| Overwriting raw files | History cannot be recomputed | Write-once raw storage |
| Analyzing raw data directly | Duplicates and errors skew results | Validate before analysis |
| Missing capture metadata | Units and meaning become unclear | Store schema and source details |
| Silent source changes | Fields shift without notice | Schema checks at landing |
| Unlimited retention | Privacy and cost risks grow | Defined retention periods |
A related misconception is that raw data is objective. It reflects the choices of whoever designed the capture, including what was measured, how often and with which instrument, so data quality assessment has to start at the source rather than at the first report.
In plain words
Raw data is like the original receipts in a shoebox before anyone has sorted them into a budget. The receipts are messy, some are faded and a few are duplicates, but they are the real record of what was spent.
A budget spreadsheet is easier to read, yet if a total looks wrong, the only way to check it is to go back to the receipts. That is why careful teams never throw the originals away.
A real-world example
A city water utility installs flow meters that report readings every 15 minutes from several thousand points in the network. Each meter sends a compact message with its identifier, a timestamp in device time and a raw pulse count rather than a volume.
The utility stores these messages unchanged in a raw zone, together with the meter model, firmware version and calibration factor at the time of capture. A separate pipeline converts pulses into cubic meters, aligns timestamps to local time and flags gaps.
A year later, engineers discover that one meter model reported pulses with a wrong multiplier after a firmware update. Because the raw messages were kept, the team recomputes the affected volumes for the whole period and corrects the leak-detection reports without any manual estimation.