Daria Pavlova

Data infrastructure

NBU Registry Export

A large registry export shaped by data volume, bounded compute and explicit validation boundaries.

Public scale

≈500,000,000

records across a multi-year regulatory dataset

Evidence status

Detailed case study in preparation

System view

Public architecture excerpt
Registry export pipelineExtract, transform, Parquet and S3 stages with validation and StarRocks relationships.EXTRACTTRANSFORMPARQUETS3VALIDATIONSTARROCKS

Problem

Exporting a multi-year registry at this scale changes the engineering problem: progress must be observable, work must stay bounded, and validation cannot be deferred until the end of the run.

Engineering approach

  1. Partition extraction and transformation work so execution remains bounded and restartable.
  2. Keep validation as an explicit system boundary before publishing downstream data.
  3. Separate columnar output, object storage and analytical serving concerns.

What the evidence supports

The approximate record count is the only public measurement currently presented. Benchmark conditions, implementation details and repository access will be added only when they can be documented precisely.