krz/domain-dig

an ios app for DNS & SSL analysis dns ios networking ssl https://krz.sh/apps/domain-dig/

Commit 99e4623af1

99e4623af1b08f36120a01b67cbe60df99668651

parent: a52dee116d

Unsigned

cmc <hello@cleberg.net> · 2026-07-25 04:35 UTC 

feat: versioned store-migration policy for persisted data (v5 step 2)

Third v5.0.0 roadmap item: define and implement a migration policy for the
on-device persisted store (tracked domains, history/snapshots, audits,
workflows, monitoring, settings), so data upgrades cleanly across app versions
instead of relying on a one-shot marker.

- DataMigrationService is reworked from a single boolean marker
  (`data.migrations.v3_4_0`) into a versioned runner keyed by an integer store
  schema version (`data.storeSchemaVersion`). It runs each step once in
  ascending order up to `currentStoreSchemaVersion`, stamping the version as it
  goes. Adding a future migration is now a `case N:` plus a version bump.

  Policy guarantees, all covered by tests:
  - Forward-only and idempotent; every step must be safe on an empty/older store.
  - Never downgrades: a store written by a newer build (higher version) is left
    byte-for-byte untouched.
  - Pre-versioning installs are handled: a set legacy boolean marker reads as
    "already at v1", so the v1 normalization never re-runs for them.

  v1 is the existing normalization pass (dedup + drop the legacy `watchedDomains`
  key + sanitize monitoring settings), now expressed as migration step 1.

- Docs/data-migration.md documents the persisted surface, the two independent
  version lines (store vs. backup export), when to use lenient decoding vs. a
  migration step, the runner contract, an "adding a migration" checklist, and
  backup-import compatibility. Linked from the README.

- DataMigrationServiceTests: 6 tests over legacy fixtures — fresh-store stamping,
  legacy `watchedDomains` migration + key drop, in-place dedup of the stored
  blob, idempotence, legacy-marker-as-v1, and the no-downgrade guard. Full unit
  suite: 58 passing.

Layout: unified · split

Docs/data-migration.md added +94
@@ -0,0 +1,94 @@
1# DomainDig Data Migration Policy
2
3How DomainDig's persisted data evolves across app versions without losing or
4corrupting a user's on-device store.
5
6## What is persisted
7
8The store is a set of independent JSON blobs in `UserDefaults`, each under a
9stable key (see `DomainDataPortabilityService.StorageKey`):
10
11| Data | Key |
12|------|-----|
13| Tracked domains | `trackedDomains` (legacy: `watchedDomains`) |
14| Lookup history (snapshots) | `lookupHistory` |
15| Audit sessions | `domainAudits` |
16| Workflows | `domainWorkflows` |
17| Monitoring settings / logs | `monitoring.settings`, `monitoring.logs` |
18| App settings | `recentSearches`, `savedDomains`, resolver URL, density |
19| Feature metadata | `purchase.cachedEntitlement`, `usageCredits.ledger` |
20
21A **backup export** (`DomainDigBackup`) is a separate, self-describing file that
22bundles all of the above with its own `schemaVersion`.
23
24## Two version lines
25
26- **Store schema version** — `DataMigrationService.currentStoreSchemaVersion`,
27 persisted under `data.storeSchemaVersion`. Describes the shape of the
28 *on-device* `UserDefaults` store. Advanced by the migration runner.
29- **Backup schema version** — `DomainDigBackup.currentSchemaVersion`, written
30 into every exported file. Describes the shape of an *export*. Checked on import
31 by `DataValidationService`.
32
33They advance independently: a store migration that doesn't change the export
34shape need not bump the backup version, and vice versa.
35
36## How models evolve
37
38Prefer **additive, lenient decoding** — it needs no migration:
39
40- New optional field → add it with `decodeIfPresent(...) ?? default` in the
41 model's `init(from:)`. Old data simply lacks the key and falls back.
42- New value in a `String`-backed enum → decode unknown values to a safe default
43 rather than throwing.
44
45Reach for a **migration step** only when lenient decoding can't express the
46change:
47
48- Renaming or removing a storage key (e.g. `watchedDomains` → `trackedDomains`).
49- Re-normalizing existing rows (dedup, canonicalizing domain casing).
50- Reshaping a blob in a way old readers would misread.
51
52## The migration runner
53
54`DataMigrationService.migrateIfNeeded(defaults:)` runs at launch (and before any
55backup export/import). Its contract:
56
571. **Forward-only.** It reads the stored version and runs each step with a target
58 greater than it, in ascending order, up to `currentStoreSchemaVersion`,
59 stamping the new version after each step.
602. **Never downgrades.** A store stamped at a version *higher* than this build
61 understands (a user who ran a newer build first) is left untouched — no
62 rewrite, no data loss.
633. **Idempotent & safe on any state.** Every step must be safe to run on an empty
64 store and to re-run, because a downgrade-then-upgrade or a partial run can
65 replay it. v1 (the `watchedDomains` drop + dedup normalization) satisfies this
66 by loading through the deduplicating loaders and writing back.
674. **Pre-versioning installs.** Before this framework, a boolean marker
68 (`data.migrations.v3_4_0`) recorded that the v1 normalization had run. A set
69 marker is read as "already at version 1," so v1 never re-runs for those users.
70
71## Adding a migration
72
731. Add a `case N:` to `DataMigrationService.runMigration(to:defaults:)` and a
74 private helper that performs the change.
752. Bump `currentStoreSchemaVersion` to `N`.
763. Make the helper idempotent and safe on an empty/older store.
774. Add a `DataMigrationServiceTests` case that seeds a pre-`N` fixture, runs
78 `migrateIfNeeded`, and asserts the upgrade plus the version stamp.
795. If the change also alters the export shape, bump
80 `DomainDigBackup.currentSchemaVersion` and update `Docs/local-api.md` /
81 backup validation as needed.
82
83## Backup import compatibility
84
85On import, `DataValidationService.validate(backup:)` compares the file's
86`schemaVersion` to the current one:
87
88- **Newer** than this build → surfaced as an error (the build can't safely read
89 it).
90- **Older** → imported under the same lenient decoders and merge/dedup rules that
91 govern the live store; a note is surfaced, not an error.
92
93Imported data flows through `migrateIfNeeded` and the same `save*` deduplication
94as everything else, so an old backup lands in the store already normalized.
DomainDataPortabilityService.swift +59 −4
@@ -342,12 +342,69 @@ enum DataPortabilityCSV {
342 } 342 }
343} 343}
344 344
345/// Versioned migration runner for the on-device persisted store.
346///
347/// The store is a set of independent JSON blobs in `UserDefaults` (tracked
348/// domains, history, audits, workflows, monitoring settings/logs, app settings).
349/// Most model evolution is handled additively by the models' own lenient
350/// decoders (`decodeIfPresent` with defaults), which need no migration at all.
351/// This runner exists only for changes lenient decoding can't express: dropping
352/// a renamed storage key, re-normalizing existing rows, or reshaping a blob.
353///
354/// `currentStoreSchemaVersion` is bumped whenever such a step is added. Each step
355/// runs exactly once, in ascending order, and must be safe to run on any prior
356/// state — including an empty store. A store written by a newer build (a higher
357/// version than this build knows) is left untouched; migrations never downgrade.
358/// The policy is documented in `Docs/data-migration.md`.
345enum DataMigrationService { 359enum DataMigrationService {
346 private static let migrationMarkerKey = "data.migrations.v3_4_0" 360 /// The schema version this build expects the on-device store to be at.
361 static let currentStoreSchemaVersion = 1
362
363 /// UserDefaults key holding the store's current schema version.
364 static let storeSchemaVersionKey = "data.storeSchemaVersion"
365
366 /// Pre-versioning installs recorded that the one-shot v1 normalization had
367 /// run using this boolean marker; `true` means the store is already at v1.
368 private static let legacyNormalizationMarkerKey = "data.migrations.v3_4_0"
369
370 /// The store's current schema version. Absent on pre-versioning installs: a
371 /// set legacy marker means v1 already ran, otherwise the store is fresh or
372 /// never-migrated at v0.
373 static func storeSchemaVersion(defaults: UserDefaults = .standard) -> Int {
374 if let version = defaults.object(forKey: storeSchemaVersionKey) as? Int {
375 return version
376 }
377 return defaults.bool(forKey: legacyNormalizationMarkerKey) ? 1 : 0
378 }
347 379
348 static func migrateIfNeeded(defaults: UserDefaults = .standard) { 380 static func migrateIfNeeded(defaults: UserDefaults = .standard) {
349 guard !defaults.bool(forKey: migrationMarkerKey) else { return } 381 // At or ahead of this build's version: nothing to do, and never rewrite
382 // a store a newer build may have reshaped (forward compatibility).
383 guard storeSchemaVersion(defaults: defaults) < currentStoreSchemaVersion else { return }
384
385 var version = storeSchemaVersion(defaults: defaults)
386 while version < currentStoreSchemaVersion {
387 let target = version + 1
388 runMigration(to: target, defaults: defaults)
389 defaults.set(target, forKey: storeSchemaVersionKey)
390 version = target
391 }
392 }
393
394 private static func runMigration(to version: Int, defaults: UserDefaults) {
395 switch version {
396 case 1:
397 normalizeAllStores(defaults: defaults)
398 default:
399 break
400 }
401 }
350 402
403 /// v1: load every store through its deduplicating loader and write it back.
404 /// This consolidates duplicate rows, drops the legacy `watchedDomains` key
405 /// (via `saveTrackedDomains`), and sanitizes monitoring settings against the
406 /// surviving tracked domains. Safe on an empty store (every step is a no-op).
407 private static func normalizeAllStores(defaults: UserDefaults) {
351 let trackedDomains = DomainDataPortabilityService.loadTrackedDomains(defaults: defaults) 408 let trackedDomains = DomainDataPortabilityService.loadTrackedDomains(defaults: defaults)
352 DomainDataPortabilityService.saveTrackedDomains(trackedDomains, defaults: defaults) 409 DomainDataPortabilityService.saveTrackedDomains(trackedDomains, defaults: defaults)
353 410
@@ -366,8 +423,6 @@ enum DataMigrationService {
366 423
367 let monitoringLogs = DomainDataPortabilityService.loadMonitoringLogs(defaults: defaults) 424 let monitoringLogs = DomainDataPortabilityService.loadMonitoringLogs(defaults: defaults)
368 DomainDataPortabilityService.saveMonitoringLogs(monitoringLogs, defaults: defaults) 425 DomainDataPortabilityService.saveMonitoringLogs(monitoringLogs, defaults: defaults)
369
370 defaults.set(true, forKey: migrationMarkerKey)
371 } 426 }
372} 427}
373 428
DomainDig.xcodeproj/project.pbxproj +4
@@ -10,6 +10,7 @@
10 05F775C7E0743AF727B008EE /* LocalAPIContract.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7572DA0A838E0A044F045260 /* LocalAPIContract.swift */; }; 10 05F775C7E0743AF727B008EE /* LocalAPIContract.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7572DA0A838E0A044F045260 /* LocalAPIContract.swift */; };
11 38316D90539394C2CC7C12BE /* Foundation.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = DE8B269A01CC5E593DA3DFC2 /* Foundation.framework */; }; 11 38316D90539394C2CC7C12BE /* Foundation.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = DE8B269A01CC5E593DA3DFC2 /* Foundation.framework */; };
12 47CD3BB1AE143733A73E0E5B /* DomainReportExporterTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */; }; 12 47CD3BB1AE143733A73E0E5B /* DomainReportExporterTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */; };
13 4F96CEB875EC3501E784CE39 /* DataMigrationServiceTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 435AFB3F99D579D3D7B58279 /* DataMigrationServiceTests.swift */; };
13 81359F63C7A23454B8FA0141 /* DomainReportBuilderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E82320955416797CFE59464A /* DomainReportBuilderTests.swift */; }; 14 81359F63C7A23454B8FA0141 /* DomainReportBuilderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E82320955416797CFE59464A /* DomainReportBuilderTests.swift */; };
14 8BBFEF092F9874AE00E8E144 /* DomainInspectionService.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8BBFEF032F9874AE00E8E144 /* DomainInspectionService.swift */; }; 15 8BBFEF092F9874AE00E8E144 /* DomainInspectionService.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8BBFEF032F9874AE00E8E144 /* DomainInspectionService.swift */; };
15 8BBFEF0A2F9874AE00E8E144 /* DomainReportBuilder.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8BBFEF042F9874AE00E8E144 /* DomainReportBuilder.swift */; }; 16 8BBFEF0A2F9874AE00E8E144 /* DomainReportBuilder.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8BBFEF042F9874AE00E8E144 /* DomainReportBuilder.swift */; };
@@ -75,6 +76,7 @@
75/* Begin PBXFileReference section */ 76/* Begin PBXFileReference section */
76 0D85A44C5F315A1644AC9073 /* LocalAPIContractTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = LocalAPIContractTests.swift; sourceTree = "<group>"; }; 77 0D85A44C5F315A1644AC9073 /* LocalAPIContractTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = LocalAPIContractTests.swift; sourceTree = "<group>"; };
77 15A25DF2B8BB52589D49986B /* DiffServiceTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = DiffServiceTests.swift; sourceTree = "<group>"; }; 78 15A25DF2B8BB52589D49986B /* DiffServiceTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = DiffServiceTests.swift; sourceTree = "<group>"; };
79 435AFB3F99D579D3D7B58279 /* DataMigrationServiceTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = DataMigrationServiceTests.swift; sourceTree = "<group>"; };
78 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = DomainReportExporterTests.swift; sourceTree = "<group>"; }; 80 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = DomainReportExporterTests.swift; sourceTree = "<group>"; };
79 5CA789D3607B55E3612E6FFA /* SnapshotFixture.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = SnapshotFixture.swift; sourceTree = "<group>"; }; 81 5CA789D3607B55E3612E6FFA /* SnapshotFixture.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = SnapshotFixture.swift; sourceTree = "<group>"; };
80 7572DA0A838E0A044F045260 /* LocalAPIContract.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = LocalAPIContract.swift; sourceTree = "<group>"; }; 82 7572DA0A838E0A044F045260 /* LocalAPIContract.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = LocalAPIContract.swift; sourceTree = "<group>"; };
@@ -229,6 +231,7 @@
229 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */, 231 54D7C7D97A4006831572F468 /* DomainReportExporterTests.swift */,
230 CC948A02EC0184228BC4630E /* DomainDataPortabilityServiceTests.swift */, 232 CC948A02EC0184228BC4630E /* DomainDataPortabilityServiceTests.swift */,
231 0D85A44C5F315A1644AC9073 /* LocalAPIContractTests.swift */, 233 0D85A44C5F315A1644AC9073 /* LocalAPIContractTests.swift */,
234 435AFB3F99D579D3D7B58279 /* DataMigrationServiceTests.swift */,
232 ); 235 );
233 name = DomainDigTests; 236 name = DomainDigTests;
234 path = DomainDigTests; 237 path = DomainDigTests;
@@ -487,6 +490,7 @@
487 C7CA9E02B0DC2708DE7A8563 /* DomainDataPortabilityServiceTests.swift in Sources */, 490 C7CA9E02B0DC2708DE7A8563 /* DomainDataPortabilityServiceTests.swift in Sources */,
488 A5AF921BC1C2E6E940CC05DC /* SnapshotFixture.swift in Sources */, 491 A5AF921BC1C2E6E940CC05DC /* SnapshotFixture.swift in Sources */,
489 EA12012CDB4C21ABB4217D86 /* LocalAPIContractTests.swift in Sources */, 492 EA12012CDB4C21ABB4217D86 /* LocalAPIContractTests.swift in Sources */,
493 4F96CEB875EC3501E784CE39 /* DataMigrationServiceTests.swift in Sources */,
490 ); 494 );
491 runOnlyForDeploymentPostprocessing = 0; 495 runOnlyForDeploymentPostprocessing = 0;
492 }; 496 };
DomainDigTests/DataMigrationServiceTests.swift added +109
@@ -0,0 +1,109 @@
1import XCTest
2@testable import DomainDig
3
4/// Characterization tests for the versioned store-migration runner. They drive
5/// `migrateIfNeeded` against legacy on-disk fixtures in an ephemeral
6/// `UserDefaults` suite and pin the policy: forward-only, idempotent, and
7/// non-destructive to a store written by a newer build.
8///
9/// The raw storage-key strings ("trackedDomains", "watchedDomains", the legacy
10/// marker) are duplicated here on purpose — they are the on-disk contract, and
11/// hard-coding them means an accidental rename shows up as a failing migration.
12final class DataMigrationServiceTests: XCTestCase {
13 private let suiteName = "DomainDigTests.migration"
14 private var defaults: UserDefaults!
15 private let base = Date(timeIntervalSince1970: 1_700_000_000)
16
17 override func setUp() {
18 super.setUp()
19 defaults = UserDefaults(suiteName: suiteName)
20 defaults.removePersistentDomain(forName: suiteName)
21 }
22
23 override func tearDown() {
24 defaults.removePersistentDomain(forName: suiteName)
25 defaults = nil
26 super.tearDown()
27 }
28
29 func testFreshStoreIsStampedAtCurrentVersion() {
30 XCTAssertEqual(DataMigrationService.storeSchemaVersion(defaults: defaults), 0)
31
32 DataMigrationService.migrateIfNeeded(defaults: defaults)
33
34 XCTAssertEqual(
35 DataMigrationService.storeSchemaVersion(defaults: defaults),
36 DataMigrationService.currentStoreSchemaVersion
37 )
38 }
39
40 func testLegacyWatchedDomainsAreMigratedAndTheOldKeyIsDropped() throws {
41 let legacy = [WatchedDomain(domain: "legacy.example", createdAt: base, lastKnownAvailability: .registered)]
42 defaults.set(try JSONEncoder().encode(legacy), forKey: "watchedDomains")
43
44 DataMigrationService.migrateIfNeeded(defaults: defaults)
45
46 let tracked = DomainDataPortabilityService.loadTrackedDomains(defaults: defaults)
47 XCTAssertEqual(tracked.map(\.domain), ["legacy.example"])
48 XCTAssertNil(defaults.data(forKey: "watchedDomains"), "legacy key is dropped after migration")
49 XCTAssertNotNil(defaults.data(forKey: "trackedDomains"), "data is rewritten under the current key")
50 XCTAssertEqual(DataMigrationService.storeSchemaVersion(defaults: defaults), 1)
51 }
52
53 func testMigrationDeduplicatesTheStoredBlobInPlace() throws {
54 let dupes = [
55 TrackedDomain(domain: "dupe.example", updatedAt: base.addingTimeInterval(-10)),
56 TrackedDomain(domain: "DUPE.example", updatedAt: base)
57 ]
58 defaults.set(try JSONEncoder().encode(dupes), forKey: "trackedDomains")
59
60 DataMigrationService.migrateIfNeeded(defaults: defaults)
61
62 let data = try XCTUnwrap(defaults.data(forKey: "trackedDomains"))
63 let stored = try JSONDecoder().decode([TrackedDomain].self, from: data)
64 XCTAssertEqual(stored.count, 1, "the persisted blob is deduplicated, not just the load result")
65 }
66
67 func testMigrationIsIdempotent() throws {
68 defaults.set(
69 try JSONEncoder().encode([TrackedDomain(domain: "a.example", updatedAt: base)]),
70 forKey: "trackedDomains"
71 )
72
73 DataMigrationService.migrateIfNeeded(defaults: defaults)
74 let afterFirst = defaults.data(forKey: "trackedDomains")
75
76 DataMigrationService.migrateIfNeeded(defaults: defaults)
77 let afterSecond = defaults.data(forKey: "trackedDomains")
78
79 XCTAssertEqual(afterFirst, afterSecond, "a second run makes no further changes")
80 XCTAssertEqual(
81 DataMigrationService.storeSchemaVersion(defaults: defaults),
82 DataMigrationService.currentStoreSchemaVersion
83 )
84 }
85
86 func testLegacyBooleanMarkerCountsAsVersionOne() {
87 defaults.set(true, forKey: "data.migrations.v3_4_0")
88
89 XCTAssertEqual(DataMigrationService.storeSchemaVersion(defaults: defaults), 1)
90
91 // Already at v1, so the v1 step must not re-run: a leftover legacy blob
92 // is left exactly as found.
93 defaults.set(Data("x".utf8), forKey: "watchedDomains")
94 DataMigrationService.migrateIfNeeded(defaults: defaults)
95 XCTAssertNotNil(defaults.data(forKey: "watchedDomains"), "v1 is treated as already applied; no re-run")
96 }
97
98 func testNewerStoreVersionIsNeverDowngradedOrRewritten() throws {
99 let future = DataMigrationService.currentStoreSchemaVersion + 1
100 defaults.set(future, forKey: DataMigrationService.storeSchemaVersionKey)
101 let blob = try JSONEncoder().encode([TrackedDomain(domain: "keep.example", updatedAt: base)])
102 defaults.set(blob, forKey: "trackedDomains")
103
104 DataMigrationService.migrateIfNeeded(defaults: defaults)
105
106 XCTAssertEqual(DataMigrationService.storeSchemaVersion(defaults: defaults), future, "must never downgrade")
107 XCTAssertEqual(defaults.data(forKey: "trackedDomains"), blob, "future-version data is left byte-for-byte")
108 }
109}
README.md +1 −1
@@ -51,7 +51,7 @@ Network inspection requests are made only to perform the requested domain checks
51xcodebuild -project DomainDig.xcodeproj -scheme DomainDig -destination 'platform=iOS Simulator,name=iPhone 16' build 51xcodebuild -project DomainDig.xcodeproj -scheme DomainDig -destination 'platform=iOS Simulator,name=iPhone 16' build
52``` 52```
53 53
54The app and local API share the canonical report pipeline through `DomainInspectionService`, `DomainReportBuilder`, and `DomainReportExporter`. The Local API's endpoints, response envelope, and `v1` compatibility policy are documented in [Docs/local-api.md](Docs/local-api.md). 54The app and local API share the canonical report pipeline through `DomainInspectionService`, `DomainReportBuilder`, and `DomainReportExporter`. The Local API's endpoints, response envelope, and `v1` compatibility policy are documented in [Docs/local-api.md](Docs/local-api.md). How the on-device store evolves across app versions is documented in [Docs/data-migration.md](Docs/data-migration.md).
55 55
56### Accessibility Audit 56### Accessibility Audit
57 57