Sprint 014 — API Intelligence: TIDB Gap Analysis

• Capability: API Discovery & API Attack-Surface Intelligence
• Sprint: 014 · Wave 8
• Status: Ratified
• Companion: Implementation Plan

1. Scope of analysis

This report inventories the Target Intelligence Database (TIDB) surface that the API intelligence capability reads and writes, identifies what already exists (reused unchanged), what is extended, and what is created new, and fixes the migration + registry + exporter steps needed so the canonical API inventory is a first-class TIDB citizen — never memory-only.

2. Existing TIDB surface reused unchanged (no writes)

These are read-only inputs: the API intelligence service folds them into observations; it never mutates them.

3. Existing TIDB surface extended (schema additions)

None of the existing API tables (tidb_apis, tidb_rest_endpoints, tidb_graphql_endpoints, tidb_soap_endpoints, tidb_rpc_services, tidb_authentication_schemes, tidb_authorization_models) require column additions or constraint changes. The Wave-8 canonical inventory is carried by new tables (below), which is the least-risk path: no migration of populated legacy rows, no breaking changes, and the registry’s name-based auto-mapping (<Name>Model ↔ <Name>) keeps the existing entities untouched.

4. New TIDB entities required (created)

All new entities derive from TidbEntity and get mirrored ORM models in infrastructure/db/sql/tidb_models/api_intelligence_models.py. Names follow the <Entity>Model ↔ <Entity> convention so the generic registry derives repositories automatically.

4.1 APIRun → tidb_api_runs

Run observability record.

4.2 APIHost → tidb_api_hosts

Canonical discovered API origin/host.

4.3 APISpec → tidb_api_specs

Located spec document.

4.4 APIVersion → tidb_api_versions

Canonical API version + version-state.

4.5 APIOperation → tidb_api_operations

Canonical endpoint operation.

4.6 APIParameter → tidb_api_parameters

Operation parameter.

4.7 APISchema → tidb_api_schemas

Request/response schema fingerprint (bounded: digest + shallow model).

4.8 APIAuthentication → tidb_api_authentications

Per API/host authentication scheme (intelligence, no secrets).

4.9 APIAuthorization → tidb_api_authorizations

Authorization model indicators.

4.10 APIRateLimit → tidb_api_rate_limits

Rate-limit indicators.

4.11 APIPagination → tidb_api_paginations

Pagination style per endpoint/API.

4.12 APIFilter → tidb_api_filters

Filter capabilities observed on list endpoints.

4.13 APIEvidence → tidb_api_evidence

Evidence fragments backing API intelligence.

4.14 APIConflict → tidb_api_conflicts

Preserved contradictions (version/identity/source).

4.15 APIChange → tidb_api_changes

Historical added/removed/changed events.

4.16 Relationship-free design decision

The canonical inventory carries its own host_id/api_id/operation_id foreign keys instead of a dedicated entity-relationship table. Cross-entity relationships into the attack-surface topology are the only edges persisted elsewhere: tidb_topology_relationships rows of type EXPOSES (host→API), SERVES (origin→API), ROUTES_TO (URL→endpoint) and SERVES (API→operation). This avoids a duplicate relationship store and keeps the topology as the single projection layer.

5. Migration plan

• New migration: alembic/versions/<rev>_api_intelligence_tables.py. down_revision = head (7ab1a304e8bb technology tables). Creates the 16 tables above (4.1–4.15 + index columns). Downgrade drops them.
• All tables follow the existing pattern: TidbModelMixin base columns (id, created_at, updated_at, first_seen, last_seen, version, revision, schema_version, deleted_at, meta) plus capability columns.
• tidb_models/__init__.py imports the new module and appends all <Entity>Model names to __all__ so registry auto-mapping and create_all/autogenerate see them.

6. Registry + repository + exporter wiring

• infrastructure/db/sql/registry.py — name-based auto-mapping already derives <Name>Model ↔ <Name>; no change required beyond the __all__ registration (verified mapping logic reads the models module __all__).
• infrastructure/db/sql/crud.py/repositories.py — generic SqlCrudRepository
  • RowMapper handle the new entities unchanged.
• Memory / test path: InMemoryTidbRepositoryFactory auto-discovers entities from the same registry — new entities become available to integration tests without new code.
• Report/exporter paths: APIRun + summary JSON feed the reporting layer; no new exporter subsystem is created (reuse existing reporting).

7. Verification steps (gates)

1. python -m alembic upgrade head succeeds.
2. python -m alembic check / autogenerate produces no unexpected diff for the new tables.
3. pytest tests/architecture passes (no forbidden imports; every new module has a docstring; no cycles; layer rules hold).
4. Integration test writes/reads each new entity through the in-memory factory.
5. pytest tests/security confirms no credentials/secrets stored and no scope expansion.