12. Non-nullable tag external_id

12. Non-nullable tag external_id#

Status#

Proposed

Context#

ADR 0010 made Tag.external_id mutable through an import-only rename pathway, to serve as the institution-editable “Competency ID” for Competency-Based Education (CBE) tags. That ADR left external_id nullable, scoped to competency taxonomies.

Building the CBE competency workflow surfaced a stronger requirement: every tag needs a value for external_id in order to be able to associate it with objects as part of competency criteria. It’s also anticipated that non-CBE taxonomies will want a mandatory, institution-assigned identifier in the future, particularly when there becomes a desire to convert regular taxonomies into Competency Taxonomies.

Today, external_id is nullable specifically to work around its own unique_together(taxonomy, external_id) constraint: NULL is the only value Postgres, MySQL, and SQLite all treat as exempt from a uniqueness constraint, so it’s the mechanism that lets multiple tags in the same taxonomy have “no external_id” at once. Making the field mandatory means every existing NULL row in every current Open edX instance needs a real value before the constraint can be added, since this is a published library whose migrations run unmodified against arbitrary downstream data. Tag.value already carries the same per-taxonomy, case-insensitive uniqueness constraint (unique_together(taxonomy, value)), so backfilling external_id from value is collision-free in virtually every case. The one edge case is a tag whose value matches a different tag’s pre-existing, hand-assigned external_id in the same taxonomy, which the backfill algorithm below handles.

Decision#

Make Tag.external_id NOT NULL at the database level, for every taxonomy, in one migration.

  • Backfill for existing rows. A RunPython step generates a value for every tag whose external_id is currently NULL, following the same backfill-then-constrain shape as the depth/lineage migration (0020_tag_depth_and_lineage.py): populate real values first, then add the constraint in the same migration once every row satisfies it.

  • Backfill algorithm. Set external_id to the tag’s existing value. In the rare case where that collides with a different tag’s pre-existing, hand-assigned external_id in the same taxonomy, the collision must be resolved deterministically without an unbounded retry loop, for example by appending an incrementing numeric counter to the copied value until it’s unique. No particular format or collision-resolution scheme is required by this decision.

  • New tags going forward. The REST API keeps treating external_id as optional for the caller, unchanged from today. When a tag is created via the API without one, the same backfill algorithm generates one automatically, rather than rejecting the request. No existing API integration that omits external_id today has to change. The import format already requires an identifier for every tag, unchanged by this decision, so auto-generation only applies to the API creation path.

  • Institutions can replace an auto-generated value. If an institution doesn’t want the auto-generated external_id, they can change it to their own value through ADR 0010’s rename pathway.

This is not treated as a breaking change to the public REST API, and doesn’t go through the DEPR process. The request side (external_id on tag creation) stays optional, so no existing caller has to change what it sends. The response side narrows from “nullable string” to “always a string,” which is the safe direction for consumers: code written to expect a possible null simply never takes that branch anymore.

Future considerations#

If an institution converts an existing regular taxonomy to a CBE taxonomy, its tags will already have a non-null external_id (auto-generated or backfilled) from this decision, with no separate migration needed for that conversion.

Rejected Alternatives#

Derive backfilled values from the tag’s primary key#

Needs the same kind of collision fallback as the value-derived approach: a tag could already have a hand-assigned external_id that happens to match another tag’s id. Also rejected because id is meant to stay internal (per the README’s model conventions, uuid is the documented identifier for external reference), and because it produces a value with no relationship to the tag (e.g. 482913), which reads as meaningless to institutions.

Require callers to supply external_id on tag creation#

Simpler mechanically than auto-generation, but rejected because it’s a breaking change for the REST API and any import/UI flow that omits external_id today.

Scope this decision to CBE/competency taxonomies only, for now#

Rejected because taxonomies later converted to Competency Taxonomies will want this too.

Changelog#

2026-07-10:

  • Proposed.

2026-08-17:

  • Scoped the import-format optionality claim to the REST API only; the import format already requires an identifier per tag row.

  • Struck the “uppercase and replace spaces with underscores” legibility example from the backfill algorithm: it isn’t injective, so it can map two distinct tag values onto the same external_id, undermining the collision-free argument this decision relies on.