Universal id to name mapping for the front end and back end

Across any app of scale, you’ll encounter the same challenge over and over: how to take an internal ID like 17 and turn it into something meaningful like "Plastics". Entities — like invoices, ingredient, or projects — are represented as IDs in the database. But users want readable names, searchable in the right language, with contextual labels, and icons.

We built a system to solve this: id-source. It’s a universal mechanism that maps identifiers to user-friendly labels across the entire product. Whether you’re showing a dropdown list, rendering a name in a table, or doing a fuzzy search, id-source handles the lookup, the localization, the formatting, and the performance optimization—automatically.

This article details system architecture, provides code samples, and explores specialized features. I’ll focus on how we implemented this system with our sibling system: structured loading. Many iterations shaped the id-source system.

The high-level features of the id-source system are:

• Efficiently mapping entity identifiers to names
• Language dependent translations of ids for user defined entities
• Searching by name on lists
• Custom labelling based on one or more entity fields, not just name
• Progressive loading and search in dropdown select controls
• Handles deprecated / removed ids
• Filtered lists for limited user selection
• Additional entity metadata, such as icons and colour tags

High-level use overview

The core feature of the id-source is to take an identifier and produce a user-readable name for id. For example, given project id 13 return the string “Plastics”, which is the name the user gave that project. When a page of the app loads data from the backend, and it gets the project id, it can use the id-source system to display the name to the user.

We use diverse abstractions here. One of the simplest is the DisplayId React component.

The idSource tells the DisplayId component how it should resolve that name. The constructEntityIdSource is a universal way to create id-sources. In this case it produces the string entity/project. As an added bonus, some entity types have pages for them, like a project overview. The DisplayId componment can link to that page.

This will end up calling an id_source API endpoint that translates this value into a name.

Lower-level use-cases

Not all code wants to display the name, instead needing the name for other reasons. For those cases, we created a React hook function that resolves one or more ids into names.

The hook functions are async, returning undefined until the value is loaded. We generally don’t show loading feedback, like a spinner, while loading, since the calls are quite fast and we want to avoid layout flicker.

Part of the motivator for id-sources was our form system. We allow user defined fields, and those need to select id-based entities as well. So while we do have a lot of static id-source construction like the above, we also have a lot of generic lookups. In these situations we refer to an id-source stored on the field definition.

Backend id-source loader

Now let's move on to to the backend, where these calls are all ultimately resolved. A function like useSelectIdName( idSource, idValue ) will get passed to a list_id_source endpoint passing those two properties.

I mentioned briefly before that idSource has a structure of a schema and a path, like entity/project. When the backend sees entity as the first part of the schema, it signals that the structured loading system will resolve the id. It uses the second part of the schema to determine which entity is being referred to.

A concept we have in our structured loading system is “essential fields”. These are fields, or columns, on a store, that have generic meaning — in contrast to the fields which are relevant only for custom business logic.

In the id-source system, there are two key essential fields we’re interested in: name and id. We can lookup these fields on the store and produce a standard query to access them.

In summary, given the id-source entity/project:

• entity says to use the structured loading system
• project says to find the Project store
• we then look for the id and name essential fields
• we then produce a query with id and name filtering by ids to find the correct names

Since we model all our entities in the structured loading system, our id-name resolution is effectively free. This wasn’t always the case though, as the id-source system existing prior to the structured loading system. We still have remnants of that system, named “custom loaders”, which were quite structured in their own right — perhaps the beginnings of the structured loading system.

We have different loaders for enumeration values and field options. The general idea applies, but with a different structure to the loader. I can cover this in a future article for comparison.

Deprecated and removed values

A common use-case of ids is the archiving or deprecation of an entity. For example, you have a process workflow where a user needs to be assigned to approve a step. The list of users should only show those people who are available, not the complete list of users on the system. However, what happens when a previously assigned user leaves the company?

In our system, we decided that if you view the assignee field, on a form or in a list, you’d still see the name of removed users, but if you edit the field, you cannot select those removed users. How does the id-source system gather and relay this information to the front-end?

First, we don’t actually delete entities in our system, instead marking them archived. When you have a lot of cross-linked entities, actual deletion is not a realistic option. That means our removed user still exists in the DB, but with an archived flag set on it.

Then, in the loader, we distinguish between fundamental and selective filters. The fundamental filters define the entire space of potential entities. For most id-sources, this is simply the entire database table. Then there are selective filters. These further limit which entities should be available for selection in the UI. This is where we exclude archived entities. Think back to the store’s essential fields. We have one called “archived” which we use for this purpose. But users can also define custom filters.

The front-end makes two different calls to the id-source system. The two calls share one API and they could be combined, but rarely are anymore. These two requests are:

• Give me a list of names for a specific list of ids
• Search for these terms and return the results

Let's illustrate the first one with the HTTP request properties:

In this case, for the need_ids properties, the backend omits the selective filters. Even if the entity is archived or excluded, the backend loads it if the front-end specifically requests its ID.

For the second case we can have an HTTP request like this:

In this case, the search_terms and count indicate we're producing a list for the user to select from. The backend will include the selective filters, hiding all the archived items. This prevents the old items from being found in a dropdown control.

Use in a dropdown control, universal progressive loading optimization

The common dropdown control, which lists values for selection by a user, is perhaps the apex feature of the id-source system. It combines a lot of features into a seemingly simple control. We never aimed for this control; it emerged from combining several other features.

The initial focus was on progressive loading, not on naming. We have massive lists of entities in the project, those with over 100k entries. There’s no way to load that in the front-end for searching — indeed, anything above 1k becomes problematic. You required a control that presents part of that list but allows for searching, which dynamically loads via an API. For places that needed to select from those large lists, we built custom controls against their APIs — not a sustainable approach.

I had to write another such control, but saw a problem if we kept repeating this for all types in the system. So I attempted to write something reusable. I created a dropdown control that worked against a generic progressive loader. This is mostly just a standardized function signature and lookup table. You pass this loaders to the control and it handles the control part dynamically.

Then I needed one for this new id-source system (which was still in its early stages then). This worked well, and suddenly exposed a lot more entities and lists for progressive selection. Add in the structured loading and we now have a near universal abstraction for progressive select controls.

From the view of this control, the deprecated values is perhaps easier to understand. When the control doesn’t have focus, it shows only its current value. This calls the id-source API with the specific ids needed, which in turn uses only the fundamental filters, thus can find deprecated values. When the user opens the control, and types in a search term, that is sent to the API as a search request. The backend then applies the selective filters, removing all deprecated items.

Custom Labels

There was a recurring problem that came up with relying on the essential name field: some entities in the system don’t have names of their own. For example, entries in the inventory system. They are inventory for something else, such as an ingredient or a recipe, thus the inventory item itself doesn’t need its own name. It uses the name of what it links to, the ingredient or recipe name.

At the SQL level, this is problematic. You can’t produce an efficient query that dynamically joins to the correct target table to get the name. Consider that in a list, each row of the inventory may link to a different type of entity. However, since the list of targets is limited, we can link to all of them and coalesce the results, assuming one will not be null.

But that single coalesce would be too limited, and hard to implicitly determine based on the source entity. It is also not efficient to search such a query.

Instead, we have a label building system. A dynamic label has two parts:

• A set of columns to use as raw data
• An expression on how to combine them.

This breaks the problem into two parts: the loading and searching, and the label production.

For example, we might have a specification for the inventory items:

By having distinct columns, we can solve the search problem. Instead of searching the resulting name, we search each of the source columns. While not identical, this result will suffice for the majority of situations. It’s also quite efficient, since it can use the indexes on the individual columns.

We wrote the expression in our value-spec language, which would require its own series of articles to cover. This language is more flexible than SQL commands. We can also call it once we have the raw results; nothing on the SQL side cares about this expression.

This lets us do more than coalesce or join values. For example, our lab requests, while they have a name, they also have a run-number. Prior to id-source, we would present these names as the name, an icon separator, and the run number, for example: “Rubber Sample 📋 LR2134”. We can recreate this in the id-source system with this specification:

The application uses these labels in most places, from dropdown controls and list pages to forms and links.

Because of a few limitations, we will use the raw name sometimes; this is problematic for entities without proper names. Explaining the reason would require a lot of deeper details. I know the solution, but it requires a high time investment. The limitations are not yet problematic enough to justify that effort.

Structured select configuration

The lists in our systems allow an administrator to configure the listings, such as setting defaults, add extra tabs, configure filters, assign to user groups. Part of this is selecting the columns to display in the listing. In the previous section on custom labels, there was a columns part to the definition.

Some of these are hard-coded in our defaults files, and some on the structured stores themselves. But this is all backed by the same structured list configuration. This means that our users can edit the configuration used for the id-source system. They can select the columns they want to see in the names.

For example, say our user doesn’t like this standard formatting of the lab request. They may not be using the run number, but they do consider project important. So they can select the name and project column instead.

I’ve left the label out, as the default will comma-separate the values. There’s an interesting twist here. Notice that I said project_id, which is not the project name. Clearly, the user would like to see the project name. How is this resolved? Well, we use the id-source system! This recusrion gets complicated, but we end up with only a single SQL query.

These custom configurations also provide the search basis, not just visuals. By modifying the custom columns, the user can control what properties they can search by to help them find what they’re interest in. In the previous example, this means the user can search by request name and project name. This is useful when naming is not globally unique, but only unique in conjunction with the project.

Wrapping up with some metadata

There’s one last feature I mentioned at the start, but didn’t cover. That’s loading metadata, such as custom icons and tag colours. Our entities can also be locked, so we can show a lock symbol along with the name.

This feature directly results from using structured loading for the id-sources. Similar to the essential fields, we also have standard entity fields available to all entities. It’s no extra effort for the id-source loader to pull in a few of those as well. Since the select control, and form fields, are completely generic on the front-end, it’s also no problem for them to format this extra information.

There’s obviously a lot of details I’ve skipped in this article, focusing on the general architecture. I’ve also ignored non-structured loading based id-sources. That might be of interest if you want to implement such a system, so let me know if you want more details.