B2B Product Catalog Management at Scale

For those who have never done eCommerce at scale, managing a product catalog seems straightforward. Fill out your fields, use unique SKUs, organize with proper categories, and if you only have a few dozen products, it actually can be straightforward!

But what about 80,000 products that need to be synced from an ERP every week, because that’s the system of record? And while this sync is happening, your production website needs to remain snappy and responsive for your customers and, above all, not crash?

Then add in the varied requirements of B2B commerce: accounting systems, layered permissions and pricing, multiple storefronts with different warehouses, managing quotes and ensuring stock is maintained, multiple languages, and more.

Product catalog management becomes an ongoing architectural challenge that touches data modeling, systems integration, and deployment discipline.

How do you juggle all of this? After years of building and maintaining large-scale Drupal Commerce platforms, we know where organizations commonly go wrong and what good practice looks like. We’ll help you navigate the labyrinth.

But first, what makes B2B product catalog management so different from its B2C counterpart? These distinctions help clarify best practices.

B2B commerce is more than just a different login screen

On the surface, a product catalog is a product catalog. You have items, they have attributes, customers find and buy them. But the similarities between B2C and B2B catalog management are mostly superficial.

A B2C catalog is generally public. Everyone sees the same products at the same prices. Usually, unless you are selling across different domains and your products differ drastically, the data model is flat. A product has a title, images, a price, maybe some variations for size or color. For more complex products, there might be other data sets.

B2B is a different animal. The same catalog might need to present different products, different prices, and different availability to different customers based on their contract, role, organization, or licensing agreement.

A standards organization might sell access to thousands of documents, training courses, and certification programs, each gated by membership tier. A distributor might carry products from dozens of manufacturers, each with its own pricing schedule that changes on different cadences. A biochemical supplier might have a million SKUs, most of which only matter to a narrow slice of customers searching by species, clonality, or molecular mass.

These differences have real architectural consequences. B2C catalog management is largely about presentation. B2B catalog management is largely about data architecture, access control, and systems integration. Getting it wrong means expensive fixes later.

How B2B and B2C catalog differences play out:

• Data complexity. B2C products tend to have a handful of shared attributes. B2B products often have dozens of specifications, many of which only apply to certain product types. A B2B organization might have five or ten distinct product types, each with its own set of attributes, relationships, and categorization schemes. There are some exceptions, of course.
• Pricing. B2C pricing is usually uniform or has simple promotional rules. B2B pricing can involve customer-specific price lists, contract-based pricing, volume tiers, and manufacturer-driven price updates that affect tens of thousands of SKUs at once.
• Access control. B2C products are public. B2B catalogs often require gating by customer role, group membership, or licensing status, and that gating might need to apply at the product category level, individual product level, or even the variation level. Getting the permissions architecture wrong at the start is a maintenance nightmare.
• Integration surface. A B2B store is often deeply integrated with an ERP, a PIM (product information management system), inventory systems, and marketing feeds. Each one has its own identifier scheme, its own update cadence, and its own opinions about what constitutes the authoritative record for a given field. A B2C store might connect to a similar number of systems, but only if it's selling at scale.

What to avoid with B2B commerce product modeling

Modeling your product data is the single most consequential decision in B2B catalog management. Here are some things to avoid.

Field proliferation

A common pattern, especially in Drupal, is for a site builder to add dozens of individual fields through the admin UI. Need to store 50 product specifications? Add 50 fields. The problem is that in at least some cases, each field creates a separate database table. Loading a single product entity now requires a join across 50+ tables. These types of operations can become a serious performance bottleneck.

Composite fields are the better approach. There are custom field types that store multiple related values together. Using the example above, if those 50 specifications naturally cluster into five groups of ten related properties, five composite fields with ten properties each reduces that 52-table join down to seven.

The address field is the canonical example: nobody creates separate fields for street, city, state, zip, and country. The same principle applies to product specifications, and the benefits go beyond just performance. When related properties live in the same field, it's far easier to implement conditional logic. For example, only showing one property when another has a particular value.

We recommend composite fields on nearly every Drupal Commerce project. However, because it requires more work upfront from developers, many teams don’t do it. They take the easy way out, even though developers will still be needed later to mitigate the performance problems.

Over-structured products

Not every piece of product data needs to be a first-class, individually structured field. The key question is: will you need to filter, sort, or facet on this data in search? If yes, it should be structured. If it's just displayed on a product detail page, it can usually be dumped into a generic specifications field to live beside other specifications.

Consider an industrial supplier selling expansion tanks. The capacity is the primary purchasing decision vector, so capacity needs to be a structured, indexable field. Every tank also has a height and diameter, which are relevant if you're checking whether it fits in a tight space, but almost nobody is browsing expansion tanks by height. Those values can live in a simple key-value specifications field that renders into a table on the product page without adding database overhead.

One large supplier with over a million SKUs took this principle to its logical extreme: they only stored the SKU, price, and product category in Drupal. The full product specifications live in a Solr index and are fetched client-side as JSON when someone views the product detail page. This setup works because those specifications aren't needed at the search level. Customers navigate to the right product through a handful of structured facets.

This is a trade-off that every B2B organization with a large, specification-heavy catalog needs to think through. Over-structuring creates performance and maintenance overhead. Under-structuring limits your ability to build meaningful search and filtering experiences.

Data junk drawers

On the other end of the spectrum from over-structuring is the temptation to serialize everything. Some teams stuff all product data into JSON blobs or serialized arrays. You can't validate data types, you can't query against individual properties, and you've essentially abandoned the benefits of a relational database.

At that point, you might as well be using WooCommerce's legacy approach of dumping data objects into the WordPress post meta table as key-value pairs. It was a pattern so inefficient that they eventually had to introduce what they diplomatically called "Higher Performance Order Storage," which was really just an admission that they needed to learn basic database architecture from the year 2000.

Key takeaways for product modeling:

• Use composite fields for related product attributes. Avoid creating fields in the UI for every single piece of data.
• Determine what Drupal needs for searching, filtering, and sorting, and avoid bloating its database beyond those needs.
• Don’t throw everything into a single, unstructured field.

Keeping catalogs in sync across systems

For most B2B organizations, the product catalog doesn't live in a single system. Product data originates in an ERP or PIM, gets synchronized to the eCommerce platform, and then needs to stay consistent across development, staging, and production environments. Third-party systems like marketing feeds, analytics platforms, or inventory management tools also need accurate data.

Polling vs. event-driven syncing strategies

There are two fundamental approaches to keeping product data in sync. The first is polling. At regular intervals—every 15 minutes, every hour, once a day—the eCommerce platform queries the source system for records that have changed since the last sync. Straightforward, but it introduces latency and can be wasteful if changes are infrequent.

The alternative is event-driven synchronization, where the source system sends a notification when a record is updated. The eCommerce platform records the notification and processes it through a queue. It provides near real-time updates and is generally more efficient, but it introduces its own complexity.

What happens when the notification can't be delivered? The remote system needs retry logic, logging for failed notifications, and a manual process for re-triggering a sync for individual records.

Whichever approach you choose, the data fetch should be done in chunks. You shouldn’t be trying to process 100,000 records at once. And critically, the processing should be offloaded to a queue system rather than happening in response to a single HTTP request. Without a proper queue, if the request times out or the browser closes, the job is simply lost with no record of what succeeded or failed.

Use the SKU as a universal identifier

One persistent headache in cross-system synchronization is identifier management. Different systems track products by different IDs (the Drupal entity ID, a UUID, the ERP's internal ID) and these identifiers are completely meaningless outside the system that generated them.

When a Google Shopping feed uses the Drupal entity ID as its product identifier, that data becomes impossible to reconcile with inventory data keyed on SKUs or analytics data keyed on yet another ID.

The SKU is supposed to be the universal identifier. It's the company's own stock-keeping unit, the identifier they chose for the product, independent of any particular platform. Wherever possible, integrations should key on SKU rather than internal system IDs.

Keying on the SKU keeps data portable. If you migrate from one platform to another, if you consolidate analytics across channels, if you need to reconcile inventory with sales data, the SKU is the common thread.

However, you need clean data. Not all systems enforce SKU uniqueness, and if they don't, you'll discover the problem during migration or sync when you try to create a variation and the SKU already exists. That should be flagged and resolved, not silently overwritten.

Data ownership and flow

When multiple systems are writing to the same eCommerce platform, it's critical to establish clear data ownership. Which system is authoritative for which fields? 

Try visually separating fields in the admin interface. For one standards organization, all ERP-sourced fields are displayed in one tab, and any fields that are enriched or owned by Drupal are displayed in another. Content editors know which fields they can modify and which fields will be pulled during the next sync.

Without this kind of discipline, you end up with integration collisions: the ERP updates a field, then a content editor updates the same field in Drupal, then the next sync overwrites their change. Or worse, two different integrations are both trying to write to the same fields, and neither one knows the other exists.

Bulk Updates at Scale

B2B catalogs aren't static. Manufacturers change pricing schedules. New products get added. Availability changes. Sometimes, tens of thousands of SKUs can be affected at once.

One client processes a file with 80,000 SKUs every week. Other organizations have price update events once or twice a year where they need to update every price in the catalog.

At this scale, naive approaches fail. Processing through the Drupal admin UI risks memory saturation and timeouts. Even batch processing through the standard entity API can stretch for hours when you're updating hundreds of thousands of records serially.

Several strategies make this manageable. 

• Use a dedicated queue processor to handle bulk updates without affecting site performance.
• For very large updates, consider whether you actually need the full entity save cycle with all its hooks and event dispatching or you can safely write directly to the database tables and invalidate the relevant caches afterward. 
• Monitor folder locations for uploaded files rather than relying on browser-based uploads, so the import process isn't dependent on a user's browser session staying open.
• Always process in chunks. Fetch and update 100 or 1,000 records at a time, log successes and failures, and provide visibility into what's been processed and what hasn't.

Permissions and product gating

Product gating, customer-specific pricing, and role-based access are fundamental requirements for most B2B implementations, and they interact with catalog structure in ways that can become very complex very quickly.

For Drupal Commerce, there are almost as many permissions modules as there are developers with an opinion. In other words, there are a lot of options to choose from.

Choose one and lean into it. Otherwise, you’re asking for a maintenance headache and compounding failures.

We've inherited projects with five different permissions modules installed all stepping on each other. These systems are fragile. Fixing one problem may surface another. The solution was to consolidate everything into a single role-based permissions module, extended to cover all the gating requirements. Simpler architecture, simpler UX for administrators, and a system that works reliably.

Avoid nested permissions, where you gate at the taxonomy level, the product level, and the variation level simultaneously. Permissions get confused when different modules are trying to control access at different levels of the hierarchy. If you add group-based access on top of that, the complexity compounds. Think through your permissions architecture at the start of the project, because refactoring it later could mean a complicated migration of access rules.

Key takeaways for product gating:

• Choose one permissions paradigm. By taxonomy, by role, by group, etc.
• Avoid nested permissions.
• Plan from the beginning of your project.

When things go wrong

A sync process will fail mid-run. A deployment will eventually introduce regressions. Your platform will have hiccups, and you’ll need to respond.

The immediate instinct is to rollback, but in a B2B ecommerce environment with active orders and external integrations, rollbacks are almost always worse than fixing forward. Rollbacks lose data. They introduce foreign key conflicts between your system and every external system you've synchronized with. You can't rollback credit card transactions that have already been processed. Orders may have already been submitted that reference new products, or were made with updated product information. Rolling back those product updates introduces a tangled web of invalid orders to sort through.

If you do find yourself in a situation where a rollback seems necessary, at minimum you need to bump all sequential IDs well past the highest ID that existed before the rollback. By a large buffer. For example, if the last product ID was 120,503, make sure to increment to something like 130,503. Same with order IDs. This discipline avoids ID collisions with records that still exist in external systems. Then you'll need to manually reconcile any orders, payments, or inventory changes that happened during the period you're rolling back from. 

It's rarely worth it.

Always fix forward. Diagnose the specific issue causing the most immediate harm and apply a targeted fix. Get the system back to a functional state, even if it's not a perfect state. Then triage the remaining issues by severity and work through them systematically. Update the data that wasn't synced properly.

The prerequisite for fixing forward confidently is having good observability. Good, clean logs and monitoring. If you're in a position where you can't even tell what went wrong, that's a process problem to solve before the next incident.

Best practices to mitigate unexpected failures:

• Deploy after your backup window, not before.
• Fail your builds on warnings so you can check the problem.
• Test on staging.
• Deploy incrementally.
• Go to maintenance mode immediately when issues appear. Don't leave broken sites live while troubleshooting.
• Have a clear escalation path that lays out a playbook for production website incidents.

Getting B2B product catalog management right

B2B product catalog management is fundamentally an architectural challenge. The decisions that have the biggest impact are made before a single line of code is written:

• How you structure your product data
• Which fields are structured vs. generic
• Which system is authoritative for which data 
• How you handle synchronization at scale
• How you manage permissions and access.

Get these right at the start, and you build a foundation that supports growth. Get them wrong, and you can spend years mitigating performance problems, cleaning up data inconsistencies, and patching fragile permission schemes.

The technology and platform does matter. Drupal Commerce, for example, can handle both B2B and B2C implementations at scale (and at the same time), and can digitize your existing processes and operational framework. Other platforms are less accommodating.

But what’s more important is that you make clear-headed decisions based on your business and customers. Document them. Have discipline in enforcing them. Your future self will thank you.