Boozt’s new EDI System: Transparent and Adaptive to business needs
Discover how Boozt revamped its operations with a new EDI system, prioritizing transparency and agility. Learn about challenges like designing a clear data model and adapting vendor-specific transformations, and how these changes boosted efficiency and empowered users, driving e-commerce growth.
By: Martin Nilson, Team Smurf Lead/ Brand Data Engineer at Boozt
For Boozt, Electronic Data Interchange (EDI) has been pivotal in our data management and operational strategies for more than a decade. This technology enables the efficient and precise exchange of business documents in a standardized electronic format, enhancing key processes like product data management, order handling, invoicing, and shipping. Boozt’s consistent use of EDI underscores our dedication to harnessing digital solutions for improving business communication and operational efficiency. As we have grown from a small enterprise to a prominent e-commerce leader in the Nordics, our EDI solutions have evolved in sync, adapting and evolving to support our growth and address the increasingly complex demands of the e-commerce industry.
The Evolution
What we initially built over a decade ago was akin to the foundation of a building — unseen but fundamental, it enabled our business to autonomously import data. Over time, our system has adeptly scaled from managing modest data volumes to processing significantly larger datasets, reflecting our expansion and the growing complexity of our operations. The primary challenge in our EDI process has always been twofold: extracting data correctly and transforming the data to meet Boozt’s specific data standards.
Recognizing the limitations of a system that is working much in the unseen, we saw an opportunity to enhance our business operations by bringing our data handling into the open. Our ambition was to cast a light on how we process the data we receive, bringing every detail into sharper focus for all stakeholders within Boozt. This shift toward transparency would deepen the understanding and connection with our data, and open doors to collaboration and insight, allowing individuals across Boozt to engage with and shape our data’s journey.
We re-envision our EDI solution, rebuilding it from the ground up with a focus on transparency. This new approach involves openly displaying how we select and transform data. To facilitate this, we moved out all data handling from backend code to settings in a web-based interface, making everything readily visible and modifiable, thereby enhancing the transparency and accessibility of our data processing methods throughout the organization.
The Technical Challenges
Transitioning functionality from code to database-stored settings required faith and careful planning to address our system’s needs.
Our initial challenge involved constructing a clearly defined data model that could fully represent the information in any EDI file. This process entailed identifying loops, determined by certain segment tags like CPS, LIN, PRI, NAD, break conditions for the loops, and different parts within the EDI file like segments, elements, and components with correct interlinking. By representing the EDI file as an object rather than as plain text, we could more easily interact with it and map out components representing data points and conditions for the data points. This enabled us to save the indices of elements and components, categorizing them as data points or conditions for data points, and linking them to data identifiers — the meaning of the data point.
A subsequent challenge arose when transforming data, and tailoring it to our internal data requirements. The goal was to track how data points were transformed by the system. To achieve this, we introduced a new data object that would store the original data point, a normalized value, and a transformed value. We would also log all modifications to the value and require all modifications to have a stated reason for why it was applied. The purpose of the normalized value was to have a clear view of changes that apply universally (e.g., standardizing various representations of ‘One Size’ to a single format). The transformed value was created to track vendor-specific changes. By retaining and displaying these different values within the user interface, we provide users with a comprehensive view of the data’s progression through the system.
We aimed to move the vendor-specific transformations from backend functions, accessible only to coders, to configurable settings in a web interface. The key challenge for accomplishing this lies in adapting complex backend functions to something that can be easily configured in a user interface. We chose to leverage the power of regex patterns and abstract them for the user to achieve this goal. This technique involved storing regex patterns and their corresponding replacement values in the database, associating them with the specific data identifiers targeted for transformation. Users could then stack these patterns for more complex operations.
This approach ensured immediate, interactive feedback within the UI, displaying how data points evolve with each transformation, giving the user the flexibility at each modification to confirm or modify the transformation to meet their objective. To enhance comprehension of the transformations each transformation was required to be accompanied by an explanatory note, to help guide other users to its purpose.
Practical example: To reformat “Cotton 90%, Wool 10%” into “90% Cotton, 10% Wool,” the system leverages preg_replace with regex pattern /([a-z\s]+)\s(\d{2,3}%)/i and $2 $1 as the replacement value. The user can layer conditions for this transformation, using preg_match; such as applying these changes exclusively when the Product Category equals ‘Jackets’.
The Result
Boozt’s upgraded EDI system has markedly improved both the efficiency and agility of our data management workflows. By transitioning from a backend-focused, code-heavy framework to a user-centric interface, we have achieved a system that keeps our codebase streamlined and stable, ensuring that it can handle increases in vendor integrations and new data points without structural changes.
The emphasis on a robust user interface facilitates swift and straightforward modifications directly by end-users, bypassing the need for complex code deployments and extensive approval processes. This not only speeds up response times to new business requirements but also broadens system accessibility, lowering the technical knowledge required to make effective use of the system. By enabling more team members to directly interact with the EDI settings, the system enhances organizational agility and empowers our staff to implement changes rapidly and confidently.
Conclusions
Boozt’s newly implemented EDI system marks a significant stride towards a more agile, transparent, and efficient data processing environment. The system’s architecture cleverly eliminates the need for complex backend modifications, thereby fostering a sustainable model as the company continues to scale. The transparent data handling mechanism empowers our teams to manage and troubleshoot data more effectively, ensuring that operational disruptions are minimized and resolved swiftly.
Moreover, the flexibility introduced through the configurable UI significantly enhances our responsiveness to changing business needs and vendor requirements, positioning Boozt as a nimble player in the competitive e-commerce landscape. As we reflect on the journey from a rigid, code-heavy system to one that champions user empowerment and transparency, it’s clear that this transformation not only supports our current operational needs but also sets a solid foundation for future growth and innovation. This evolution underscores our commitment to leveraging technology to streamline operations, improve stakeholder engagement, and drive business success in an ever-evolving market.
Author: Martin Nilson, Team Smurf Lead/ Brand Data Engineer at Boozt
Boozt: Website, Career page, LinkedIn
