Introduction

As a tech leader who’s spent years building software in high-stakes environments, I’ve watched the rise of vibe coding with equal parts excitement and caution. In this post, I want to demystify what vibe coding really is, show where it can turbocharge development, and also lay out why it’s not a cure-all — especially not in enterprises like banking where mistakes are costly. I’ll share some personal experiences from my role as CPTO at Zafin, where we’ve been actively experimenting with AI-assisted development. Our AI journey has resulted in a company-wide AI rollout, both in our product engineering and company operations, and has showed me how, and where, to harness the “vibes” in a pragmatic, responsible way.

What Is “Vibe Coding,” Exactly?

Crucially, vibe coding is not just using AI to help code — it’s handing over the keyboard (figuratively) and trusting the AI to handle the details. A vibe coder might copy an error message into the AI and ask for a fix, rather than digging through stack traces themselves. They might bolt on a new feature by prompting the AI, even if they don’t fully understand the code the AI wrote last time. The codebase grows and evolves, often beyond the developer’s personal understanding.

If you’re thinking this sounds a bit wild, you’re not alone. In traditional software engineering, we pride ourselves on understanding our code inside-out. Vibe coding breaks from that mindset. It’s about speed and vibes over meticulous control. It’s the ultimate embodiment of the “move fast and break things” ethos, perhaps too fast and ready to break things, as we’ll discuss. So, who benefits from this approach, and what’s its future? Early enthusiasm suggests vibe coding can make software development more accessible and lightning-quick, but seasoned engineers (myself included) have some hard questions about maintainability and correctness. Let’s look at both sides.

Where Vibe Coding Shines: Rapid Prototyping and UI “Magic”

Despite the controversy, there’s a reason vibe coding has caught on so quickly. It delivers speed. Tasks that used to take days or weeks of coding can sometimes be accomplished in hours with the right AI assistance. For early-stage ideas, hackathons, or any scenario where getting a working prototype matters more than perfect code, this approach can feel revolutionary.

Imagine a product manager with a bold idea for a new feature, or a startup founder who doesn’t code. In 2023, they might have been stuck waiting for engineering resources or struggling with a drag-and-drop app builder. In 2025, they can fire up an AI coding tool, describe what they want in plain language, and watch a basic app materialize. I’ve seen non-engineers create simple web apps by just chatting with an AI about what they need. Large Language Models (LLMs) have effectively democratized prototyping, allowing anyone comfortable with a computer to build basic digital products.

Even for experienced developers, vibe coding can accelerate tedious work. User interface (UI) development is a great example. UIs often involve a lot of boilerplate and repetitive patterns. With vibe coding, a developer can say, “Make a responsive login form with email/password fields and a ‘Remember Me’ checkbox,” and get the full React/HTML/CSS code generated in seconds. If the design needs tweaking, maybe the button should be blue and aligned to the right, they can just tell the AI to adjust that. It’s a far cry from manually wrestling with CSS for hours. For early-stage projects or internal tools, this speed is a game-changer. You can iterate on ideas much faster when you’re not spending 80% of your time on setup and syntax.

My team’s experience at Zafin drove this home. We set out to build an MVP of a new product. Traditionally, this kind of project would be scoped for a small team over a couple of months given the design, front-end, back-end logic, integration, testing, and so on. Using an AI-assisted approach, we got a working prototype up and running in about 5 person-days, essentially one week. That’s a massive acceleration. It wasn’t fully production-ready, but it proved the concept and let us demo live functionality for stakeholders. The AI handled a lot of the grunt work which freed our engineers to focus on refining the business logic and user experience. For prototyping purposes, the “vibes” were very good indeed.

Why You Can’t Only “Vibe” in Enterprise Software (Maintaining Rigor in Regulated Industries)

Given how empowering vibe coding can be in early development, it’s tempting to ask: Why not use it for everything? Here’s where the other shoe drops. In mature software environments — especially enterprise and regulated industries like banking — maintainability, auditability, and correctness are absolutely non-negotiable. And these are precisely the areas where reckless vibe coding runs into trouble.

Professional software engineering isn’t just about getting things working quickly; it’s about building systems that keep working, correctly and securely, as they evolve. That means humans need to understand and trust the code. When an AI is generating code on your behalf, you still carry responsibility for what that code does. In a bank, for instance, every change to the codebase might need to go through code review, QA, and even compliance checks. If a developer says, “Well, I let the AI write this chunk, I’m not really sure what it’s doing under the hood, but it seems to work,” that’s a huge red flag. Auditors and regulators will not accept “it mostly works” as an answer. They need to know that proper controls are in place and that someone can explain how the software meets requirements.

Maintainability suffers if no one on the team truly understands the code’s logic. It’s only fine until something goes wrong. If your team treated the codebase as a black box generated by an AI, they’ll struggle when a severe bug or outage hits. Debugging complex issues requires understanding the system’s internals, no matter how it was originally produced.

There’s also the matter of correctness in the sense of business logic and compliance. AI doesn’t inherently know your industry’s regulations or your company’s specific policies — unless you explicitly guide it, and even then, it might not get all the nuances. In a financial application, a subtle mistake can lead to mis-calculated interest, incorrect fees, or worse, a compliance violation. A human developer might catch a suspicious calculation or an edge-case scenario that looks off, but an AI writing code from a prompt won’t inherently flag those domain-specific issues.

Security is another huge concern. Code that “mostly works” might still hide dangerous vulnerabilities. In fact, early experiments with AI-generated code have shown a tendency to produce insecure code if not prompted carefully. AI will cheerfully produce code that runs, even if it’s doing something like hard-coding credentials, ignoring error handling, or using an outdated cryptography method. A novice vibe coder, excited that the app runs, might have no idea that they’ve just opened a security hole large enough to drive a truck through. Enterprise developers must be far more paranoid. We need to ask: Does this API call properly authenticate? Did the AI just expose something it shouldn’t? Are we logging sensitive data? You can’t just assume the AI “knows” what it’s doing in these matters. Security and compliance reviews cannot be skipped, no matter how fast we generated the code.

In short, the more critical the software, the more critical human oversight becomes. Vibe coding flips the traditional ratio — less time coding, more time reviewing and testing. The vibes alone won’t keep customer data safe or the bank in compliance.

Our Journey with AI-Assisted Coding at Zafin: From Pilot to Playbook

To demonstrate a pragmatic approach, let me share how we’ve adopted AI coding assistance in my organization. Zafin provides SaaS solutions for banks, which means our software must meet high standards of security, correctness, and auditability. We saw the potential in AI programming tools early, but we also knew we’d have to integrate them carefully. Here’s how we did it:

1. Starting Small with a Focused Pilot: Rather than unleash AI coding on our entire engineering team overnight, we began with a pilot program. We invited about 25 engineers (diverse in skill levels and roles) to try out an AI-powered code editor called Cursor. For context, Cursor is an AI-native IDE based on VS Code, with built-in AI assistants (using powerful models like Anthropic’s Claude) to help write and refactor code. Our goal was to see how real developers could leverage it on real tasks, and identify where it helped or hindered. We deliberately chose a non-mission-critical project for this pilot — in our case, that MVP I mentioned earlier. The team’s mandate was: build it fast, but document everything. We treated it as an experiment in boosting productivity. The result, as I noted, was astounding: a functional MVP in 5 days instead of the usual few months. That early win built confidence that AI could dramatically accelerate development, at least for prototype-scale projects.

2. Gradually Scaling to the Wider Team: Off the success of the pilot, we expanded access to Cursor and AI coding to more engineers. Over the next quarter, we ramped up from 25 pilot users to over 150 engineers — a majority of our development organization. This wasn’t a free-for-all; we rolled it out in phases, checking in frequently via surveys and team debriefs. One interesting insight was usage patterns: some developers integrated the AI into their daily flow for things like writing unit tests or stubbing out functions, while others were initially hesitant, unsure if it would truly save them time. We encouraged peer learning, where early adopters shared tips (for example, how describing the problem in a certain way got better code results, or how to use the tool’s refactoring commands). This bottom-up knowledge sharing was crucial in driving adoption. As more people saw colleagues succeeding with AI assistance, the skeptics began to give it a try. Within a few months, AI pair programming went from a novelty to a normal part of our development culture.

3. Developing a Playbook and Best Practices: As we scaled our AI-assisted development, we knew that clear standards and guardrails would be essential. Alongside our rollout of Cursor, we created an AI Coding Playbook, a set of evolving guidelines grounded in real-world usage and feedback. But beyond best practices, we took it a step further by implementing Cursor rules to ensure that AI-generated code conforms to our coding standards and architectural patterns by default. These rules have become a powerful lever for us. They help drive customized AI behavior, ensure consistency across the codebase, and embed things like directory structures, naming conventions, and compliance checks directly into the development environment. As a result, we’ve seen improvements in productivity, onboarding speed, and the context-awareness of AI assistance. For example, we defined explicit rules for database migrations, such as where files should live, naming patterns, and structure enforcement. This means the AI won’t just write code that compiles, it will write code that aligns with our architecture. Our Playbook complements these rules by offering practical guidance: when not to rely on AI (e.g., for security-critical logic), how to prompt effectively, and how to review AI-generated code with the same rigor you would apply to any teammate’s pull request. We also address data security — what internal information can safely be included in prompts and what must be excluded. The combination of policy and automation, the Playbook and Cursor rules, has allowed us to integrate AI into our workflows responsibly and at scale. It’s not just about moving faster, it’s about moving smarter, with standards baked into every suggestion the AI makes.

Let me summarize a few key takeaways from our adoption of AI coding at Zafin:

• Start small and learn fast: Our initial pilot with a contained team and project was crucial. It let us prove value quickly and establish rules and procedures before scaling up. This incremental approach built trust across the organization.
• Keep humans in the loop: We treated AI like a super-smart assist, not an autonomous coder. Every line of AI-written code was reviewed, tested, and sometimes rewritten. Developers remained responsible for the final output, which kept quality high.
• Create guidelines (and update them): Having a playbook of do’s and don’ts helped align everyone on best practices. As the tech evolves, we continue to refine our guidelines. This living playbook ensures we maintain standards without stifling the agility AI offers.
• Listen to your team: We made decisions like discontinuing GitHub Copilot based on direct feedback from engineers. Their on-the-ground experience is invaluable for understanding what actually helps or hinders productivity. By involving them in the process, we got buy-in and uncovered insights leadership alone might miss.

Overall, the organizational impact has been strongly positive. We’ve seen a significant reduction in development time for many tasks and an uptick in developer happiness when they can focus on creative problem-solving instead of the boilerplate. Interestingly, some engineers reported that working with the AI forced them to articulate problems more clearly (since you have to explain to the AI what you want), which in turn improved their design thinking. That’s a side benefit I hadn’t anticipated.

Conclusion

At the end of the day, someone has to be accountable for the software products we build, and that’s still us, the humans. By embracing AI with eyes wide open, we can harness the “vibes” to build faster and smarter, without losing sight of the responsibility that comes with writing software that matters. As a leader in a regulated industry, I’m proud that we didn’t shy away from AI out of fear; instead, we leaned in and established a blueprint for how to do it right. In my view, that’s the path to staying ahead: boldly adopt new technology, manage the risks intelligently, and always keep the end-goal in focus — delivering reliable, innovative solutions for your customers.

“Vibe coding is not a target destination but it has entered the heterogenous mixture of the software development lifecycle and increased the accessibility to a much larger group of people to enter and contribute to the flow (as long as the proper guardrails are in place to check, review, and allow),” reflects Branavan Selvasingham, Head of AI at Zafin.

The vibe is real, and so are the results, as long as we lead with both enthusiasm and prudence.

Vibe Coding in the Real World: Embracing AI’s Speed While Managing Its Risks was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

If you haven’t already, we recommend reading our initial article on Zafin’s BIAN adoption first. In this next article, we’re going to explore the blue layer of BIAN’s Architecture Layers View where we’ll be discussing real APIs.

Zafin has made 3 APIs that operate the following 4 BIAN Service Domains:

• Product Directory
• Party Reference Data Directory
• Sales Product Agreement
• Customer Product And Service

Below are the three Zafin APIs to operate these service domains:

This article explains the Zafin Product Directory API and its underlying control record. It serves as a prelude to the third and main act, our piece on the Sales Product Agreement API.

Origin

Zafin’s main asset is its billing engine that calculates fees, discounts and/or penalties on deposit products sold by a bank. For a bank to configure the pricing parameters to conduct billing, our platform contains a Product Catalog. Once new pricing parameters have been approved by a checker of the bank, our product catalog sends a message to our billing engine for the updated pricing to take effect. We use that same message to populate our Product Directory software. This is for a bank to retrieve products, attached fees, rates or discounts, via BIAN inspired APIs.

We used BIAN’s Product Directory BOM, and its helper diagrams to design the Zafin Product Directory Data Object Model. We also used BIANs Product Directory semantic API as a template for our API.

Whenever we release a new version to banks for this API, we deliver the following artefacts:

• A Data Object Model representing our control record as a document that every human developer with little BIAN experience can read.
• A API Swagger File per Service Domain for the contracted endpoints.
• A Type Value model that specifies all enumerations in use by the APIs.

The Data Object Model (DOM)

Our Product Directory DOM builds upon BIAN’s Product Directory with several key additions.

At the product level, we introduced a new object: Product Category. This object acts as an intermediary between the Banking Product and the Directory objects. In contrast with BIAN’s guidance, we aim to prevent direct associations between products and directories.

To illustrate the practical application:

Imagine a Tier-2 bank with a retail presence in 10 countries, looking to configure products for multiple brand labels within a single Zafin Product Catalog. Each brand logo would become a Directory entry in the Zafin Product Directory. The bank would define at least one Category (e.g., “Retail”) and associate relevant products with it.

Banks often use multi-level product hierarchies — some organize products into families, others into groups or classes. To support flexible, n-level hierarchies and allow products to belong to multiple categories (e.g., a credit-line facility offered under both SME and Self-Employed segments), we introduced this deviation from the standard BIAN model.

Taking a closer look into Product Features, more drastic changes to BIAN had to be made. This was to create a data model that is performant, maintainable and foremostly, integrable with our existing platform:

• Product Feature became a single object without explicit subtypes.
  In our model, Product Features can be Fee Plans or fee-items, Interest plans or items, Disclosure references or Product Services, like app-access. Where BIAN applies subtypes to almost everything, we found that subtypes in the physical schema, became unmaintainable and sluggish.
• BIAN uses a concept called Fee Arrangement to store Fee Amounts, or Rates, in Product Directory. BIAN associates the Fee Arrangement to a “Charge Bearer” (you and me for our monthly daily banking fee) via “Arrangement Involvement” (see figure below). That didn’t work for us. Corporate banking customers can have fee plans that contain dozens of fee items, and it became too complex to associate all these individual items to customers directly.

• Furthermore, fees can change over time or be subject to conditions. For example, my $17 monthly subscription fee is waived if I maintain a $4,000 daily closing balance. The temporality of fees and linking limit-conditions that apply to fees became too complex for us to manage using BIAN’s specified approach.
• Instead, we extended Product Feature Modality with pricing parameters such as Fee Amount, Interest Rate and Discount Percentage (see below diagram). By linking Product Arrangements to Product features (such as a fee, or interest-plans) via Arrangement Involvement, we stayed true to BIAN. We later learned from some of our clients that are BIAN-inspired have made these exact same modifications to their software.
• We’ve added versioning to Product Features. Our Product Catalog allows product managers to create future-dated fee plans (features) in so-called “temporals”-think: $0 for the first 4 months, then $20/month after that ‘. We isolated versions in separate “State”-objects with their own lifecycle status, effective dates, and end dates. This sets us up for future graph-based reads. Light blue objects represent these versioned subtypes.

On Product Feature level, our next major version (v3) will evolve to the following similar format:

Type Values

We consider type values (or enumerations for the API community) to be bank-specific configuration. That is why we don’t document enumerations in the swagger itself but instead we maintain Type Values, as a separate artefact. In upcoming releases, we aim to further isolate Type Values in a separate Service Domain (Public Reference Data Management).

The API

Our API-payloads and the schema-sections therein, resemble our Zafin DOMs (thus our control records), to the largest extent.

A few exceptions have been made in favor of lineage to the BIAN idiom, but those exceptions are rare. Additionally, our diagrams are plain text plantuml files which allow us conversion to any other idiom. ChatGPT is our new best friend when it comes to maintaining a crisp trace from BIAN BOMs to Zafin DOMs and API schemas, for human or artificial intelligence to contribute to.

We do not expose our internal behaviour, such as primary keys or inactive versions, in our control record unless required — we focus on designing outside-in.

Our APIs are meant for the omni-channel to use, and to the extent possible, our APIs remain customer (and employee)-journey-agnostic.

Conclusions

We hope to have made clear that some of the BIAN concepts are hard, if not impossible to implement in real software. Zafin (and possibly also BIAN-adopting banks) would like to contribute back to BIAN, on mismatches between the Business Architecture and the Information System Architecture. Some of these mismatches (i.e. the pricing parameters in Feature Modalities instead of Fee arrangements) impede the easy substitution of one vendor with another and hinders the overall integration between banks and vendors.

Combining the conceptual BIAN models with the power of Artificial Intelligence is speeding up our software delivery pipeline in unprecedented ways.

Some Human Intelligence to design real software from the BIAN models will always be required we think since BIAN can’t possibly be aware of local implementation constraints. The more we can make a real standard across banks based on BIAN, the less customization per implementation is needed. This will also allow for less integration effort and thus a cheaper, less error-prone solution overall.

References

• BIAN. (n.d.). BIAN Service Landscape. Banking Industry Architecture Network. Retrieved from https://bian.org
• BIAN. (n.d.). BIAN Business Object Model (BOM). Banking Industry Architecture Network. Retrieved from https://bian.org
• Hasura. (n.d.). Instant GraphQL APIs on your data. Retrieved from https://hasura.io
• Zafin. (n.d.). Zafin’s API Documentation and Product Offerings. Retrieved from https://zafin.com

BIAN Adoption at Zafin (Part 2): Product Directory was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

In Part 1 of this series, I walked through what it means to be a CPTO — the hats I wear, the situations I manage, and why this role is uniquely positioned to drive alignment between vision, execution, and customer value. For us at Zafin, that alignment is critical. The product we’re building is deeply technical, the space we operate in is highly regulated, and the clients we serve expect precision and performance at scale. The CPTO role was our answer to those demands.

So, how did we land on this model? And what has it actually unlocked for us in practice?

In part 2 of this 2-part series, I’ll take you behind the decision — why the CPTO model made perfect sense for Zafin, how it’s helped us move faster and smarter, and what challenges we’ve faced (and learned from) along the way. From platform complexity to customer intimacy to organizational clarity, I’ll share the reasons we combined the CTO and CPO roles — and why we’d do it again.

Why We Combined the CTO and CPO Roles at Zafin

In our case, a few factors made the combined role particularly appealing:

• A highly technical product: Zafin’s platform is inherently technical — we’re talking about cloud-based software that integrates with banks’ core systems, handles complex logic, and meets stringent security and uptime requirements. When your product is deeply technical at its core, merging product and tech leadership can bridge the gap between product vision and technical feasibility, and help ensure our product roadmap is ambitious and grounded in technical reality
• Need for tight collaboration and speed: Innovation and time-to-market are critical, especially as we help banks modernize. Combining product and tech roles is a way to dismantle silos and accelerate decision-making.
• Unified strategic vision: Like many companies our size, we want clarity in our long-term strategy. Having product and technology under one umbrella ensures that our technology strategy fully supports our product vision, and vice versa. There’s no “daylight” between what we promise our customers and what our platform can deliver — because one team (with integrated product-engineering DNA) defines and builds it.
• Organizational simplicity: An added practical benefit is clear accountability. Instead of two C-level leaders juggling overlapping responsibilities, one executive is accountable for the end-to-end success of the product. Our CEO and board know exactly who to come to for anything from roadmap updates to platform reliability. From a CEO’s perspective, having one throat to choke (or one partner to strategize with) on product development matters can simplify leadership dynamics. It certainly streamlines our internal communications — no need for separate product and tech status updates when they’re inherently one conversation.

At its core, we chose to combine the CTO and CPO roles to tighten the alignment between what we build, why we build it, and how we build it. We believed that by fusing these leadership roles we could better serve our customers, and move faster in a rapidly evolving market.

Challenges We’ve Faced — and Why the CPTO Model Works

Combining the CTO and CPO roles is not without challenges. The CPTO role demands breadth, stamina, context-switching on steroids, and the ability to lead through both vision and execution. Here are some of the challenges I’ve encountered in the seat — and how we’ve turned them into strengths.

1. The Scope is Large — and Unrelenting:

The CPTO role puts you at the center of two of the most dynamic and high-pressure areas in the business: product and technology. That means you’re responsible for customer outcomes and platform uptime, for roadmap delivery and scalability. There’s no place to hide. It’s thrilling — but it’s also cognitively demanding. The risk is spreading yourself too thin and losing the strategic altitude the role requires.

How we manage it: I’ve invested heavily in strong second-line leaders — people who own and drive execution while keeping me focused on the higher-order priorities. I’ve learned that in this role, delegation isn’t a luxury — it’s survival.

2. The Bias Trap is Real:

Every CPTO brings a natural tilt — usually toward the function they came up through. My background is technical, which means there’s a gravitational pull toward architectural decisions, dev tooling, and infrastructure strategy. If left unchecked, that pull can overshadow product discovery, customer research, and design.

How we manage it: Self-awareness is key. I intentionally block time for user feedback sessions, roadmap reviews, and research. I surround myself with exceptional product thinkers who challenge my assumptions and keep the user front and center. We’ve built in balanced dynamics — like a ‘2-in-a-box’ ownership model for all of our products between product managers and engineering leaders — to ensure both sides of the business are getting equal oxygen. It’s a constant balance. But when you get it right, it creates incredibly well-rounded thinking — and the teams feel that.

3. You Have to Be Fluent in Two Languages — At All Times

One of the more subtle challenges of the CPTO role is communication. In any given week, I’m speaking to engineers, designers, product managers, the CEO, clients, sales, security teams, and regulatory stakeholders. Each of these groups speaks a different language. Part of my role is acting as a real-time translator — bridging vocabulary, incentives, and mental models to keep everyone aligned. It’s not just about simplifying tech for business, or explaining strategy to engineers. It’s about aligning everyone to a unified narrative about where we’re going and how we’ll get there.

How we manage it: We emphasize context. Whether it’s an architecture review or a product demo, we frame everything around business impact and customer value. And because the CPTO leads both functions, we can reinforce a single source of truth. Our platform vision isn’t fragmented — it’s integrated. That pays dividends in how we communicate, plan, and execute.

4. Prioritization is Relentless

The hardest part of the job? Saying no. Every quarter, there are more great ideas than capacity. And unlike a split model, where product and tech leaders can advocate independently, the CPTO is the one making the trade-offs. Do we push for a bold new feature that excites clients? Or invest in modernization to de-risk scale?

How we manage it: We constantly evaluate trade-offs in the open. The benefit of having product and tech under one roof is that these trade-offs happen faster and with greater integrity. There’s no turf war — just clarity. But that doesn’t make the decisions easier. It makes them more consequential.

5. It’s Easy to Become the Bottleneck

Ironically, the very alignment that makes the CPTO role powerful can also make it dangerous if you don’t manage scale. When everyone — from sales to engineering — knows that product and tech decisions flow through one role, there’s a temptation to route everything through the CPTO. That can slow things down, create decision fatigue, and limit team autonomy.

How we manage it: We’ve worked hard to create a structure where the CPTO provides direction, not dependency. Our squads operate with autonomy, our leaders are empowered, and we’ve defined clear swim lanes for decision-making. My job is to enable momentum — not to control every call.

Conclusion: The Future of Product-Tech Leadership

My journey as a CPTO at Zafin has been one of the most challenging and fulfilling chapters of my career. By combining product and technology leadership, we unlocked a new level of agility and cohesion in our organization. We’ve seen faster innovation cycles, more consistent execution, and a culture that truly fuses product vision with technical excellence.

Looking ahead, I suspect the industry will see more hybrid leaders like this, especially as software becomes more central to every business and the lines between product and technology continue to blur. Modern software practices — agile, DevOps, cross-functional teams, continuous delivery — naturally break down the old silos. In many ways, the CPTO role is a reflection of how product development itself has evolved to be more integrated. I foresee roles like CPTO becoming more common in tech-driven enterprises where alignment and speed matter greatly. We’ll likely see new generation leaders who consciously develop both skill sets.

For my peers in the industry — whether you’re a CTO, a CPO, or an aspiring CPTO — the key takeaway is the value of tight partnership between product and engineering. Title aside, what truly drives success is that unity. The CPTO role at Zafin has taught us to always keep customer needs and technical feasibility in lockstep. That’s a lesson I believe will define the future of product-tech leadership: leaders who can wear multiple hats and bridge worlds, whether formally or informally, will be in high demand.

Part 2: Bridging Product and Technology with the CPTO role was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

At Zafin, our mission is to help banks along their modernization journey, shaping the future of how banks price, package, and deliver value to their customers. That demands a product strategy that’s tightly coupled with architectural discipline, platform innovation that’s grounded in real customer need, and execution that’s fast, scalable, and rock solid. And that’s exactly what the CPTO model enables. As the Chief Product and Technology Officer (CPTO) at Zafin, I have the unique privilege — and challenge — of wearing these two hats at once.

In this 2-part series, I want to share why Zafin chose this path, how we’ve structured the role, what we’ve learned, and — most importantly — how this model is helping us deliver stronger outcomes for our clients and move faster in a rapidly changing industry. I’ll share my thoughts on why I believe, in a modern, cloud-native SaaS platform like ours, in a mission-critical industry like banking, combining product and technology leadership under one roof isn’t just smart — it’s essential. If you’re a product or technology leader (or both!), I hope these insights help you navigate the evolving intersection of product and tech leadership in your own organization.

Part 1: One Role, Two Hats; exploring what the CPTO role entails

As CPTO, my job is to lead our end-to-end product development function — from market insight and customer discovery to technical architecture, software delivery, and post-release evolution. Instead of having a Chief Product Officer (focused on the why and what of the product) and a Chief Technology Officer (focused on the how of implementation), a CPTO is responsible for all of the above — the full spectrum from product vision to technical execution. It’s a broad remit, but one that creates real focus.

I wear five main hats:

1. Strategic Vision — Setting the north star for our product and platform. Where’s the market going? How do we get ahead of it? What investments today will differentiate Zafin tomorrow?

I am responsible for defining what we build and why. This means working closely with our clients, market research, and our sales teams to shape our product vision. I set the roadmap priorities: which features add the most value to customers? What market problems should we tackle next? All to ensure that our product evolves in the right direction to meet customer needs and business goals. For example, in the past year we’ve introduced capabilities like Dynamic Cohorts (for agile customer segmentation) and Zafin Canvas (a low-code integration tool) to address specific pain points banks have around personalization and integration. Those initiatives started with a strategic decision that “yes, this is what we need to build to deliver value to banks that’s sustainable and rewarding” and that decision ultimately sat with me in the CPTO role. A product leader’s mindset is crucial here — customer-centric, value-driven, always asking “how will this make our customers’ lives better or our product more competitive?”

2. Technology Stewardship — Owning platform architecture, security, resilience, and scalability. Our promise to clients is enterprise-grade delivery — and that requires deep, ongoing technical leadership.

I’m also accountable for how we build and deliver these product visions. This is the CTO side of the house. It involves defining our architecture, choosing and actively reviewing the right tech stack, ensuring scalability, security, and performance, and generally making sure our engineering approach can execute the product strategy. In practice, I might be reviewing our cloud infrastructure one hour and discussing API design guidelines the next. A big part of my job is to make sure that our technology roadmap aligns with and enables the product roadmap. For instance, if we plan to launch a product, I need to ensure we have the infrastructure in place ahead of time. Technical debt management and platform resilience also fall in my bucket — deciding when to build new features vs. when to shore up the foundation. In other words, I act as the guardian of our platform’s health while it rapidly evolves.

3. Customer Advocacy — Spending time with banks, understanding their challenges, and translating insights into actionable roadmap priorities. Zafin builds with our clients, not just for them.

As CPTO, I am able to speak to customers with a holistic view — I can talk about the product roadmap and features in one breath and discuss integration details and technical performance in the next. Customers don’t care about our internal org chart; they care that their needs are heard and addressed. In my role, I aim to be a bridge between our clients’ business objectives and our technology solutions. This tight coupling with customers feeds back into better product decisions (nothing beats hearing feedback straight from the source) and technical decisions (like knowing which performance improvements will matter most to clients). It also builds trust — our clients know that the person setting our product direction also deeply understands the tech under the hood, which gives them confidence in our execution.

4. Drive Execution — Ensuring we deliver high-impact features on time and with quality. This includes structuring agile teams, balancing innovation with stability, and making difficult trade-offs.

Owning both product and tech means I’m ultimately responsible for delivering on our plans. Strategy is useless without execution. I oversee cross-functional execution teams and drive an agile delivery process to hit our targets. This includes setting up the right team structures and processes. I spend a lot of time ensuring that our cadence of releases is steady and that we’re meeting quality benchmarks. When we promise a new feature to a bank by Q4, I’m on the hook to make sure all the gears — product design, coding, testing, deployment — mesh together to make that happen. There’s a general manager aspect to the CPTO role — you’re not just dreaming up ideas, you’re responsible for seeing them through to successful outcomes.

5. Culture Champion — Building a world-class product and engineering team that works as one. We’re proud of our high-performance, high-trust culture at Zafin, and it’s a critical part of our success

The CPTO is a people leader for both the product and engineering organizations. At Zafin, I lead a diverse group of product managers, software engineers, architects, UX designers, and more. One of my most important jobs is building a unified culture across these traditionally separate domains. In many companies, product and engineering teams can develop an “us vs. them” dynamic — product wants more, engineering says slow down, etc. In our model, I work intentionally to foster one team with a shared mission. We’ve emphasized principles like ownership, collaboration, and customer-centric thinking at every level. Because ultimately, both product and tech teams share the same goal: create products that our customers love. We celebrate wins together — it isn’t a product win or an engineering win, it is a Zafin win

The most important part? These responsibilities aren’t in conflict. They reinforce each other. Because they live in the same office.

Conclusion

I hope I’ve been able to share a bit about what the CPTO role entails, and where I spend a lot of my time. Please stay tuned for part 2 of this series, where I will dive into why we, at Zafin, specifically chose to combine the CTO and CPO roles.

Part 1: Bridging Product and Technology with the CPTO role was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

I’m Teresa — Group Product Manager at Zafin, where I’ve spent the last four years turning big-bank pain points into products that actually make banking easier. When I’m not wrangling my two small kids, I’m happiest tackling real customer problems alongside people who are smarter than I am. Along the way I’ve helped weave product-thinking into the company’s DNA (and, frankly, my own), and I’m excited to pull back the curtain on what we’ve gotten to so far

When your grandmother asks you what you do for a living, what do you say?

I would say: “You know, grandma, when you’re in the grocery store, just picking up some lemons, you want to pay for them, and you have to stay in line for 20 frustrating minutes? Basically, that was a product manager that did a poor job.”

A product manager’s mission is deceptively simple: make sure real people get what they need from the product. And enjoy the ride while they’re at it.

Those two sentences already trigger six classic questions:

1. Is a smooth checkout really the product manager’s job?
2. Shouldn’t the cashier just be faster?
3. Who exactly are the “users”?
4. How do you know what those users actually want?
5. Where do you draw the line around “the product”?
6. How do you ensure the thing you dreamed up is the thing your team ships?

Over the next few minutes — let’s call it a comfortable commuter read — we’ll unpack each question and show how we tackle it at Zafin.

#1 Welcome to the Fight Club

Rule #1: We don’t pretend B2B and B2C product management are the same.

Software product management is still the new kid on the corporate block. The title showed up in a 1931 Procter & Gamble memo, but the discipline as we know it only matured after the Agile Manifesto in 2001. Since then, flavours have proliferated — none more divisive than B2C vs. B2B.

Zafin is unapologetically B2B: a SaaS platform that serves banks. If you’re building for consumers, you can test on dozens of friends, launch to thousands of strangers, and pivot in weeks. If you’re building for a tier‑one bank, you sell to a committee — Procurement, Risk, Finance, Legal, Ops, Support… and somewhere down the hall, the actual end users.

Different arena, different rules. Success means crafting a product that delights the whole ecosystem, not just the person who logs in.

#2 One Ring to Rule Them All

Whether you’re in B2B or B2C, the true north stays the same: the user. Every Zafin user story starts the same way: “As a [very specific persona] I want to [action] so that I can [goal].”

Personas aren’t simple stick figures. They’re living dossiers: job, KPIs, caffeine source, tools they curse at, incentives they chase, fears they whisper. You can’t Google this stuff — you have to talk to humans. (LinkedIn + polite curiosity + one Estonian friend of a friend = interview gold.)

That legwork pays dividends later: users rarely articulate what they need, they talk about what they want. Your job is to read between the lines. A relationship manager who complains that monthly fee adjustments take all Friday doesn’t want a faster spreadsheet — they want their weekend back.

When the product ships, the One Ring doesn’t magically switch fingers. If users adopt new workflows, new AI buddies, or face new regulations, the product must evolve with them — or evaporate. Lose sight of the user and even the prettiest backlog turns into Gollum’s cave.

#3 “You Either Die a Hero, or You Live Long Enough to See Yourself Become the Villain”

Welcome to Gotham: every feature will die someday. The trick is to let it die a hero, not a budget‑sucking villain. Our anti‑entropy mantra is Design → Sell → Build → Repeat.

Design. Start embarrassingly small — a whiteboard sketch, a Lovable.dev flow, anything that makes the idea tangible. Run lightning interviews, grab coffee with Sales, and steal thirty minutes from an architect.

Sell. Put the napkin in front of someone authorized to pull out a purchase order. In B2B, one serious buyer beats a thousand Medium claps. It didn’t sell after trying hard enough? Congratulations — that’s where the product stops, and you’ve saved months of engineering burn. Prototype 0.1 rests in peace.

Build. Got your anchor client? Great. Spin up the squad, translate promises into tickets, and ship a usable slice. Enterprise clients don’t mind MVPs; they mind vaporware.

Repeat. First release? Pure prologue. Gather telemetry, run retro interviews, and loop again. Captain America had it right: “I can do this all day.” Our internal motto: “The first iteration is exactly that — a first iteration.”

And yes — every product still dies someday. Your real super‑power is knowing when to double down and when to eulogise.

#4 “There’s No Fate But What We Make for Ourselves”

A product manager without delivery chops is just a motivational poster. Requirements, strategy, and wireframes are half the job; the other half is rolling up your sleeves and shipping.

At Zafin, PMs double as product owners. They sit in stand‑ups, parse architecture diagrams, and know which APIs come out of the box. Why? Because strategy meets reality in the sprint backlog. A missed dependency today is a blown release date tomorrow.

It’s a learning curve — my background is pure business — but every “What database are we on?” chat makes the product sharper. Engineers light up when they see their work move the needle for an actual user, and nothing unites a squad faster than a PM who’s in the trenches.

#5 On AI — “Do or Do Not. There Is No Try”

Generative AI is our lightsaber — powerful, but prone to self‑amputation if waved around blindly. We’re clear about terms: here “AI” means the family of generative, LLM‑powered helpers at everyone’s fingertips.

We wield it where it earns its keep: ChatGPT‑assisted user stories, cursor‑powered code reviews, v0.dev prototypes spun up before lunch, Replit spikes over the weekend. Productivity — yes. Hype — no.

We run every proposed AI feature through three gates:

1. User value. Will it tangibly improve a banker’s day?
2. Commercial value. Will buyers pay, or renew, because of it?
3. Technical viability. Can our stack (and budget) do it responsibly? Stripe and Adyen nailed payments without sprinkling LLM dust on every API call — we’re fine following suit.

If the triforce doesn’t light up, we holster the saber. No bank wants a hallucinating chatbot refunding mortgages.

Curtain Call

Product management is messy, exhilarating, and never finished. User‑centricity, hypothesis‑driven discovery, metrics that matter, engineering symbiosis, and a skeptical eye toward hype — these are the pillars we’ve staked our reputation on at Zafin.

Over the coming months we’ll bring in industry friends to debate the finer points: B2B vs. B2C metrics, when a product should sunset, and how much technical depth tomorrow’s PMs really need. Until then, remember the first rule of product management: if the line in the grocery store is still queued up, your job isn’t done.

Product Management at Zafin was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

In industries ranging from fast-paced tech startups to multi-national financial institutions, one thing remains consistent — transferring data efficiently is essential. In service of this need, Apache NiFi has emerged as a powerful solution to tackle complex data flow challenges. Let’s explore how NiFi automates data flow and why it’s an important tool in the Zafin Integrate & Orchestrate (IO) platform.

What is Apache NiFi?

Apache NiFi is an open source, robust data flow management system that enables the automation of data movement between various sources and destinations. Developed by the Apache Software Foundation, NiFi provides an intuitive graphical interface for designing, controlling, and monitoring data flows in real-time. Its visual, flow-based programming paradigm simplifies the creation of data pipelines without the need for complex coding, making it accessible to users with diverse technical backgrounds.

Key Features of Apache NiFi

Drag-and-Drop Interface: NiFi’s user-friendly interface allows users to design data flows by simply dragging and dropping processors onto the canvas. This graphical approach enhances productivity and reduces the learning curve for building data pipelines.

Processor Ecosystem: NiFi comes with a rich ecosystem of built-in processors for handling various data integration, transformation, enrichment, and routing tasks. Additionally, users can develop custom processors to extend NiFi’s functionality and integrate with third-party systems or APIs.

Data Provenance: NiFi offers detailed data provenance capabilities, which track the lineage of each data element as it moves through the system. This feature is invaluable for auditing, troubleshooting, and ensuring data quality and compliance.

Scalability and Extensibility: NiFi is designed to scale horizontally to handle large volumes of data efficiently. Load balancing distributes the data in a flow across the NiFi nodes in the cluster based on the configured load balancing strategy.

Also, its modular architecture enables easy integration with external systems through custom processors and extensions, providing flexibility to adapt to diverse use cases.

Data Security: Security is a top priority in data management, and NiFi offers robust features to ensure data protection. It supports SSL encryption, role-based access control (RBAC), and fine-grained authorization policies to safeguard sensitive information.

Real-Time Monitoring and Alerts: NiFi provides comprehensive provenance monitoring capabilities, allowing users to track the performance and health of data flows in real-time. Administrators can set up alerts to proactively identify and address issues, minimizing downtime and ensuring continuous data availability.

Why did we choose NiFi?

NiFi is a perfect fit for Zafin’s data integration needs with Banks’ core systems and Zafin’s SaaS platform. Zafin Integrate & Orchestrate (IO) handles this data integration by providing batch, change data capture (CDC), and streaming capabilities. We treat batch files as bounded streams, and NiFi enables Zafin to stream the incoming file data to applications like Apache Flink for further processing and transformations. This has helped Zafin to integrate with a wide variety of banks easily while keeping the integration interfaces and underlying structures of the Zafin SaaS platform intact.

NiFi’s role in Zafin IO

We use Nifi to effectively incorporate pre-processing and post-processing steps in our Zafin IO data pipelines to ensure the quality and consistency of our data flow automation outcomes.

Key pre-processing uses of Nifi include:

1. Data Ingestion: Bringing data into NiFi from various sources at the bank such as databases, files, messaging systems, or APIs.
2. Data Validation: Checking the integrity, format, and quality of incoming data to ensure that it meets specified standards or requirements.
3. Data Enrichment: Adding additional context or information to incoming data by joining it with reference data, performing lookups, or applying business rules.
4. Data Transformation: Converting data from one format to another, standardizing data structures, or normalizing values to facilitate downstream processing.
5. Routing and Prioritization: NiFi enables flexible data routing based on content, attributes, or dynamic conditions. This feature is particularly useful for Zafin IO layer in scenarios where data needs to be routed to different destinations based on the characteristics of the data.
6. Data Filtering: Filtering out irrelevant or unwanted data based on specified criteria or conditions to reduce noise and focus on relevant information.

We have created standard reusable components in Zafin IO that can apply filters, perform joins, enrich data with metadata, and execute complex processing logic on the fly. Zafin IO utilizes NiFi to standardize and cleanse data as it moves through the data pipeline, ensuring consistency and accuracy across different systems. For example, Zafin IO uses NiFi to encrypt/decrypt files coming from Bank’s legacy system, convert files from EBCDIC to ASCII, and sanitize the data.

Key post-processing uses of Nifi include:

1. Data Transformation: Applying additional transformations or calculations to the processed data to conform to bank’s needs e.g. Batch output needs are different from what’s provided by Zafin SaaS out of the box.
2. Data Publication: Publishing processed data to downstream systems, applications, or users for consumption or further analysis.
3. Data Routing: Routing processed data to different destinations based on business rules or integration requirements.
4. Data Archival: Archiving processed data for compliance or backup purposes.

A typical flow which involves NiFi inside Zafin IO:

Key Benefits of Nifi at Zafin

1. Reliability and Fault Tolerance: NiFi is designed to be highly reliable and fault-tolerant, with built-in mechanisms for data provenance, flow control, and error handling. By incorporating NiFi into the architecture, Zafin IO has been able to ensure that data is processed reliably and consistently, even in the event of system failures or network disruptions.
2. Integration with Ecosystem: NiFi integrates seamlessly with other components of the ecosystem like Redpanda, Flink, etc. This interoperability enables Zafin IO layer to leverage additional capabilities for data storage, processing, and analytics as needed. By leveraging NiFi’s integration capabilities, Zafin build a scalable and extensible data processing pipeline that meets the evolving needs of its customers.
3. Improved Time to value: Using custom and built-in processors, we have been able to make reusable components that will allow faster data flow automation and integration with the banks. A great example of a reusable custom Nifi processor is Zafin IO’s EBCDIC to ASCII convertor.
4. Improved Speed: We have used NiFi’s built-in Apache Avro support for both streaming and writing files which makes processing faster and saves a lot of storage space since Avro stores the data in binary format. For one of our clients, Zafin IO processes more than 1TB of data every day achieving a throughput of 20,000 records/second while ingesting the data, converting from EBCDIC to ASCII, filtering data that’s not required, and transforming the schema to conform to that required by the Zafin SaaS platform.

Conclusion

Apache NiFi’s combination of visual data flow design, dynamic routing, built-in monitoring, extensive processor ecosystem, scalability, security, and community support make it a highly valuable tool for organizations seeking to streamline and automate their data integration and processing workflows. These capabilities make NiFi a great fit for Zafin IO.

There is no one size fits all. Depending on the characteristics of the data and the task that needs to be performed, Zafin IO provides one of several options including Apache Nifi, Apache Flink, and Redpanda Connect.

We hope you found this blog to be helpful. We are always learning, so if you have any suggestions or experiences with similar approaches, please share them in the comments section. We love to hear and learn from others.

Stay tuned for next part of our blog series on Zafin IO & Nifi:

• Installing NiFi and the Zafin IO Local Development Environment
• Role of the NiFi Registry in Release Management

Acknowledgments

We would like to thank Jijoe Vurghese for his review of this blog. His feedback helped improve the quality of this blog.

References

1. Zafin Website (2024) Zafin. Available at: https://zafin.com/ (Accessed: 31 July 2024).
2. Zafin integrate & orchestrate (2024) Zafin. Available at: https://zafin.com/zafin-io/ (Accessed: 31 July 2024).
3. Engineering at Zafin (2023) Announcing Zafin Integrate & Orchestrate (IO) and Data Fabric, Medium. Available at: https://medium.com/engineering-zafin/announcing-zafin-integrate-orchestrate-io-and-data-fabric-fc5b638974df (Accessed: 31 July 2024).
4. Apache NiFi (no date a) Apache Nifi, Apache NiFi. Available at: https://nifi.apache.org/ (Accessed: 31 July 2024).
5. Apache Flink® — stateful computations over data streams (no date) Apache Flink. Available at: https://flink.apache.org/ (Accessed: 31 July 2024).
6. Digital Platform: Transforming Banking experiences with Dynamic Solutions (2024) Zafin. Available at: https://zafin.com/platform/ (Accessed: 31 July 2024).
7. EBCDIC (2024) Wikipedia. Available at: https://en.wikipedia.org/wiki/EBCDIC (Accessed: 31 July 2024).
8. Redpanda (no date b) The streaming data platform for developers, Redpanda. Available at: https://redpanda.com/ (Accessed: 31 July 2024).
9. Redpanda (no date a) Redpanda connect: 220+ pre-built connectors, Redpanda. Available at: https://redpanda.com/connect (Accessed: 31 July 2024).
10. Gallego, A. (2024) Redpanda Connect, Redpanda. Available at: https://redpanda.com/blog/redpanda-connect (Accessed: 31 July 2024).
11. Apache NiFi Registry (no date b) Registry, Apache NiFi. Available at: https://nifi.apache.org/projects/registry/ (Accessed: 31 July 2024).

Zafin Integrate & Orchestrate (IO) and Apache NiFi: Unlocking the Potential of Data Flow Automation was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

This article outlines Zafin’s adoption of the BIAN framework and discusses how we utilized it to build a viable iteration of Zafin Agreements. It articulates conformance and explains divergence from the BIAN v11.0 standard. This article is the first of a series that puts our new BIAN inspired software and its APIs on display.

Zafin Agreements is a new software that bridges the gap between Enterprise Customer Data and an Enterprise Product Catalog. It provides a scalable and central system-of-reference for the full lifecycle of bank customer agreements.

What is BIAN?

“The Banking Industry Architecture Network (BIAN) is a global, not-for profit association of banks, solution providers, consultancy companies, integrators and academic partners with the shared aim of defining a semantic standard for the banking industry.

BIAN was formed in 2008 by a group of banks and solution providers. Other standards bodies, like ISO and IFX, later joined.

BIAN’s expectation is that a standard definition of business functions and service interactions that describe the general construct of any bank will be of significant benefit to the industry”.

Zafin agrees with BIAN’s standards.

Upon landing on the BIAN reference model page, the abundance of models can be overwhelming, but we utilized just the two models in red. Also the Service Landscape Matrix view, while daunting itself, is a handy tool to locate business functions and service domains with. Use Ctrl-f instead of the search bar on the top-right of the page.

If you browse from the Information Architecture Overview to the Agreement BOM Diagram, you’ll find the picture that has inspired Zafin Agreements the most.

In yellow are the primary Business Objects that make up Zafin’s business domain. The business objects in bright blue associate the primary objects. Ignoring the green sections for now, several important concepts stand out:

1. An Account is not directly related to a Party, Agreement links the two.
2. An Agreement associates a Product to a Party via Agreement Involvement.
3. An Arrangement lives only within the context of an Agreement.
4. Agreements can be linked via Agreement_Agreement Relationships.

These 4 concepts have been adopted as-is, in the Zafin Agreements design.

The green boxes depict a fifth important BIAN concept that we have embraced:

5. All BIAN Business Objects are of a Type.

BIAN uses Type Values (TVs) to classify instances of Business Objects with. We found this to be an elegant way to allow for bank-specific idiom, whilst retaining a strongly typed model.

The sixth adopted BIAN concept is the breakdown of business domains into service domains. We found the BIAN service domains to be too atomic to serve as the foundation for our software modules but:

6. All BIAN semantic APIs, for the service domains we have in scope, are easily recognizable, by any BIAN-aware bank.

The BIAN reference models are not without a few shortcomings. At times business concepts seem to be missed (i.e. product hierarchies). In other cases, Type Values are insufficient for the North American market, begging for more standardization by BIAN. This is why Zafin Agreements is still inspired by BIAN and not fully BIAN adept. Before we get into how the BIAN concepts have been applied, there is an important question to answer first; why adopt the BIAN standard?

Why adopt BIAN?

Where many banks are dissecting their monolithic estate to implement best-of-breed solutions to connect their digitally transformed channels with thin account management cores, these banks are increasingly adopting BIAN. Many of the larger North American banks have been BIAN members for years, no less than 22 new banks became a BIAN member last year. Banks believe BIAN to “enable more efficient and effective development and integration of software solutions for and between banks”. And rightfully so with Open Banking regulations on the rise and the integration of ISO, out of the box.

Why would you as a banking engineer want to adopt the BIAN models? How much time has been sunk into endless conversations with clients on element definitions or allowed enumerations? How much more efficient would it be to not impede your design work for a couple of sprints, with a statement like this?

“While we await requirements, refer to this API for semantic guidance. These are objects that Zafin has harvested for this service domain already, for you to amend or add to”. Less conversations needed on interface specifications between engineers, “lowers overall integration efforts significantly” resulting in faster time to market.

The adoption of your software is all about the quality of your APIs. The success of your software is determined by your ease of integration and the delivered business value it will add. If there is a common vocabulary for all things banking and everybody in the industry speaks it, why invent another?

The Open Banking (OB) APIs are here to stay and entering governments are likely to adopt standards of maturing government bodies. There is a “current need for more industry integration through the usage of open APIs”. BIAN has the best coverage across banking business domains, with the inclusion of ISO (and ACTUS recently) to become the Open Banking glossary.

As with any standard, its legitimacy comes with its degree of adoption. While standards like ISO and FPML remain relevant, IFX and IFW fade away. For BIAN to “support the adoption of more flexible business service sourcing models and enhance the evolution and adoption of third-party software”, adopting BIAN should become a self-fulfilling prophecy with both banks and banking software vendors, to converge to a concept-of-one.

What is Zafin Agreements?

With the WHAT and WHY of BIAN explained, we touch on the WHAT of Zafin Agreements before we delve into its design. This is because for Zafin to adopt BIAN Service Domains (SDs), and its semantic APIs, we need to understand the scope of the service domains that Zafin manages and Zafin interacts with.

Overlaying BIAN service domains on the context of a bank interacting with Zafin SaaS, we would depict a Context diagram like this:

To understand our current state in terms of BIAN service domains, the following table shows some of the SDs we support through our software solutions.

The table suggests that we have built duplicate features into Zafin Agreements, which is partly true, we will explain that in a bit.

Zafin Agreements Business Object Model

This is Zafin’s interpretation of the BIAN Agreements BOM. Our Minimum Viable Product (MVP) implements this Entity Relationship Diagram (ERD) to the letter, it became our service domain model of sorts as well. The diagram articulates well we think, Zafin’s conformance to the first four BIAN inspired concepts.

Zafin Agreements APIs

To support concept 6 (our APIs are recognizable by BIAN-adept banks) we must first scope our BIAN Service Domains. The following BIAN SDs are in scope:

1. Sales Product
2. Customer Products and Services
3. Customer Product And Service Eligibility
4. Party Reference Data Directory
5. Product Directory

Because of the size of our team, we decided to merge domains 1, 2 & 3 into Sales Product Agreement (SPA). Both directories (PRDD & PD) are separate modules, loosely coupled to SPA. With scope in terms of BIAN semantic APIs now defined, let’s scope the endpoints within these APIs.

These are some of the endpoints we support in MVP1 of Zafin Agreements, they link to the BIAN semantic APIs on which they are based:

• POST /salesProductAgreement/v1/productAndServiceAgreement/evaluate
• PUT /relationshipManagement/v1/customerProductsAndServices/{PRDD_id}/request
• POST/partyReferenceDataDirectory/v1/{PRDD_id}/register
• POST/relationshipManagement/v1/customerProductAndServiceEligibility/{PRDD_id}/evaluate
• GET/productDirectory/v1/{PD_id}/salesAndMarketing/retrieve

In the BIAN Sales Product SD for example we found semantic inconsistencies between the SD control record and the Agreements BOM. In early stages of our MVP, while constantly adding elements and entities, we found the translation from BOM to API control records, to be too cumbersome and less automate-able. We decided to stick to our Agreements BOM also for API payloads.

So why did we duplicate features of the Product Directory SD and the Product and Service Eligibility SD, within the new Zafin Agreements software?

For the Product Eligibility SD, we needed service-orchestration capabilities to fetch existing agreements. Like so adding more business value to the existing eligibility checks as our current software cannot orchestrate APIs across other Zafin software assets. We therefore implemented a BIAN “Legacy Wrapper” pattern and we leverage Temporal for orchestration.

We also made a BIAN-based read copy from our existing Enterprise Product Catalog (EPC) and named it Zafin Product Directory. The existing EPC software has a DSL-driven development approach. This approach works very well for quickly delivering custom product catalogs to banks, but the generated DB schema and its generic event publications make it more challenging to build new software on top it.

The idea here is to create a one common Zafin Product Directory data model to serve as a schema foundation for new solutions and as a highly resilient high performing read cache. An example would be the Product Explorer, coming soon, that combines the bank product configurations with analytical (revenue) insights from Zafin Data Fabric.

Conclusion

We hope you found this article to be helpful. We are always learning, so if you have any suggestions or experiences adopting BIAN, please share them in the comments section. We love to hear and learn from others.

The next BIAN related article will be on our take to BIAN business scenarios and we’ll discuss to go less relational with our physical database and store most of the BOM in a JSONB to not have to change our schemas when a bank asks us to store more data points.

References

1. BIAN Organization Home (no date) BIAN. Available at: https://bian.org/ (Accessed: 01 May 2024).
2. The ISO 20022 repository (no date) ISO20022. Available at: https://www.iso20022.org/financial-repository (Accessed: 01 May 2024).
3. BIAN Architecture Overview Portal v.12.0 (no date) InSite. Available at: https://bian.org/servicelandscape-12-0-0/index.html (Accessed: 01 May 2024).
4. BIAN Service Landscape V12.0 matrix view (no date) InSite. Available at: https://bian.org/servicelandscape-12-0-0/views/view_51891.html (Accessed: 01 May 2024).
5. BIAN Agreement bom diagram (no date) InSite. Available at: https://bian.org/servicelandscape-11-0-0/views/view_52432.html (Accessed: 01 May 2024).
6. BIAN Business Object (no date) Insite. Available at: https://bian.org/servicelandscape-12-0-0/object_28.html?object=64173 (Accessed: 01 May 2024).
7. BIAN Business Domain (no date) Insite. Available at: https://bian.org/servicelandscape-11-0-0/object_10.html?object=65111 (Accessed: 01 May 2024).
8. BAIN Agreement (no date) Insite. Available at: https://bian.org/servicelandscape-12-0-0/object_26.html?object=36783 (Accessed: 01 May 2024).
9. BIAN Arrangement (no date) Insite. Available at: https://bian.org/servicelandscape-12-0-0/object_25.html?object=33524 (Accessed: 01 May 2024).
10. Canada, D. of F. (2023) Government of Canada — Final Report — Advisory Committee on Open Banking, Canada.ca. Available at: https://www.canada.ca/en/department-finance/programs/consultations/2021/final-report-advisory-committee-open-banking.html (Accessed: 01 May 2024).
11. BIAN Sales Product API Specification (no date) SwaggerHub. Available at: https://app.swaggerhub.com/apis/BIAN-3/SalesProduct/11.0.0 (Accessed: 01 May 2024).
12. Financial products markup language (no date) FpML. Available at: https://www.fpml.org/ (Accessed: 01 May 2024).
13. Implementing Bian Service domains using the IFX Business Message Specification (no date) BIAN. Available at: https://bian.org/deliverables/white-paper/implementing-bian-service-domains-using-the-ifx-business-message-specification/ (Accessed: 01 May 2024).
14. The Information Framework (IFW). Available at: https://public.dhe.ibm.com/software/data/mdm/pdf/IFW_Poster_2006.pdf (Accessed: 01 May 2024).
15. 4.3.1 Legacy Wrapping Approaches — Bian Semantic Api Practitioner guide V8.1. Available at: https://bian.org/wp-content/uploads/2020/10/BIAN-Semantic-API-Pactitioner-Guide-V8.1-FINAL.pdf (Accessed: 01 May 2024).
16. Temporal.io — Open source durable execution (no date) Open Source Durable Execution | Temporal Technologies. Available at: https://temporal.io/ (Accessed: 01 May 2024).
17. DSL Guide (no date) martinfowler.com. Available at: https://martinfowler.com/dsl.html (Accessed: 01 May 2024).
18. Zafin Studio: Your gateway to innovation in banking (2024) Zafin. Available at: https://zafin.com/studio/ (Accessed: 01 May 2024).
19. Zafin Data Fabric (2024) Zafin. Available at: https://zafin.com/zafin-data-fabric/ (Accessed: 01 May 2024).
20. Oyama, F. (2024) PostgreSQL and JSON — how to use JSON data in PostgreSQL, freeCodeCamp.org. Available at: https://www.freecodecamp.org/news/postgresql-and-json-use-json-data-in-postgresql/ (Accessed: 01 May 2024).

BIAN Adoption at Zafin (Part 1) was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Modern analytics workflow requires embedded analytics where reports, dashboards, models, and data visualizations can be accessed through a single interface. The goal of embedded analytics is to bring analytics as close to business operations as possible, reducing the amount of unnecessary context-switching between application interfaces. Conventional business intelligence (BI) solutions introduce friction in analytics operations since analytics and operations are configured as separate processes. For example, a workflow that consists of designing a product offering and gathering information about client behaviors requires two different workspaces, since they are imagined as categorically different processes: one operational and the other analytical. The former is performed through an internal tool and the latter through BI software. The purpose of embedded analytics is to bridge this gap between the operational and analytical processes by bringing analytics into the familiar context of operational workflow. With embedded analytics, the same workflow of designing a product offering can be redefined as a multi-step form where for each condition the user sets, a dynamic chart will display the clientele breakdown accordingly. To enable embedded analytics, a performant and easy-to-use data visualization solution is a key requirement.

In application development, building data visualizations is brittle and time-consuming. First, data has to be prepared based on the data series that needs to be rendered on the front-end, and then an API needs to be implemented to assemble and return the data series as JSON. Finally, the front-end needs to unmarshall the JSON and bind it to the charting libraries. This is already labor-intensive, but it gets worse: If we want to change the charts on the front end, perhaps to go from a daily aggregate to a weekly aggregate, then the data model, API, and front end all need to be changed.

The cumbersome process of building data visualizations is only one of many challenges. A bigger issue is the inconsistencies that arise when multiple downstream applications define their versions of the same metrics. In stand-alone BI solutions, commonly used metrics are defined internally. As a result, other downstream applications must define their versions of the metrics. Managing metric definitions becomes a task in itself. To address the difficulties that arise in building data visualizations, we introduce a semantic layer in the data pipeline, powered by Cube.

Semantic Layer

A semantic layer in the data stack serves as a central repository of metric definitions. The semantic layer is where raw data are made intelligible for analytics use cases. Consider the typical demands of business reporting: a sales VP would likely look at Key Performance Indicators (KPIs) like close rates, deal velocities, and transactions by segment, region, and team. These KPIs are metrics with fixed definitions to measure business operations. Other examples of metrics include aggregations that define categories and segmentations of data, such as active users and business regions. Seemingly crisp segmentations in fact leave many ambiguities if analysts define them on the fly each time. Do the weeks start on Sunday or Monday? Are British customers part of the EU jurisdiction? While some of these definitions are part of the data table, many are further aggregations on top of the existing information. The semantic layer provides a central location where metrics are defined once and for all for downstream applications. The layer also enables implementation of data controls by supporting fine-grained access to metrics.

A semantic layer provides data modeling for the underlying database, simplifying the data provisioning for making visualizations. As mentioned above, provisioning data for visualizations is an arduous process. Most modern applications use an Object-Relational Mapper (ORM) that maps underlying data representations in databases to those in object-oriented programming to allow programmatic interactions with databases instead of SQL. While ORMs abstract away SQL queries, each downstream application must implement its ORMs. The semantic layer provides a universal data modeling layer that is independent of downstream data applications. Downstream applications no longer need to build their ORMs. Instead, they query the database through the data representations, including metric definitions, defined in the semantic layer. Internally, the semantic layer transforms queries and metric requests into SQL queries. This architectural shift is depicted below, where data modeling and metric definitions move from individual applications to the central location of the semantic layer. Because the semantic layer provides the metric definitions for analytics use cases, it is also referred to as Headless BI, where the headless means the lack of a graphic interface or the capacity for rendering visuals.

Why We Chose Cube as Our Semantic Layer

At Zafin Analytics, we use a semantic layer, Cube, to manage our metric definitions and accelerate product development in embedded analytics. The semantic layer is part of the Zafin Data Fabric that transforms and provisions data for analytics use cases. We initially started to investigate semantic layer libraries because we needed to build embedded analytics in the analytics product suite. The existing development process for making visualizations is time-consuming since each chart requires a REST endpoint and a dedicated microservice. We are looking for an embedded analytics solution to accelerate the development process. And since our product suite has several products that share the same underlying data source, we need a semantic layer to define metrics and data models once and for all. Another important requirement in our selection process is performance. We would like to cache frequent queries to accelerate query response and allow scaling as the user activity grows.

Since the semantic layer is relatively new in the data engineering toolkit, the options for semantic layer products are constrained. BI tools such as Power BI still dominate the market, and major vendors support embedding dashboards in custom applications. For Zafin’s use cases, using BI tools for embedded analytics feels like overkill. Also, as mentioned above, corporate BI products are black boxes with metric definitions tied to the product. Therefore, we mainly surveyed semantic layers that can serve as a headless BI component and integrate with our existing tech stack such as dbt’s semantic layer, Atscale and Stardog.

We landed on Cube or CubeJS as our semantic layer of choice. Built with performance in mind, Cube is a popular open-source semantic layer built in Rust. We chose Cube as our semantic layer because it offers the following advantages:

• Support for complex queries and visualizations: Cube accelerates the data provisioning process for visualizations. Instead of making tailored APIs for each chart, data provisioning in Cube is achieved through Cube queries written in JavaScript. Managing charts no longer requires backend changes. Instead, one only needs to add the charting component along with the Cube queries to fetch the underlying data source. In addition, Cube also provides out-of-box support for building visualizations through its development mode and React library.
• Caching: Caching is a critical part of any large-scale data-intensive applications that help optimize performance and reduce cloud data storage costs. Cube provides two forms of caching, in-memory cache and materialized queries called pre-aggregations. When Cube receives a query, it will search for an existing cached query. Using cached queries significantly accelerates the query speed. The pre-aggregations are refreshed on a customized schedule.
• Self-hosted and horizontal scaling: Cube.dev offers open-source self-hosted and managed solutions. Unlike its closed-source competitors, Cube is under the Apache License 2.0. Cube gives us the flexibility to customize and self-host our Cube instance. As a self-hosted solution, Cube also provides options for horizontal scaling, enabling expansion and resource optimization as data needs evolve. Cube also has a managed offering which we may use in the future to lower our operational overhead.

Building Metrics at Zafin

Design user-defined metrics with Cube data models

As a pilot project, we integrated Cube with our existing application, Zafin Studio, to build data visualizations. Our visualization use cases are centered around the notion of user-defined metrics. An example of user-defined metrics is the monthly sum of average deposit balances for that exceeds $2000 at the account level. After some research, we found that we can leverage metric definitions in Cube data models to build these user-defined metrics. In Cube, we use measures to define common aggregations that form the building blocks of user-defined metrics in Zafin Studio. Each table in the underlying database is mapped to a data model file. For the balances table, a model file can look like below, where we define aggregations, such as sum, minimum, and maximum on the column average_balance. In Cube, these measures can grow more complicated as the corresponding SQL definition grows more complex.

Based on the data model, the user-defined metric can be composed as a Cube query shown below. In the query, we specify the aggregation average_balance_sum in the balances table, with the month as our granularity. We also added two filters, one on average_balance and another on customer_id, to our search. Since these queries are composed on the client side, building visualizations based on the metric becomes a matter of composing queries based on Cube data models. Here, the team has made an architectural decision not to hardcode user-defined metrics as Cube measures, which sit on the server side, but convert them into Cube queries instead. This decision is based on several considerations. In terms of operation, updating data models on the server based on user activities is in general not advised since the server should be stateless according to the RESTful principle. In our case, updating data models based on unpredictable user activities is impractical. In terms of privacy, user data should not be exposed as data models for all users of the Cube server.

Translate data representations to Cube queries

To integrate Cube with Zafin Studio, we also built a few utility microservices, the most important being a microservice that translates the internal representation of user-defined metrics to Cube queries. In Zafin Studio, there is a one-to-one relationship between user-defined metrics and Cube queries. The user-defined metrics have an internal data model that represents the example metric as the JSON object below. The microservice is responsible for converting the internal representation of user-defined metrics to Cube queries.

Allow clients to query Cube through a dedicated API

In addition to web application integration, we built a client-facing API to allow users to query user-defined metrics. We designed the API based on the client’s need to streamline data access and increase transparency. Through this API, users gain direct access to the dataset associated with the metrics they created through Zafin Studio. The client-facing API also gives users more flexibility to specify the time range and interval.

Together, there are multiple points of data access for analytics consumers through Cube, as shown in the diagram below.

Build CI for production-grade Cube cluster

Moving Cube to production and integrating with Zafin’s CICD pipeline is another major endeavor in the project. To meet the performance demand, we choose to deploy Cube in Zafin’s Azure Kubernetes cluster. The Kubernetes cluster consists of two main components, the Cube API and Cubestore. As depicted in the diagram below, the Cube API handles incoming requests and decides whether the query exists in the cache. If so, it queries the Cubestore for the cached query. Otherwise, it fetches the query from the database and saves the query to Cubestore.

In addition to the Kubernetes cluster, we are also building a few components to streamline the DevOps process:

• A script to generate Cube data models based on database changes
• Enable rolling updates to eliminate server downtime

Conclusion

As an adaptive semantic layer, Cube made it easy made it easy for teams to use their existing data warehouse as a single source of truth, allowing them to serve consistent metrics to both enterprise customers and internal stakeholders. At Zafin, we chose Cube as our semantic layer to build embedded analytics, letting us shrink the time to embed visualizations from four person days to just one. As we progress in Cube implementation, we plan to expand into incremental pre-aggregations to leverage Cube’s powerful caching features and introduce Cube to other products at Zafin.

References

1. Introducing JSON. Available at: https://www.json.org/json-en.html (Accessed: February 16, 2024)
2. The Semantic Layer for every data app. Available at: https://cube.dev/ (Accessed: February 16, 2024)
3. Semantic Layer. Available at: https://www.getdbt.com/product/semantic-layer (Accessed: February 16, 2024)
4. Semantic Layer Solution — BI & Data & Analytics Software. Available at: https://www.atscale.com/ (Accessed: February 16, 2024)
5. The Enterprise Knowledge Graph Platform. Available at: https://www.stardog.com/ (Accessed: February 16, 2024)
6. Introduction to RESTful Web services — IBM Developer. Available at: https://developer.ibm.com/articles/ws-restful/ (Accessed: February 16, 2024)
7. Stancil (2021) The missing piece of the modern data stack. Available at: https://benn.substack.com/p/metrics-layer (Accessed: Feb. 22, 2024)

Building Embedded Analytics with Cube was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

The Apple iPhone, introduced in 2007, caused a seismic shift in our expectations of software. Have you noticed that we’ve come to expect frequent updates to our apps? Furthermore, this expectation leads us to assume (rightly or not) that an infrequently updated app is abandonware. This expectation extends to SaaS products and platforms.

Deployment frequency is one of the four measures of organization performance identified by Google’s longest-running research by the (DORA) team. However, teams must balance deployment speed with system availability.

Continuous Delivery is a crucial tenet of lean, agile value delivery while maximizing system uptime. As Martin Fowler put it succinctly, “Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time.”

Banking regulations require explicit customer acceptance and approvals for changes to products. Zafin continuously delivers our SaaS product updates to customer validation environments pending customer acceptance. Subsequently, “continuous” delivery techniques complete the last-mile delivery into production environments.

We have several tools in the tool chest to achieve Continuous Delivery, including blue-green deployments, canary releases and feature toggles. In the fast-paced world of software development, deploying updates and new features to production environments requires careful planning to avoid downtime and potential service disruptions.

Blue-Green Deployment has emerged as a powerful strategy that enables organizations to release software changes while maintaining continuous service. As we continue our GitOps journey, we will dive into the benefits of Blue-Green deployment and how it is effectively implemented in Zafin to ensure smoother and more reliable software releases.

What is Blue-Green Deployment?

Blue-Green Deployment is a software deployment strategy used in the field of DevOps and continuous delivery to minimize downtime and risk during the release process of a software application or service. The primary goal of this approach is to ensure that the latest version of the application is deployed and tested thoroughly before it is made available.

In a Blue-Green Deployment, two identical production environments are maintained, each with its own set of resources, and infrastructure.

Case Study on what and how we did it in Zafin

Zafin’s cloud deployment architecture on Azure was lacking Blue-Green deployment capability. This resulted in

• Service non-availability/downtimes during platform updates and application deployments.
• Uncertainty around downtime required for maintenance.
• Pressure on deployment managers to execute the maintenance activity in a stringent, contractually agreed maintenance window.

The significance of 24/7 service availability increased when Zafin’s clients moved away from batch mode processing to more of real time data processing via APIs and Kafka events.

To address the challenges and changing needs, we decided to migrate to a modern, cloud-native infrastructure that supports Blue-Green Deployment. The goals are, achieve zero downtime during deployments and updates, improve application performance, and enhance overall experience.

Figure 1 illustrates our deployment process.

1. The existing production application runs in the Blue environment, serving current traffic.
2. In the event of a new application version release/upgrade, an identical Green environment is provisioned in the existing Kubernetes cluster.
3. Traffic routing is managed by Azure Traffic Manager (Global DNS Load Balancer), allowing seamless switching between Blue and Green environments without impacting availability.
4. The basic sanity test of the application will be done in the Green environment.
5. Once the Green environment is deemed stable, the traffic switch in the Azure Kubernetes platform will be initiated, directing incoming traffic to the Green environment.
6. The Green environment becomes the new production (Blue) environment.

Pre-Requisites

1. Ensure that the following roles have been configured and are active before the deployment starts:

a. Azure Kubernetes Service RBAC Admin

b. Azure Kubernetes Service Contributor Role

2. Enable project access to the Azure DevOps pipeline.

Pre-Deployment Steps:

1. Perform a sanity check on the Application UI and API availability.
2. Take appropriate data/config map backups.
3. Review the below steps with the Cloud Engineering team

a. Infra Sizing

b. Confirm that the environment is running the latest Kubernetes version.

c. The migration path is correctly configured.

Deployment Steps

1. Current state

2. Terraform changes to the below configuration:

• This will ensure the Green pods are up and running while the traffic is still being served by the Blue.
• Compare the config maps and docker images with the Blue to ensure that all the properties are in sync. Any missing properties are to be added to the green pods.

3. Run a quick check to ensure the Application UI/APIs are available for the Green pods by using the Traffic manager.

4. Run the implementation specific DB scripts, deploy implementation changes to the Green pods and ensure the execution is successful.

5. Perform all the necessary sanity checks on the Green deployment (Application UI access, API connectivity, etc.).

6. Run the ‘Phase 3’ pipeline, that will switch the traffic from Blue to Green pods, while the Blue will still be up and running. A UI/API check is mandatory here to ensure that post the switch, all services are up and running.

7. Finally, run the ‘phase 4’ pipeline that will scale down all the Blue pods. Post this step, there will only be Green pods. The following is the pipeline configuration for Phase 4:

Perform a UI/API sanity check and check for any downtime during the switch over process.

The Benefits

• Fully Automated Deployment: With the cloud-native infrastructure in place, we could set up a robust Continuous Integration and Continuous Deployment (CI/CD) pipeline using Azure DevOps. The CI/CD pipeline automated the entire deployment process, which enabled the deployment team to push updates to these environments effortlessly. Kubernetes orchestrated the deployment process, ensuring consistency and scalability across the application’s life cycle.
• Improved Application performance: Auto scaling was implemented in the Kubernetes cluster to address the dynamic user requests/load. With auto scaling feature, the infrastructure could dynamically adjust its resources based on demand. This eliminated the need for manual intervention during traffic fluctuations, improving application performance.
• 24x7 Service availability: Azure’s managed services efficiently handled infrastructure patch upgrades without any downtime for the customer’s application. By leveraging managed services, such as Azure Kubernetes Service (AKS), Zafin could focus on building and deploying their application while Azure took care of underlying infrastructure updates and maintenance.

Lessons Learned

1. Communication and Collaboration: Successful migration to Blue-Green technology necessitates close collaboration between Product, Engineering, Operations, and other stakeholders. Effective communication is crucial to keep everyone informed about the deployment process, potential challenges, and progress.
2. Embracing Cloud-Native Solutions: Migrating to Blue-Green technology often involves leveraging cloud-native solutions and services, which offers greater flexibility, scalability, and automation capabilities. Embracing cloud-native practices can lead to substantial improvements in application management and deployment efficiency.
3. Training and Skill Development: The transition to Blue-Green technology requires upskilling or retraining the operations and deployment teams. Ensuring that the teams are proficient in managing the new environment and its tools will contribute to a smoother deployment process.
4. Continuous Improvement: Adopting Blue-Green technology should not be considered as a one-time event. The learning here is that one should continually assess the deployment processes, identify areas for improvement, and iterate on the practices to further enhance efficiency and reliability.
5. Compliance and Security: As with any technology migration, it is critical to ensure the compliance and secrutiy measures are maintained during the transition. One should validate that the Blue-Green environment meets all necessary compliance standards and security requirements.

By learning from these lessons and continuously refining the practices, we can successfully make the shift to Blue-Green technology and unlock the benefits of reduced downtime, improved reliability, and enhanced user experiences.

Conclusion

By migrating to a cloud-native infrastructure with Blue-Green Deployment support and leveraging Kubernetes services on Azure, we were able to successfully transform the application release process. The implementation enabled us to achieve higher availability, scalability, and agility, while also improving the overall reliability and performance of the implementation.

References

1. Fowler, M. (2010) Bliki: Blue green deployment, martinfowler.com. Available at: https://martinfowler.com/bliki/BlueGreenDeployment.html (Accessed: 16 January 2024).
2. Fowler, M. (2013) Bliki: Continuous delivery, martinfowler.com. Available at: https://martinfowler.com/bliki/ContinuousDelivery.html (Accessed: 16 January 2024).
3. Hodgson, P. (2017) Feature toggles (aka feature flags), martinfowler.com. Available at: https://martinfowler.com/articles/feature-toggles.html (Accessed: 16 January 2024).
4. Portman, D.G. (2020) Use four keys metrics like change failure rate to measure your devops performance | google cloud blog, Google. Available at: https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance (Accessed: 16 January 2024).
5. Sato, D. (2014) Bliki: Canary release, martinfowler.com. Available at: https://martinfowler.com/bliki/CanaryRelease.html (Accessed: 16 January 2024).

Quantifying our benefits of using a Blue-Green Delivery strategy was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, we will discuss Zafin’s approach to AI, using Zafin Studio’s Product & Pricing Index (PPI) as a focal point. At its core, PPI is a global banking data aggregation tool, with the aim to provide our clients with easy access to the global banking landscape and assist them in formulating new product propositions.

To achieve this ambitious goal, we’re leveraging AI in both the preparation of the index (a back-end data transformation pipeline); and as a new medium for interaction with the index (a channel-agnostic Natural User Interface or NUI experience).

The Effects of Democratized AI (or A Primer on AI Data Transformation Pipelines)

A by-product of the overall industry’s mission to democratize AI is the increased accessibility to capabilities that historically took entire organizations to achieve. We’re at a point where these capabilities can be used as foundational elements — that is, they enable us to supercharge our tools without getting distracted from our core focus of banking modernization.

Index preparation and maintenance is a particularly nuanced challenge that requires an understanding of the domain space at a micro and macro level. There are many variations to how banking products are structured; how features are packaged; how geographies are categorized; and yet, there is a connective consistency throughout all of them. The challenge is in recognizing and giving weight to these subtle variations in structure and values; and simultaneously, standardizing and normalizing down to a consistent index. Such an index will allow for comparisons across products, generalizations on trends, and the distillation of meaningful insights.

Large Language Models (LLMs) are well suited for this challenge. The AI powered data transformation pipeline (shown in Figure 1) starts with the scraping of 200 financial institutions which are stored into a blob store. This raw text is cleansed, resized, and chunked as needed to fit into GPT-4’s context window and to improve the performance of the pipeline by feeding GPT-4 only relevant information. The blobs of text are processed by a chain of LLM prompts. The outputs are essentially database insert suggestions consisting of all the features, rates and fees for a particular product.

The validation process of each suggestion is a bit of an art. First we filter for hallucinations and volatility by checking for consistency in the database insert suggestions across identical runs (the exact number of identical runs is something that fluctuates). Then, we look for variations in column values across suggestions using multiple sources for the same product (i.e. webpage vs. PDF). During this multi-source merge process, we automatically fill in any non-conflicting but missing fields that were not available from one source but were observable in the other. This gives us a final unvalidated product profile merged from multiple runs per available source.

Then, an automated flagging process creates tasks for human review that highlight conflicts between sources in the column value suggestions. These conflicts are placed higher in priority while other due diligence validation tasks are placed lower in priority. The human review process takes time and may seem contrary to an AI-driven approach, but it is in fact a core pillar of our approach to AI: human in the loop.

Once the validation tasks are complete, the unvalidated data is moved to a validated database. We create an intentional gap between this database and the actual production database which serves more as a host for static tables rather than any real-time index preparation activities. Thus, Zafin Copilot and any user accessing the Product & Pricing Index application can easily browse a standardized schema that compresses the global banking landscape.

Widening the Accessibility Aperture with NUI

The technology space is undergoing a tectonic shift in how users interact with platforms. Historically, with the entry of each new paradigm in user interface, we’ve seen a massive increase in the accessibility of capabilities and the diversity of users that the platform serves. Consider the shift from terminal / command-line user interfaces (TUIs) to interactive point-and-click graphical user interfaces (GUIs). This shift ignited a universal transformation across industries where the power of computing transcended from a niche skill set to a digital revolution.

Similarly, another massive increase in accessibility to capabilities is happening with the advent of mainstream natural user interface (NUI), primarily ushered in by OpenAI. It is in alignment with this coming shift in user behaviour, experience expectation, and greater accessibility to capabilities, we have embarked on the journey of Zafin Copilot.

Zafin Copilot is both a knowledge assistant and an agent of action to support our banking users. We want to enable a conversational interface that answers relevant banking product questions such as:

• What are the current trends in banking products, and what are the demands in the market?
• What products and features are offered by our competitors?
• How can we adjust pricing models to maximize profitability — without sacrificing competitiveness?

Zafin Copilot can understand and decompose high-level intent from the user and turn questions into actionable steps. It can execute those tasks by forming queries on the Product & Pricing Index and interpreting the results.

It is designed with specific instruction sets on understanding the Index’s schemas, including the meaning behind various data and relevant validation steps. It generates dynamic context based on the query, enabling it to respond in a meaningful way to a wide array of intents. We’ve ensured that it stays focused on the domain and does not stray too far away from its enterprise context. Though we’ve put in place definitive constraints on it, we’re noticing it can come up with approaches which do not have any supporting tangential or related examples given by us. This is because our design lets the copilot first figure out ‘how should I approach this’ and keeps the human in the flow for confirmation of approach. This is the most exciting part. Lastly, it has access to full-functionality REST APIs, ensuring Zafin Copilot has scalable, flexible, and lightweight access to up-to-date information across the platform. It is through this that Zafin Copilot can become a cross-product engine that permeates across our product platform.

Our Guiding Principles for Deploying AI

As Zafin deploys AI capabilities, three key principles guide our development activities:

1. Consistency and accuracy — Responses must be consistent, accurate, derived, and explainable based on an explainable process.
2. Acceleration and expansion — We want to thoughtfully expand the landscape of information, knowledge, and capabilities available to more users while meeting them where they are.
3. Human in the loop — always. We are committed to always have a human review and validate activities that ensure quality and reliability.

With these principles in mind, we can move forward with speed and determination into exciting and unfamiliar territory.

Challenges of Today

A feature of working on this frontier is constantly solving for novel technical challenges.

For example, initial token limits prevented sending all available context to the generative AI model with each call, so we had to intelligently select and supply access to the right information at the right time via our Task Manager (Figure 2). Another example is the fact that large language models struggle to break down complex problems on their own. Zafin Copilot uses Conversation Manager to deliver single, well-defined tasks to a set of purpose-specific agents. Lastly, for most of 2023, OpenAI’s APIs were stateless, meaning conversational AI applications such as Zafin Copilot had to preserve and manage conversation context on its own using runtime caches and conversation histories.

Even as we build out these innovative solutions for our application stack, we are hyper-aware that the generative AI models we use are improving at an unprecedented rate.

Opportunities for Innovation Ahead

Last month’s OpenAI Dev Day brought the announcement of game-changing updates to their API (though not yet available for consumption in full on Azure OpenAI at the time of writing).

The forthcoming release of OpenAI’s GPT-4 Turbo with a larger 128K context window will open up a wide range of opportunities for Zafin Copilot and the AI Data Transformation pipeline. Our platform’s use of chunking can be reduced and we won’t have to worry about leaving relevant information out of our prompts.

GPT-4 Turbo’s JSON mode will enable lighter prompting and decreased variability in responses. This will reduce much of the burden of formatting responses and in turn, free up more resources for responding to the user’s inquiry. The seed parameter setting will further allow us to reduce the randomness of the AI models. Greater predictability improves the conversational experience and increases confidence that the tool can complete the tasks it was designed for.

The new Assistants API will accelerate integration with in-house models and tools through new capabilities like code interpreter, knowledge retrieval, and function calling. GPT-4 Turbo’s improved speed will allow us to reduce our average answer time. Our less computationally intensive tasks can be sent to Turbo to help free up resources for the main GPT-4 model. Finally, the team is also excited to incorporate the new multimodal capabilities, such as text-to-speech to enrich user experience further.

Our architecture is well-positioned to pivot quickly and take advantage of these powerful updates. We take a modular approach so we can rapidly upgrade elements to the state-of-the-art with minimal re-work. We expect to fully adopt these new features into Zafin Copilot in Q1 2024.

As we do, we will share deep dives into several key aspects of our approach for building with AI in the age of rapid disruption. Check back with us soon — we’ll have some exciting updates to share along the way.

References

1. Zafin Studio: Your gateway to innovation in banking (2023) Zafin. Available at: https://zafin.com/studio/ (Accessed: 22 December 2023).
2. Natural user interface (2023) Wikipedia. Available at: https://en.wikipedia.org/wiki/Natural_user_interface (Accessed: 22 December 2023).
3. Large language model (2023) Wikipedia. Available at: https://en.wikipedia.org/wiki/Large_language_model (Accessed: 22 December 2023).
4. Boyd, E. (2023) Introducing GPT-4 in azure openai service, Microsoft Azure Blog. Available at: https://azure.microsoft.com/en-us/blog/introducing-gpt4-in-azure-openai-service/ (Accessed: 22 December 2023).
5. OpenAI. Available at: https://openai.com/ (Accessed: 22 December 2023).
6. OpenAI devday. Available at: https://devday.openai.com/ (Accessed: 22 December 2023).
7. OpenAI platform (no date) GPT-4 and GPT-4 Turbo. Available at: https://platform.openai.com/docs/models/gpt-4 (Accessed: 22 December 2023).
8. Assistants Overview — OpenAI API — platform.openai.com. Available at: https://platform.openai.com/docs/assistants/overview (Accessed: 22 December 2023).

Building with AI (not for AI) was originally published in Engineering at Zafin on Medium, where people are continuing the conversation by highlighting and responding to this story.