Key Features and Comparative Advantages

1. Read/Write Splitting

• Automatically routes SELECT queries to read replicas and INSERT, UPDATE, DELETE, and other write queries to the primary node.
• Uses SQL parsing to determine the type of query.
• Helps in scaling read-heavy workloads by offloading read operations to replicas.

Advantage over PgPool-II:

• More accurate query classification
• Lower latency routing decisions
• Simpler configuration

2. Connection Pooling

• Reduces the overhead of establishing new connections by pooling and reusing existing ones.
• Supports per-user and per-database connection pools.
• Configurable pool sizes and timeouts.

Alternative : Pgbouncer

• PgCat is a multi-threaded proxy written in Rust using the Tokio asynchronous runtime. Pgbouncer is a single-threaded proxy written in C based on libevent. PgCat is multi-threaded so it can benefit from more CPU cores on the machine it runs on.
• PgBouncer: Requires a restart or admin commands to reload configuration, which can disrupt active connections. PgCat: Supports hot reloads with zero downtime — config changes (like adding pools or nodes) apply live.

https://www.tembo.io/blog/postgres-connection-poolers

3. Replica Lag Awareness

• PgCat can track replication lag of read replicas and avoid sending queries to lagging replicas.
• Helps ensure data freshness for read queries when consistency is important.

Alternative : PgPool|| and Patroni HA proxy

• Pgpool-II supports replica lag monitoring, but it is not as advanced or as easily configurable as PgCat in this regard. It is a more traditional solution with broader features, but it is more limited and complex when it comes to lag awareness.
• Patroni is a high-availability solution for PostgreSQL. When used together with HAProxy, HAProxy can route queries based on lag status or health check results. However, lag awareness in this setup may still require custom configuration and scripting.

4. Automatic Failover Handling

Supports automatic failover to a new primary if the existing primary becomes unavailable. (Alternative: Patroni & PgPool||)

5. Multi-Tenant and Multi-Database Support

• Can serve multiple databases and tenants from a single PgCat instance.
• Each pool configuration can have its own primary and replicas.

6. Config Reloading Without Restart

• Configuration can be reloaded on the fly (e.g., adding/removing replicas or changing settings) without restarting PgCat.

7. Lightweight and Fast

• Written in Rust for performance and safety.
• Designed to handle thousands of concurrent connections efficiently.

8. Observability

• Exposes Prometheus-compatible metrics.
• Logging and monitoring features help track query distribution, pool usage, and replica health.
• Sample metrics endpoint settings:

curl http://localhost:9930/metrics

Hands-On Demonstration

Let’s set up the application and create an example using Docker Compose

version: '3.8'
x-postgres-common:
  &postgres-common
  image: postgres:14-alpine
  user: postgres
  restart: always
  healthcheck:
    test: 'pg_isready -U postgres --dbname=postgres'
    interval: 10s
    timeout: 5s
    retries: 5

services:
  postgres_primary:
    <<: *postgres-common
    ports:
      - 5434:5432
    environment:
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres
      POSTGRES_PASSWORD: postgres
    command: |
      postgres 
      -c wal_level=replica 
      -c hot_standby=on 
      -c max_wal_senders=10 
      -c max_replication_slots=10 
      -c hot_standby_feedback=on
      -c hba_file=/etc/postgresql/pg_hba.conf
      -c logging_collector=on
      -c log_directory=/var/lib/postgresql/data/logs
      -c log_filename=postgresql.log
      -c log_statement=all
      -c log_connections=on
      -c log_disconnections=on      
    volumes:
      - ./pg_hba.conf:/etc/postgresql/pg_hba.conf
      - ./00_init.sql:/docker-entrypoint-initdb.d/00_init.sql

  postgres_replica:
    <<: *postgres-common
    ports:
      - 5433:5432
    environment:
      PGUSER: replicator
      PGPASSWORD: replicator_password
    command: |
    
      bash -c "
      until pg_basebackup --pgdata=/var/lib/postgresql/data -R --slot=replication_slot --host=postgres_primary --port=5432
      do
      echo 'Waiting for primary to connect...'
      sleep 1s
      done
      echo 'Backup done, starting replica...'
      chmod 0700 /var/lib/postgresql/data

      exec postgres \
      -c logging_collector=on \
      -c log_directory=/var/lib/postgresql/data/logs \
      -c log_filename=postgresql.log \
      -c log_statement=all \
      -c log_connections=on \
      -c log_disconnections=on \
      -c log_destination=stderr
      "
    depends_on:
      - postgres_primary
  pgcat:
    build: .
    command:
      - "pgcat"
      - "/etc/pgcat/pgcat.toml"
    volumes:
      - "${PWD}/examples/docker/pgcat.toml:/etc/pgcat/pgcat.toml"
    ports:
      - "6432:6432"
      - "9930:9930"
    depends_on:
      - postgres_replica

init.sql

create user replicator with replication encrypted password 'replicator_password';
select pg_create_physical_replication_slot('replication_slot');


GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO replicator;
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT ALL ON TABLES TO replicator;

pgcat.toml.yml is the configuration file used to define PGCat settings
Configurations for Read-Write Splitting:

[general]
host = "0.0.0.0"
port = 6432

# Whether to enable prometheus exporter or not.
enable_prometheus_exporter = false

# Port at which prometheus exporter listens on.
prometheus_exporter_port = 9930

# How long to wait before aborting a server connection (ms).
connect_timeout = 5000

# How much time to give `SELECT 1` health check query to return with a result (ms).
healthcheck_timeout = 1000

# How long to keep connection available for immediate re-use, without running a healthcheck query on it
healthcheck_delay = 30000

# How much time to give clients during shutdown before forcibly killing client connections (ms).
shutdown_timeout = 60000

# For how long to ban a server if it fails a health check (seconds).
ban_time = 60 # seconds

# If we should log client connections
log_client_connections = false

# If we should log client disconnections
log_client_disconnections = false

admin_username = "postgres"
admin_password = "postgres"
[pools.postgres]
pool_mode = "transaction"
default_role = "any"


query_parser_enabled = true
query_parser_read_write_splitting = true

primary_reads_enabled = true
sharding_function = "pg_bigint_hash"
[pools.postgres.users.0]
username = "postgres"
password = "postgres"
pool_size = 50
statement_timeout = 0
[pools.postgres.shards.0]
servers = [
    [ "postgres_primary", 5432, "primary" ],
    [ "postgres_replica", 5432, "replica" ]
]
database = "postgres"

First, access the primary PostgreSQL container:

docker exec -it pgcat-main-postgres_primary-1 sh
psql -U postgres
CREATE TABLE product (
    id SERIAL PRIMARY KEY,
    product VARCHAR(255)
);
INSERT INTO product (product) VALUES ('Laptop');
INSERT INTO product (product) VALUES ('Smartphone');

Let’s run the container and connect to the PgCat using psql:

docker exec -it pgcat-main-postgres_replica-1 sh
psql -h 127.0.0.1 -p 6432 -U postgres 

Select * from product;

You can easily observe this in the secondary logs.

tail -f /var/lib/postgresql/data/logs/postgresql.log

Now, let’s continue exploring the configuration file in more detail.

Thread Settings

path: general.worker_threads
default: 5

Number of worker threads the Runtime will use (4 by default).

Pooling Settings

Furthermore, you can configure connection pooling on a per-user basis under the pools.<pool_name>.users.<user_index> section.

pools:
  my_pool:
    pool_mode: "transaction"  # Connection mode: "session" or "transaction"
    load_balancing_mode: "random"  # Load balancing algorithm: "random", "loc" (least connections)
    default_role: "any"  # Default routing: "any", "replica", "primary"
    query_parser_enabled: true  # Whether SQL query parsing is enabled
    primary_reads_enabled: true  # Whether primary can be used for read queries
    idle_timeout: 40000  # Connection idle timeout (milliseconds)
    connect_timeout: 3000  # Connection establishment timeout (milliseconds)

Multi Database Connection Settings

path: pools.database
default: "shard0"
path: pools.database2
default: "shard1"

primary_reads_enabled

If the query parser is enabled and this setting is enabled, the primary will be part of the pool of databases used for load balancing of read queries. Otherwise, the primary will only be used for write queries. The primary can always be explicitly selected with our custom protocol.

path: pools.<pool_name>.primary_reads_enabled
default: true

Conclusion

PGCat helps you scale your PostgreSQL infrastructure efficiently — with minimal configuration and maximum flexibility.If you’re looking for a transparent, performant way to enhance your database connectivity layer without rewriting your application logic, PGCat is definitely worth trying.

References
https://github.com/postgresml/pgcat

Example:
https://github.com/omerfarukicen/pgcat

PgCat: The Modern Proxy Solution for PostgreSQL was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

One of the key challenges in software development is understanding business needs clearly and turning them into effective technical solutions. Building a shared language between development teams and business stakeholders, and defining clear system boundaries, are essential parts of this process. Domain-Driven Design (DDD) offers a helpful approach to dealing with such complexities.

In this guide, we’ll explore the core ideas of DDD, how it connects naturally with microservice architecture, and how it can be adapted to both new and existing projects. The goal is to provide a perspective that supports better alignment between software and business understanding.

What Is Domain-Driven Design?

At its core, Domain-Driven Design (DDD) is a methodology for developing software based on the structure and language of the business domain. A domain refers to a specific area of knowledge or activity, such as logistics, e-commerce, or finance.

DDD focuses on building models that reflect this domain. One of the most powerful concepts in DDD is the Bounded Context — a logical boundary that encapsulates a specific part of the domain and its internal model. Within this boundary, the team uses a Ubiquitous Language shared between developers and business experts, enabling precise communication and understanding.

To effectively implement DDD, it’s crucial to grasp two complementary layers of design: tactical and strategic. Tactical patterns help shape the internal structure of your domain model, while strategic patterns guide how different models relate and interact across larger systems.

Tactical Patterns: Modelling the Domain

The diagram below visualizes the core tactical patterns in Domain-Driven Design within a bounded context. Entities and Value Objects make up the core of the domain layer. Aggregates group related entities and value objects to keep data consistent. Domain Services contain business logic that doesn’t naturally belong to entities. Application Services handle use cases and connect the domain with other systems. Repositories abstract data persistence. Infrastructure Services handle technical operations like messaging and storage. Together, these patterns create a well-structured, maintainable domain model aligned with business realities.

Strategic Patterns: Defining Context Relationships

While tactical patterns focus on the internals of a single model, strategic patterns define how multiple bounded contexts coexist and communicate within a larger system. Context Maps visualize and document these interactions between contexts.

By establishing clear boundaries and interaction styles, strategic patterns manage complexity and enable teams to develop independently while working together cohesively. These patterns help ensure models remain consistent, integration points are controlled, and the overall architecture scales effectively.

Bounded Context Relationship Patterns

With a solid understanding of DDD’s building blocks, the next step is to see how these ideas can drive architectural decisions — especially when designing distributed systems like microservices.

DDD and Microservices: A Natural Fit

When designing microservices, aligning each service with a bounded context is a smart strategy. Each microservice should encapsulate a specific business capability and operate independently — just like a bounded context in DDD.

This alignment allows for:

· Clear domain understanding within each team

· Loosely coupled development and deployments

· Independent release schedules aligned with each service’s business needs

Microservices also offer architectural benefits such as:

· Technology flexibility

· Failure isolation

· Elastic scalability

However, they also introduce challenges:

· Latency due to network communication

· Complexity in data consistency

· Challenging debugging and monitoring

· Higher operational overhead due to distributed infrastructure

To address these, services should communicate through APIs — using REST, GraphQL, or asynchronous messaging (e.g., Kafka or RabbitMQ) — and avoid direct code-level dependencies.

This is where Event-Driven Architecture plays a critical role. By modelling business events explicitly and sharing them across services asynchronously, event-driven design complements both DDD and microservices — enabling reactive systems that scale and evolve independently.

Event-Driven Design in DDD-Based Microservices

Events are key to building loosely coupled systems:

· Domain Events: Represent meaningful events in the business (e.g., OrderPlaced)

· Integration Events: Used to notify other systems or services about domain changes

Event-driven approaches facilitate eventual consistency and reactive workflows, enabling services to remain decoupled while responding to changes. This architecture is particularly useful when applying DDD principles at scale, allowing services to focus on their own bounded contexts and still react to important events elsewhere in the system.

Techniques like event storming can be used to collaboratively discover domain events and workflows, strengthening the link between business understanding and technical implementation.

Data Persistence and Isolation

Each microservice should own its data — aligning with its bounded context. Avoid shared databases.

Persistence Strategies

· Shared Database (anti-pattern)

· Separate Database per Service (recommended)

Challenges with separate DBs:

· Cross-service data consistency

· Distributed transactions

Solutions:

· CQRS (Command-Query Responsibility Segregation): Split write and read concerns

· SAGA Pattern: Handle long-running transactions using compensation steps

Applying DDD in Real Projects

Greenfield Projects

Fresh builds offer freedom in design. You can either:

· Start with microservices from day one (Ground-Up), or

· Begin with a monolith and split into microservices over time (Monolith-First)

Brownfield Projects

In legacy systems:

· You may refactor the existing monolith into modular microservices, or

· Replace legacy components one by one, managing technical debt along the way

Each strategy has trade-offs, but DDD principles help ensure alignment between the evolving architecture and the business domain.

Final Thoughts

Domain-Driven Design is not just a set of patterns — it’s a mindset. It brings your development efforts closer to business reality and aligns architecture with real-world complexity.

When combined with microservices, DDD offers a path toward maintainable, scalable, and meaningful software — enabling teams to work autonomously, evolve rapidly, and speak the same language as the business.

Aligning Architecture with Business Reality: A Practical Guide to Domain-Driven Design and… was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, we’ll explore how GitLab enables teams to implement DevSecOps practices, enhance collaboration between dev, sec, and ops teams, and deliver secure software faster.

What Is DevSecOps?

DevSecOps is a cultural and technical shift that integrates security practices within the DevOps process. Rather than treating security as a gate at the end of the development pipeline, DevSecOps encourages a “shift-left” mindset — bringing security earlier into the software development lifecycle (SDLC).

Why GitLab for DevSecOps?

GitLab isn’t just a Git repository manager anymore. It’s an end-to-end DevOps platform that also embeds security testing into the CI/CD pipeline. Instead of relying on a patchwork of third-party tools, GitLab offers a single application where you can code, test, secure, and deploy — all in one place.

Here’s how GitLab stands out:
Native security scans in the CI/CD pipeline
Auto remediation suggestions
Merge request approvals based on security findings
Policy-driven governance and compliance controls
Centralized visibility for security across projects

Built-In Security Scanning

GitLab provides a wide range of out-of-the-box security scanners, automatically triggered during the CI/CD process.

These include:
Static Application Security Testing (SAST): Analyzes source code for
vulnerabilities.
Dynamic Application Security Testing (DAST): Scans running applications for runtime issues.
Dependency Scanning: Detects known vulnerabilities in dependencies (e.g., npm, pip, Maven).
Container Scanning: Inspects Docker images for security issues.
License Compliance: Ensures dependencies meet your license policies.
Secret Detection: Flags hardcoded secrets like API keys or passwords.

Each scan runs as a job in your CI pipeline and returns detailed results directly in the merge request. Developers can act on these findings without switching tools or waiting for manual reviews.

Security in Merge Requests

One of GitLab’s strongest features is its tight integration between security and merge requests. When a developer creates a merge request, GitLab can:
Run all enabled security scans.
Display scan results inline.
Automatically block the merge if high or critical vulnerabilities are found.
Suggest fixes or safer versions of packages.

This allows security to become part of the daily workflow, not a bottleneck down the line.

Compliance and Governance

For regulated industries, GitLab offers compliance frameworks, audit logs, and security policies that help ensure traceability and enforce best practices. Security policies can be configured to enforce scan coverage, block deployments, or require approvals from security reviewers. The Security Dashboard gives security teams visibility across projects, helping them identify risk areas and prioritize remediation without disrupting development velocity.

Developer-Friendly Security

DevSecOps only works when developers adopt it — and GitLab makes that easy. The interface is familiar, the feedback is fast, and the remediation suggestions are actionable. By integrating everything into the existing GitLab workflow, there’s no need to jump between tools or wait on long security reviews. Security becomes just another part of CI/CD — automated, consistent, and built-in.

Final Thoughts

With cyber threats evolving and software delivery accelerating, integrating security into your development pipeline is not optional — it’s essential. GitLab’s DevSecOps capabilities make it easy to bake security into every commit, without slowing down your team.

If you’re already using GitLab, many of these features are just a .gitlab-ci.yml snippet away. If not, it’s worth exploring how GitLab can unify your DevOps and security efforts into a single, streamlined workflow.

DevSecOps with GitLab: Shifting Security Left Without Breaking Your Pipeline was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

Understanding the Basics
Sam Newman, a renowned expert in the field, begins by laying a solid foundation. He explains what monolithic applications are; self-contained systems where all components are interconnected and interdependent. While monoliths can be simpler to develop initially, they often become cumbersome to manage, scale, and update as they grow.

In contrast, microservices architecture breaks down applications into smaller, independent services. Each service handles a specific function and communicates with other services through APIs. This modular approach offers several benefits, including easier scalability, improved fault isolation, and the ability to deploy updates more frequently.

The Case for Evolution
One of the key themes in Newman’s book is the concept of evolutionary transformation. Instead of a risky, all-at-once migration, Newman advocates for an incremental approach. This method allows organizations to gradually transition to microservices while maintaining stability and minimizing disruptions.

Newman emphasizes the importance of understanding the existing system thoroughly before making any changes. This includes identifying the pain points, understanding the business domain, and mapping out dependencies. By doing so, teams can make informed decisions about where to start and how to prioritize the transition steps.

Patterns for Decomposition
Newman introduces several evolutionary patterns for decomposing monoliths into microservices. Each pattern addresses specific challenges and offers a structured approach to breaking down monolithic systems.

1- Strangler Fig Pattern
Inspired by the strangler fig tree, this pattern involves gradually replacing parts of the monolith with microservices. New functionality is built as microservices, while existing components are incrementally replaced. This approach allows teams to deliver value continuously while slowly transforming the architecture.

2- Branch by Abstraction
This pattern involves creating an abstraction layer between the monolith and the new microservices. By introducing this layer, teams can decouple components and incrementally migrate functionality. It helps in managing dependencies and ensures that the system remains functional throughout the transition.

3- Anticorruption Layer
When integrating microservices with a monolithic system, inconsistencies in data models and interfaces can create challenges. The anticorruption layer acts as a mediator, translating and adapting between the two systems. This pattern helps maintain data integrity and prevents the monolith’s complexity from leaking into the microservices.

Managing Data in a Microservices World
One of the significant challenges in microservices architecture is data management. Newman discusses various strategies to handle data in a distributed system. He highlights the importance of maintaining data consistency and integrity while ensuring that services remain loosely coupled.

1- Database per Service
A key principle in microservices is that each service should have its own database. This approach ensures that services are independent and can evolve without affecting others. However, it also introduces complexities in managing transactions and maintaining consistency across services.

2- Event Sourcing and CQRS
Newman explores advanced techniques like Event Sourcing and Command Query Responsibility Segregation (CQRS). Event Sourcing involves storing changes as a series of events, allowing services to reconstruct the current state from these events. CQRS separates read and write operations, enabling different models for querying and updating data. These techniques can help manage data consistency and scalability in a microservices architecture.

Embracing Change and Continuous Improvement
Transitioning to microservices is not a one-time project but an ongoing journey. Newman stresses the importance of a culture that embraces change and continuous improvement. This includes investing in automation, monitoring, and robust testing practices.

1- Automation and DevOps
Automation plays a crucial role in managing the complexity of microservices. Newman highlights the need for automated testing, continuous integration, and continuous deployment. These practices ensure that changes can be delivered quickly and reliably, reducing the risk of introducing errors.

2- Monitoring and Observability
In a microservices architecture, understanding the system’s health and performance is vital. Newman emphasizes the importance of monitoring and observability. By collecting and analyzing metrics, logs, and traces, teams can gain insights into the system’s behavior and identify issues before they impact users.

Conclusion
“Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith” by Sam Newman is an invaluable resource for any organization looking to embark on the journey to microservices. Newman’s pragmatic approach, grounded in real-world experience, offers actionable insights and strategies for navigating this complex transition.

By embracing evolutionary patterns, managing data effectively, and fostering a culture of continuous improvement, organizations can unlock the full potential of microservices. The journey may be challenging, but with the right guidance, it can lead to a more scalable, resilient, and agile software architecture.

If you’re considering or currently navigating the shift from monolith to microservices, Newman’s book is a must-read. It not only provides a roadmap for the transition but also equips you with the knowledge to tackle the inevitable challenges along the way.

Transforming Monoliths to Microservices: Insights from Sam Newman was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is a Multi-Cloud Strategy?

A multi-cloud strategy refers to the practice of using cloud services from multiple providers, such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), and others, to meet different business needs. This differs from a hybrid cloud strategy, which typically combines private and public clouds. In a multi-cloud setup, an organization may choose to use AWS for its storage needs, Azure for its artificial intelligence tools, and GCP for its data analytics services.

The key appeal of this approach is the ability to avoid vendor lock-in, ensuring businesses can select the best tools from each provider based on their specific requirements.

Benefits of a Multi-Cloud Strategy

1. Avoiding Vendor Lock-In: Relying on a single cloud provider can limit flexibility and innovation. Multi-cloud allows businesses to pick and choose services that are best suited to their needs without being tied to one provider. This also provides leverage when negotiating pricing and service terms.
2. Enhanced Resilience and Redundancy: Distributing workloads across multiple cloud platforms ensures that if one service provider faces downtime, others can maintain business continuity. This enhances system resilience, minimizing the impact of service outages.
3. Optimization of Costs: Different cloud providers offer unique pricing models. A multi-cloud strategy allows businesses to allocate resources in the most cost-efficient way by selecting the best pricing options across various platforms.
4. Flexibility and Innovation: Every cloud platform excels in different areas. AWS might offer superior storage solutions, while Azure might have better integration with enterprise tools. By using multiple clouds, companies can leverage the best-in-class solutions for each aspect of their operations.
5. Geographic Reach: Some cloud providers may have a stronger presence in certain regions. With a multi-cloud approach, organizations can optimize their infrastructure and services to meet local compliance regulations and latency requirements.

Challenges of a Multi-Cloud Strategy

While the benefits of a multi-cloud approach are clear, implementing this strategy comes with challenges that businesses need to address.

1. Complexity in Management: Managing multiple cloud environments can be complex. Each provider has its own APIs, interfaces, and management tools. This can increase the operational overhead and require specialized skills in different cloud platforms.
2. Security Risks: A multi-cloud environment can expose an organization to additional security risks. With data spread across multiple providers, businesses need to ensure consistent security policies, identity management, and encryption standards across all platforms.
3. Increased Costs: While a multi-cloud strategy can lead to cost savings, it can also lead to cost overruns if not managed carefully. Without proper governance, businesses might end up paying for redundant services or over-provisioning resources across multiple clouds.
4. Data Integration and Migration: Moving data between different cloud providers can be challenging due to varying data formats and protocols. Ensuring seamless integration between services hosted on different clouds requires careful planning and often involves the use of third-party tools.

Best Practices for a Successful Multi-Cloud Strategy

1. Centralized Monitoring and Management: Invest in tools that provide centralized monitoring and management across all cloud platforms. This helps streamline operations, improve visibility, and ensure consistent performance across environments.
2. Cloud-Native Applications: When building applications, adopt cloud-native development practices. Containerization, microservices, and serverless architectures are particularly well-suited to multi-cloud environments, as they allow for easier migration and scaling across platforms.
3. Strong Governance and Cost Management: Implement clear governance policies for provisioning, managing, and decommissioning cloud resources. Regularly monitor cloud usage to avoid sprawl and manage costs effectively.
4. Consistent Security Policies: Use a unified security approach across all platforms. This includes ensuring that access controls, encryption, and compliance standards are uniformly applied across each cloud provider.
5. Leverage Automation: Automation can simplify the management of multi-cloud environments. By automating tasks like resource scaling, backups, and patching, businesses can reduce operational complexity and ensure smoother performance.

Conclusion

A multi-cloud strategy offers businesses a powerful way to increase flexibility, optimize costs, and reduce risk. However, successfully implementing this approach requires careful planning, robust management, and consistent security practices. As organizations continue to rely on cloud services for their digital transformation efforts, adopting a multi-cloud strategy may prove to be a critical advantage in staying competitive and resilient in a rapidly changing technological landscape.

Navigating the Future of IT with a Multi-Cloud Strategy was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, we will dive into what serverless computing is, its benefits, and its practical applications.

What is Serverless Computing?

Despite its name, “serverless” doesn’t mean there are no servers involved. Instead, it refers to the fact that developers don’t need to manage or provision any servers. In a traditional cloud model, developers must configure and manage servers to run their applications, whereas serverless computing abstracts this responsibility entirely.

Serverless computing is powered by Function-as-a-Service (FaaS) platforms like AWS Lambda, Google Cloud Functions, and Azure Functions. In these platforms, you write individual functions that are triggered by events, such as HTTP requests or database updates, and the cloud provider automatically scales the underlying infrastructure to handle these requests.

Key Characteristics of Serverless

1. No Server Management: Cloud providers manage the infrastructure, so developers don’t need to worry about server maintenance, patching, or scaling.
2. Scalability: Serverless functions automatically scale based on demand. Whether you have one user or one million, the system adjusts dynamically to meet the demand.
3. Pay-as-You-Go Pricing: You only pay for the resources your application uses, down to the millisecond of execution time, which makes serverless computing highly cost-effective.
4. Event-Driven Architecture: Serverless architectures respond to specific events, such as an API call, file upload, or a database trigger. This allows for highly responsive and flexible applications.

The Benefits of Serverless Computing

1. Cost Efficiency
   One of the most attractive features of serverless computing is its pricing model. With traditional cloud computing, you pay for a server instance, even if it’s idle most of the time. In contrast, with serverless, you only pay for the exact execution time of your functions. This can lead to substantial cost savings, particularly for applications with variable traffic.
2. Scalability Without Complexity
   Scaling an application to meet fluctuating demand can be challenging in a traditional server-based environment. With serverless computing, scalability happens automatically. There’s no need for developers to configure scaling rules or worry about over-provisioning resources. The cloud provider manages all of this behind the scenes.
3. Faster Time to Market
   Since developers don’t need to worry about infrastructure management, they can focus entirely on writing code. This speeds up the development process, allowing teams to iterate faster and bring products to market quicker. Additionally, the serverless approach allows teams to adopt a microservices architecture, which can streamline development and enable more agile workflows.
4. Focus on Core Business Logic
   Serverless computing abstracts away much of the operational burden, allowing developers to focus on building features and improving user experience rather than managing infrastructure. This focus on business logic and innovation is a key driver of productivity gains in serverless environments.

Use Cases for Serverless Computing

1. Web Applications
   Serverless is well-suited for web applications, particularly those with unpredictable traffic patterns. For example, a news website might experience a surge in traffic during a breaking news event. With serverless, the application can scale instantly to accommodate millions of users without any intervention from developers.
2. APIs and Microservices
   Serverless architectures work seamlessly with APIs and microservices, where small, independent functions handle specific tasks. This is especially useful for projects built using a microservices approach, where each service operates independently, making serverless computing an ideal match.
3. Data Processing
   Serverless platforms are often used for data processing tasks, such as batch jobs, ETL (extract, transform, load) operations, and real-time data analysis. These processes can be triggered by events, such as a new file being uploaded to a storage bucket, and can scale to handle large volumes of data efficiently.
4. IoT Backends
   With the rise of IoT (Internet of Things) devices, serverless computing is a great choice for handling backend processing. These devices generate massive amounts of data, and serverless systems can respond to incoming data streams without needing a constantly running server.

Challenges of Serverless Computing

While serverless offers many advantages, it’s not without its challenges. Cold starts — when a function is invoked after being idle for a while — can lead to higher latency. There are also limitations around execution time and memory, making serverless unsuitable for certain types of long-running or resource-intensive tasks.

Additionally, the shift to event-driven architectures may require significant refactoring of existing codebases, which could be a barrier for teams with legacy applications.

Conclusion

Serverless computing is a game-changer in the world of cloud development. Its ability to eliminate infrastructure management, provide automatic scalability, and offer cost-efficient pricing makes it an attractive option for many use cases, from web apps to data processing.

However, like any technology, it’s essential to understand both its advantages and limitations. As serverless continues to evolve, it’s poised to become a core pillar of cloud architecture in the years to come.

By leveraging serverless platforms, developers can focus more on innovation and less on operations, ultimately delivering better experiences to users faster. Whether you’re building a simple API or a complex IoT backend, serverless computing can help you get there with minimal infrastructure overhead.

Understanding Serverless Computing: A New Era for Cloud Development was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the ever-evolving landscape of cloud computing, businesses are reaping unprecedented benefits from the flexibility, scalability, and innovation that cloud services provide. However, with these advantages comes a significant challenge: managing cloud costs effectively. Enter FinOps, a transformative practice designed to bridge the gap between finance, operations, and technology. In this article, we’ll explore what FinOps is, why it’s essential, and how organizations can implement it to optimize their cloud investments.

What is FinOps?

FinOps, short for Financial Operations, is a practice that combines financial management principles with the operational efficiencies of DevOps. At its core, FinOps is about creating a culture of accountability and collaboration between engineering, finance, and business teams to manage cloud spend effectively. It involves continuous monitoring, analyzing, and optimizing cloud usage to ensure that every dollar spent brings maximum value.

The Phases of FinOps

FinOps implementation is typically broken down into three distinct phases, each building upon the other to create a cohesive and effective strategy:

Inform

This phase is about achieving visibility and transparency into cloud usage and costs. It involves setting up the necessary tools and processes to collect data and generate reports. Key activities include tracking cloud spend, establishing cost allocation and tagging strategies, and creating dashboards for real-time insights.

Optimize

With visibility in place, the next phase focuses on cost optimization. This includes identifying inefficiencies, such as underutilized resources or redundant services, and implementing strategies to reduce waste. Activities in this phase might include rightsizing instances, scheduling non-essential resources to shut down during off-hours, and leveraging discounts and reserved instances.

Operate

The final phase is about operationalizing FinOps practices and embedding them into the organization’s culture. This includes continuous monitoring and improvement, regular reviews of cloud costs and usage, and fostering a culture of accountability. Key activities include setting up governance policies, conducting cost audits, and ensuring all teams are aligned with FinOps principles.

The Core Principles of FinOps

FinOps is built on three core principles

1.Collaboration: FinOps requires a collaborative effort across finance, operations, and technology teams. This ensures that financial decisions are made with a full understanding of the technical requirements and implications.

2.Ownership: Engineering and development teams are given ownership of their cloud usage and costs, encouraging them to take responsibility for optimizing their spend.

3.Continuous Improvement: FinOps is not a one-time effort but an ongoing process. It involves continuous monitoring, analysis, and optimization to ensure that cloud investments are always aligned with business goals.

Why is FinOps Important?

The shift to cloud computing has brought unprecedented flexibility and scalability. However, it also introduces complexity in managing costs. Traditional budgeting methods are often ill-equipped to handle the dynamic nature of cloud spending. Here’s why FinOps is essential:

Cost Control: FinOps helps prevent cloud overspend by identifying and mitigating inefficiencies.

Agility: It supports rapid innovation by providing teams with the financial insights needed to make quick, data-driven decisions.

Accountability: By making costs visible and attributable to specific teams or projects, FinOps ensures that everyone is accountable for their cloud usage and spending.

Implementing FinOps in Your Organization

Adopting FinOps requires a shift in mindset and processes. Here are some steps to get started:

1.Establish a FinOps Team: Form a cross-functional team with representatives from finance, engineering, and business units. This team will be responsible for driving FinOps initiatives and ensuring collaboration across departments.

2.Set Clear Goals: Define what success looks like for your FinOps practice. This could include specific cost-saving targets, efficiency improvements, or other KPIs.

3.Leverage Tools and Automation: Use FinOps tools to gain insights into cloud usage and costs. Automation can help streamline processes such as cost allocation, budget alerts, and optimization recommendations.

4.Educate and Empower Teams: Provide training and resources to help teams understand the importance of FinOps and how they can contribute to cost-efficiency. Empower them with the tools and data they need to make informed decisions.

5.Continuous Improvement: FinOps is not a one-time project but an ongoing practice. Regularly review and refine your processes, metrics, and goals to adapt to changing business needs and cloud environments.

Real-World Impact of FinOps

Many organizations have successfully implemented FinOps to achieve significant cost savings and operational efficiencies. For instance, media company **A** reduced their cloud spend by 25% in the first year of adopting FinOps. By continuously monitoring their usage and implementing automated cost-optimization strategies, they were able to eliminate waste and invest more in innovation.

Tech giant **B** used FinOps to gain better visibility into their cloud costs across various departments. This transparency enabled them to allocate budgets more accurately and hold teams accountable for their spending, leading to a more efficient and predictable financial model.

Conclusion

As cloud adoption continues to grow, so does the need for effective financial management. FinOps offers a robust framework to manage cloud costs, drive efficiency, and foster a culture of accountability. By unifying finance and DevOps, FinOps ensures that cloud spending is aligned with business goals and maximizes the value of your cloud investment.

If your organization is looking to optimize its cloud strategy, consider embracing FinOps. It’s not just about cutting costs — it’s about making smarter, data-driven decisions that support your business’s growth and innovation.

Note: This article is the first part of a three part series. The next part is here.

Unlocking Financial Efficiency in the Cloud Era: FinOps was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

Note: This is the last part of a three part series. The first part is here, and the previous part is here.

04- Compute

🗹 Right-size your resources/services and utilize auto-scaling whenever available:

These two basic topics are strongly coupled and form the basic architectural principle for cloud, so I won’t include an explanation for them (you can find a lot in aws docs and whitepapers). But one thing I should remind, is that, (application) auto scaling is available beyond EC2 (Fargate, Lambda), even beyond compute (RDS, Search …). Also, don’t forget to check our “Tools to observe your cloud expenditure” chapter to find opportunities for right-sizing.

Ease: 3/5: The setups are ingrained into architecture, most of the time.

Effect: 5/5: You’ll definitely be missing a lot if you’re not auto scaling and/or right sizing your resources. So, this is one of the MUST DOs.

🗹 Use (Compute) Savings Plans:

EC2 has the reserved instances option, which can save 30%-40% for one year commitments and 50%-70% for 3 year commitments. It is definitely a huge gain if you know you will need those instances. If you specify also the zones the instances will be created in, those instances will literally be reserved for you (capacity reservations), meaning you’ll most definitely will be assigned that instance whenever you need (as long as you are consuming/using that plan). So, you can utilize/use newer instance types (processor, network ..etc) with zonal plans, which are more (cost-wise) efficient than older ones. If your reserved instance totals cost more than $500 K, you get more and more volume discounts, step-by-step. If you do not need your reserved instances anymore, you can sell them on the marketplace.

Compute Savings Plans are available for all compute resources, including not only EC2, but Fargate and Lambda, as well. Compute Savings Plans do not have capacity reservations, by default (they can be enabled manually), but the (cost) gains are very similar to reserved instances. Compute Saving Plans are also more flexible and customizable than Reserved Instances but cannot be put on marketplace, as of now.

Savings Plan pages have a purchase recommendations page, which you can base your plans:

You should monitor your coverage for saving plans (and also reserved instances); coverage shows how many of your (candidate) resources are covered by your plans:

You should also check utilization reports for your savings plans (and also reserved instances); utilization shows the percentage of your reservations you are using:

You can also create alarms for coverage and utilization and get notified whenever your usage drops below a predefined value.

Ease: 4/5: It is easy to enable these, but planning effectively requires time and not everyone is ready to plan for commitment.

Effect: 5/5: If planned with care, this is one of the MUST DOs, for long term commitment.

🗹 Use Spot Fleets:

Compute Savings Plan can be a good choice for long term commitments but what about intermittent compute needs (like report generation, data science pipelines, … etc) ? You can use spot instances for these and similar purposes. For fargate it is called fargate spot. You can set a price for the instance type you want, whenever the current spot price drops below your price, you acquire that instance. You can view pricing history for different zones and instance properties:

The drawback of spot instances is that, they can be (terminated) and taken away from you whenever AWS needs them. This may seem intimidating, but actually less than 5% of Spot instances are terminated in any given month; so they are (in practice) fairly reliable. You can form machine pools for your daytime workloads, called spot-fleets, combining savings plan (or reserved) instances and spot instances. This way, your always-on instances take your (ceil) idle load and spot instances kick in when there’s spike/increase in load. This actually is becoming the go to architecture even for kubernetes workloads, be it EC2 or Fargate. This way, you get the full cost benefit of long-term commitment and short-term workload provisioning.

Ease: 3/5: It requires a bit of work.

Effect: 5/5: If planned with care, this is also one of the MUST DOs.

Network:

🗹 Use CloudFront:

CloudFront’s main use case is content delivery at the edge. If you have lots of cachable (eg: static, get requests, …etc) content in your services, it will help produce low latency and high performance responses to your users. This caching comes at a price but reduces your compute and network costs, even sometimes by a heavy tradeoff margin.

If you’re using AWS WAF with CloudFront (and you’ll mostly be), you can also purchase a Savings Bundle.

Ease: 2/5: It is that not easy to setup and manage.

Effect: 3/5: If planned with care, can result in a fair amount of cost reduction.

🗹 Choose network components wisely:

It’s better to check how AWS network costs occur, before making any decisions. Some network interplay may require additional components, like VPN, transit gateway, private-link, …etc. Most of them come with an always-on cost, accumulated in a timely manner, and a pay-as-you-go cost, accumulated whenever network packets pass through. There may be more than one possible (alternative) component for your needs. For example; gateway endpoints (which are only available for S3 and DynamoDB, for now,) are free but interface endpoints have both types of costs.

As another example, internet gateway does not incur any cost (except net-in and out), itself, but NAT Gateway has both types of costs:

Ease: 1/5: It is that not easy to setup, manage and keep track.

Effect: 3/5: If you do not keep track, you may face with a lot of extra network costs.

Summary

The complete checklist, with ease and effect scores, is summarized in the below table:

References:

• https://learn.finops.org
• AWS FinOps Simplified: Eliminate cloud waste through practical FinOps, by Peter Chung
• https://docs.aws.amazon.com
• https://aws.amazon.com/whitepapers
• Cloud FinOps: Collaborative, Real-Time Cloud Financial Management, by J. R. Storment, Mike Fuller.

Note: This is the last part of a three part series. The first part is here, and the previous part is here.

Cost optimisation checklist for AWS — Part 2/2 was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

Note: This is the second part of a three part series. The first part is here, and the next part is here.

AWS entered our lives many years ago and made them a lot easier. It started with simple services, like object storage and virtual machines, then a lot of managed services are put in service. But these all come at a cost (pun intended), and with a little more attention and a bit of work, you can reduce a fair amount of your cloud bills. This article series intend to give you a checklist for quick and simple advices to do that, in an organized manner. I tried to be brief and concise, so there are hyperlinks to detailed documentation links and examples, wherever applicable.

“Cost optimisation checklist for AWS” comprises the last two parts of the three part FinOps series and contains five sections. First section, “Tools to observe your cloud expenditure” introduces AWS’s own tools that help you to understand your expenditure so that you can act on avoidable ones with the tools you learn in the following sections. Second section lists general items. Third section explains cost reduction tips on storage and db related services. Fourth section shows tips on compute related services. Finally, the last section tries to make you understand your network related costs and adds a summary part.

01 — Tools to observe your cloud expenditure:

This chapter is all about the “Inform” phase of FinOps. Premature optimisation is a trap to avoid, so we should better gather information.

🗹 Use Cost Explorer & Reports:

Cost explorer is the main resource to find where excess costs are occurring in your account. After you enable it, you can view current expenditures and forecasts, both using graphs and detailed reports.

Ease: 3/5 : Dashboard style general reporting is easy to understand but you may have to dive deep into reports to make fine grained decisions.

Effect: 2/5: First few visits may reveal big gains but after applying other items (in this list), you will feel you don’t need that much any more.

🗹 Use Budget alarms:

You can add cost budgets (and alarms), which inform you if your current costs are above a set threshold. Budget alarms can also be used to track (and fire alarms for future) cost estimations, in addition to actual costs. Below image shows the steps to create cost bucgets, and in the subsequent steps, budget alarms.

Ease: 4/5: It is very easy to set a few budget alarms

Effect: 1/5: Alarms do not cause/show a direct gain but can be anchor to your spending, if you’ve mistakenly enabled something.

🗹 Use (cost allocation) tags

Adding (cost related) tags to resources makes it easy to group and track related costs.You can then use cost explorer and see your reports grouped by those tags.

Ease: 2/5: It is not that easy to set rulesets and usage restrictions for cost allocation tags.

Effect: 2/5: This does not directly reduce your spendings but gives a better view of them.

🗹 Use (Cost section of) Trusted Advisor:

Trusted Advisor gives advices for a handful of sections and cost section is one of these. It can be an easy visit, before you dive into more advanced tools.

Ease: 5/5: It is very easy to enable it.

Effect: 3/5: It can actually give pretty good advices.

🗹 Watch metrics & logs:

CloudWatch Metrics give information about resource utilization and can direct you to unexpected resource usage. Application logs and access logs may give information when unexpected costs occur. As an example; your load balancer may be under complicated DDoS or bot attack and you can detect this via tracing ELB access logs, ELB and compute metrics. You can even create metric or log alarms, if you see a pattern.

Ease: 1/5: Metrics and logs are readily available for some services but it can be hard to extract useful information out of them. It usually needs time and experience to do it efficiently.

Effect: 3/5: It can detect many areas where others fail.

🗹 Use Storage Lens:

S3 Storage Lens can be used to get an overview of S3 usage, current and historical.

Interrupted multipart uploads can cause unnecessary S3 storage costs. You can scroll down to “Top N overview” in the storage lens dashboard window and find “Incomplete MPU Bytes” which depicts the amount of incomplete multipart uploads, as bytes. But there is a better solution to this: You can configure an S3 bucket lifecycle policy to remove old incomplete MPUs automatically.

Ease: 3/5: It has a useful dashboard but going into details can be tiresome.

Effect: 2/5: It is service specific and has a limited focus.

02 — General

🗹 Use AWS Organisations:

In addition to making it easy to manage and organize your accounts and resources, using AWS Organizations can also have a direct effect of reduced costs, for some resources. S3 is an example. The more objects you store in S3, your unit pricing drops, step by step.

Below shows sample steppings:

If you combine all (or most) of your accounts in AWS Organisations and utilize consolidated billing, AWS behaves all your resource usage and expenditures as if they are coming from a single account, so your costs hit reduced pricing steps earlier. Below is an example calculated for storing 100GB (on the average) in 10 accounts:

Total cost, without consolidated billing, is 50000*0.023 + 50000*0.022 = $2250 per account, so $22500 for 10 accounts.

If consolidated billing is used, it will be 50000*0.023+450000*0.022+500000*0.021 = $21550 for 10 accounts, which is $1000 lower than sans consolidated billing.

AWS Organisations have also other cost related benefits, such as sharing reserved instances and saving plans across accounts, so you may get more benefits from other items by enabling and setting up AWS Organisations.

Ease: 2/5: It’s main purpose is not about cost and It is may be difficult to setup, if you’ve not done before.

Effect: 1/5: Has a limited cost reduction rate.

🗹 Work with an AWS Partner:

AWS has a partnership program, with more than two thousand companies already enrolled in. If you work with an advanced or premium tier service partner, you will most probably be entitled to discounts, amount changing according to your use case. You can find a partner here.

Ease: 5/5: You won’t be doing much.

Effect: 2/5

03- Storage and DB

🗹 Use S3 storage classes and lifecycle policies:

S3 has different storage classes that have different storage costs and access costs. They are inversely related (classes having lower storage prices have higher access prices). Rarely accessed data can be moved to Glacier storage classes.

There is also a new class, called intelligent tiering, which automatically moves between tiers (except Glacier classes, afaik). It can also be used, if it seems tedious to observe and switch tiers. But in this case, S3 lifecycle policies can also be utilized:

Note that, each tier switch, manual or by policy, costs around $0.01 for a thousand requests (objects).

Ease: 4/5: Setting up lifecycle policies is not that difficult. Intelligent tiering also automates some of the work..

Effect: 3/5: You can achieve quite a lot of gains if you are storing lots of large objects..

🗹 Use Reserved Instances for datastore services:

“Compute Savings Plan” is the preferred path to go for compute services in AWS but Reserved Instances is still the choice for RDS, OpenSearch and the services like (since there’s no compute savings option for them, yet). Keep in mind that, reserving intra-region instances has more advantages, like using them as stand-by replicas and being able to be more specific for instance types (since AZs will already be selected). It is also advised to enable auto-expand for all instances.

Ease: 3/5

Effect: 4/5: Reserved instances usually have high returns, especially with high utilization.

🗹 Use OpenSearch ultra-warm nodes:

In addition to using reserved instances for OpenSearch nodes, you can also enable ultra-warm storage (which utilize S3), wherever applicable. Note that ultra-warm nodes can only be used for read-only indexes.

Ease: 2/5: Monitoring and weigthing the effectiveness may be difficult.

Effect: 2/5: Has limited use cases..

🗹 Use DynamoDB capacity modes and reserved capacity:

If you are using DynamoDB, you’ll already be using one of the capacity modes.Default is the on-demand mode.You can switch to provisioned capacity and provision read + write units after viewing ConsumedReadCapacityUnits and ConsumedWriteCapacityUnits DynamoDB metrics. A better way is to enable (provisioned capacity) autoscaling:

Below is an active monitoring of how auto-scaling works. If you define a larger target utilization, you pay less but you risk hitting the capacity on usage spikes (eg: check ~12 UTC at the graph):

Ease: 4/5: Not that difficult to enable and monitor provisioned capacity, and auto scaling reduces the risks.

Effect: 3/5: Definitely provides some cost gains.

🗹 Use EBS Snapshot Archives and EBS Recycle Bin:

You will probably be storing EBS snapshots, if you are working with EC2 instances a lot, on production. It is one of the easiest methods of backing up instances and a way of creating golden AMIs.

These backups can be archieved in a region after a while (eg: 3 months) and sent to the requesting region(s) whenever required. EBS Recyle bin provides a way to send these archives to trash (eg: after a year), with an option to recover, for a limited time period. You can also define retention rules for archives and recycle bin, to automate these processes.

Ease: 4/5: Setting up rules and archives is easy.

Effect: 2/5: You can achieve a bit of gain, to an extend, if you are keeping a lot of images.

🗹 Use EFS storage classes:

EFS has storage classes, similar to S3. EFS also have intelligent tiering and lifecycle policies:

Ease: 4/5: Setting up lifecycle policies is not that difficult. Intelligent tiering also automates some of the work.

Effect: 2/5: You can achieve quite a lot of gains if you are storing lots of large objects but EFS is not as widespread as S3.

Note: This is the second part of a three part series. The first part is here, and the next part is here.

Cost optimisation checklist for AWS — Part 1/2 was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.

Github: https://github.com/obss/BIOBSS

What is BIOBSS used for?

Wearable sensors are becoming more and more popular these days, and with that, comes the need to process the signals these sensors collect. To help with this, several open-source tools and packages have been developed, and BIOBSS is a relatively new one among them, developed by OBSS R&D team.

BIOBSS is a signal processing and feature extraction library that can process electrocardiography (ECG), photoplethysmography (PPG), electrodermal activity (EDA), and 3-axis acceleration (ACC) signals. With BIOBSS, you can create end-to-end pipelines by adding the necessary processes from BIOBSS or other Python packages.

Here are the things you can do with BIOBSS:

• Applying basic preprocessing steps (*)
• Assessing the quality of PPG and ECG signals
• Extracting features from ECG, PPG, EDA, and ACC signals
• Performing Heart Rate Variability (HRV) analysis using PPG and ECG signals
• Extracting respiratory signals from PPG or ECG signals and estimating respiratory rate (*)
• Calculating activity indices from ACC signals
• Creating and saving pipelines

(*): Not all methods were implemented from scratch but imported from existing packages

In simple words, BIOBSS is a tool that can help you have a deeper look and insight into the data your wearable sensor collects by processing and analyzing it.

What sets BIOBSS apart from similar packages?

BIOBSS has some functionalities that set it apart from its peers. Below, you can see a table that compares existing packages with BIOBSS and highlights its unique functionalities.

Here are the key features that make BIOBSS stand out:

1) One of the biggest advantage of BIOBSS is its ability to create end-to-end pipelines for signal processing and feature extraction. This means, you can use the functions independently or string them together to create a custom pipeline that fits your specific needs. Other Python packages (e.g. Neurokit2) may have the option to splice functions together, but BIOBSS’s well-structured pipeline module makes it easy to define input/output signals and add processes. Additionally, BIOBSS’s pipeline module has the flexibility of incorporating processes from both BIOBSS’s own modules and other external Python packages that may better suit your needs.

2) With BIOBSS, you can extract commonly used features from ECG, PPG, EDA and ACC signals. To the best of our knowledge, BIOBSS is the only package to calculate PPG features. It also has modules for fiducial point detection and feature extraction of VPG (first derivative of PPG) and APG (second derivative of PPG) signals.

3) BIOBSS can calculate commonly used activity metrics from ACC signals by processing raw acceleration signals, properly, for each index. While other Python packages may calculate specific activity metrics, BIOBSS has the advantage of preprocessing the raw signal properly and returning multiple metrics calculated for various datasets (preprocessing alternatives) at one time.

4) BIOBSS enables you to evaluate the quality of ECG and PPG signals by utilizing morphological filtering. You can apply rule-based steps and a template matching method either separately or together. While the signal quality assessment module requires further development, it is still a useful tool for those who want to understand the standards a high-quality signal should meet.

How to install and import BIOBSS?

You can install BIOBSS via pip:

pip install biobss

or build from source,

git clone https://github.com/obss/biobss.git
cd BIOBSS
python setup.py install

To import BIOBSS:

import biobss

How to use BIOBSS?

We created tutorial notebooks for specific signal processing and feature extraction scenarios. You can have a look at them and start using BIOBSS for your needs.

• For tutorial notebooks :

https://github.com/obss/BIOBSS/tree/main/examples

To learn more about BIOBSS, you can also visit the official documentation page:

https://biobss.readthedocs.io/en/latest/

Final words

BIOBSS is a versatile package that can help you process signals from wearable sensors in a variety of ways. Whether you’re looking to extract features, create pipelines, or assess signal quality; BIOBSS has the tools you need to get the job done. It is important to note that, BIOBSS is still under development and we are working to improve existing functionalities and add new features to it.

Stay tuned!

Acknowledgement: Thanks to Devrim Çavuşoğlu and Cagatay Tasci for their valuable feedback.

BIOBSS: A biological signal processing and feature extraction library was originally published in Codable on Medium, where people are continuing the conversation by highlighting and responding to this story.