Summary

At Inato, we migrated from fp-ts to Effect in early 2024. Given our substantial codebase (around 500k lines of typescript code), we needed a way of ensuring any new code could be written using Effect while allowing existing fp-ts code to coexist. We achieved this goal in just two months, dedicating around 10% of our time to it. In this article, you will find our detailed migration strategy, the helpers we developed (which you can find in this repository), and how we ensured a smooth transition of our codebase.

Dev.to version of this article is available here.

Migrate to Effect, why?

At Inato we were very motivated early on to adopt functional programming, so we started using fp-ts in our codebase at the beginning of 2020. If you want to know more about this, have a look at Our journey to functional programming.

Let’s now get to the heart of the matter: at the beginning of this year, we officially decided to switch to Effect! Why?

• The main maintainer of fp-ts (gcanti 👋) joined the Effect team which presumably means less active development on the fp-ts side and positions Effect as a rather obvious next step.
• Because of the learning curve associated with fp-ts and the lack of documentation. Developers who joined Inato in the last years have frequently mentioned it: learning fp-ts is not really straightforward. This is a strong point for Effect with top-notch documentation and a lot of resources for training.
• For even more reasons, visit the Effect website which compares fp-ts and Effect!

Migrating our codebase to Effect is a great goal, but doing it turned out to be more challenging and required careful planning. We also wanted to limit the time spent on this project, so we agreed on a 2.5-month deadline. With all this in mind, we came up with the following strategy.

The Migration Strategy

First of, here’s a representation of our server-side codebase: we have use cases that represent our business actions, these use cases have multiple dependencies (services, repositories, etc. — we’ll refer to them as ports), and we also have runners that will execute our use cases:

When we started the migration we had around 400 use cases and 80 ports and their adapters to migrate.

Our objective for this migration was clear: by the end of our 2.5-month window, any new use case or port will be written using Effect. To have a smooth transition that would allow us to have fp-ts and Effect code cohabitating, we came up with the following plan:

1. Ensure our ports return ReaderTaskEither to facilitate the transition to Effect [*]
2. Create Effect proxies of our ports: only one implementation in fp-ts, but the ability to use an fp-ts “real” version and an Effect proxy version of each port
3. Start (re)writing use cases in Effect
4. Create fp-ts proxies of Effect use cases
5. Start (re)writing ports in Effect
6. Create fp-ts proxies of Effect ports: at this point, we would already have fulfilled our objective of writing new use cases and ports with Effect. But we wanted to go the extra mile to have the full flow covered!
7. Be able to run both Effect and fp-ts use cases

[*] ReaderTaskEither (we will refer to it as RTE later on) was a prerequisite to facilitate the migration to Effect. Why? Conceptually, a ReaderTaskEither can be represented as follows:

ReaderTaskEither<R, E, A>
= Reader<R, TaskEither<E, A>>
= (context: R) => () => Promise<Either<E, A>>

If we look at the representation of an effect given on the official Effect website, we can see that these are very similar concepts (which is something that we will leverage during our migration):

Effect<A, E, R> ~ (context: Context<R>) => E | A

The Migration Process

Let’s deep dive into the code now! Here are the steps we are going to follow:

• The program to migrate
• Create Effect proxies of the ports
• Rewrite a use case in Effect
• Convert ports to Effect
• Use ManagedRuntime to run Effect usecases
• Bonus: simplify fp-ts ↔ effect tag mapping management

To illustrate our migration process, we will focus on an example program that is representative of how our codebase is organized.

Note: all the code and helpers that will be presented are available in 👉 this repository 👈

The program to migrate

Let say that our domain model is composed of a simple Foo class:

// domain.ts
export class Foo {
  constructor(readonly id: string) {}
  static make = (id = "random-id") => new Foo(id);
}

We define a repository port to get and store a Foo:

// FooRepository.ts
export interface FooRepository {
  getById: (id: string) => RTE<unknown, Error, Foo>;
  store: (foo: Foo) => RTE<unknown, Error, void>;
}

export interface FooRepositoryAccess {
  fooRepository: FooRepository;
}
export declare const FooRepository: {
  getById: (id: string) => RTE<FooRepositoryAccess, Error, Foo>;
  store: (foo: Foo) => RTE<FooRepositoryAccess, Error, void>;
};
export declare const makeFooRepository: () => Promise<FooRepository>;

Note:

• We follow the module pattern when defining the FooRepositoryAccess interface to enable context aggregation when composing multiple ReaderTaskEither:

declare const a: RTE<{ serviceA: ServiceA },never,void>
declare const b: RTE<{ serviceB: ServiceB },never,void>
const ab: RTE<{ serviceA: ServiceA; serviceB: ServiceB },never,void> 
    = rte.flatMap(a,() => b)

• We define a companion object FooRepository that exposes the same methods as the repository itself, except that they each require a context with FooRepositoryAccess. This makes for more concise code later on:

const theLongWay: RTE<FooRepositoryAccess, Error, Foo> = pipe(
 rte.ask<FooRepositoryAccess>(),
 rte.flatMap(({ fooRepository }) => fooRepository.getById('id'))
);

const theEasyWay: RTE<FooRepositoryAccess, Error, Foo> 
 = FooRepository.getById('id')

We also define a service port to transform a Foo:

// TransformFooService.ts
export interface TransformFooService {
  transform: (foo: Foo) => RTE<unknown, Error, Foo>;
}

export interface TransformFooServiceAccess {
  transformFooService: TransformFooService;
}

export declare const TransformFooService: {
  transform: (foo: Foo) => RTE<TransformFooServiceAccess, Error, Foo>;
};

declare const makeTransformFooService: () => Promise<TransformFooService>;

Next we can write two use cases: one to create a new Foo , and another one to transform a Foo:

// usecases.ts
export const createFooUseCase = (id:string) => 
 pipe(
   rte.of(Foo.make(id)),
   rte.tap(FooRepository.store)
 );

export const transformFooUseCase = (id: string) =>
  pipe(
    FooRepository.getById(id),
    rte.flatMap(TransformFooService.transform),
    rte.flatMap(FooRepository.store)
  );

Finally, we can write our main that will create all the port adapters and invoke our use cases:

// index.ts
const main = async () => {
  const fooRepository = await makeFooRepository();
  const transformFooService = await makeTransformFooService();
  await createFooUseCase("my-foo-id")({
    transformFooService,
    fooRepository,
  })();
  await transformFooUseCase("my-foo-id")({
    transformFooService,
    fooRepository,
  })();
};
main();

Create Effect proxies of the ports

This step consists in generating new companion objects FooRepository and TransformFooService for our ports that are exposing an Effect version of the member methods.

First we rename the companion objects, adding a Fpts suffix:

// FooRepository.ts
e̶x̶p̶o̶r̶t̶ ̶d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶:̶ ̶{̶
export declare const FooRepositoryFpts: {
  getById: (id: string) => RTE<FooRepositoryAccess, Error, Foo>;
  store: (foo: Foo) => RTE<FooRepositoryAccess, Error, void>;
};

// TransformFooService.ts
e̶x̶p̶o̶r̶t̶ ̶d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶:̶ ̶{̶
export declare const TransformFooServiceFpts: {
  transform: (foo: Foo) => RTE<TransformFooServiceAccess, Error, Foo>;
};

// usecases.ts
export const createFooUseCase = (id:string) => 
 pipe(
   rte.of(Foo.make(id)),
   r̶t̶e̶.̶t̶a̶p̶(̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶.̶s̶t̶o̶r̶e̶)̶
   rte.tap(FooRepositoryFpts.store)
 );

export const transformFooUseCase = (id: string) =>
  pipe(
    F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶.̶g̶e̶t̶B̶y̶I̶d̶(̶i̶d̶)̶,̶
    r̶t̶e̶.̶f̶l̶a̶t̶M̶a̶p̶(̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶.̶t̶r̶a̶n̶s̶f̶o̶r̶m̶)̶,̶
    r̶t̶e̶.̶f̶l̶a̶t̶M̶a̶p̶(̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶.̶s̶t̶o̶r̶e̶)̶
    FooRepositoryFpts.getById(id),
    rte.flatMap(TransformFooServiceFpts.transform),
    rte.flatMap(FooRepositoryFpts.store)
  );

Then we use the portToEffect helper function to generate the Effect companion objects from the previous companion objects:

// FooRepository.ts
export const FooRepositoryTag = Context.GenericTag<FooRepository>(
 "FooRepository"
);
export const FooRepository = portToEffect(FooRepositoryFpts, {
  fooRepository: FooRepositoryTag,
}); // { getById: (id: string) => Effect<Foo, Error, FooRepository> ... }

// TransformFooService.ts
export const TransformFooServiceTag = Context.GenericTag<TransformFooService>(
  "TransformFooService"
);
export const TransformFooService = portToEffect(TransformFooServiceFpts, {
  transformFooService: TransformFooServiceTag,
}); // { transform: (foo: Foo) => Effect<Foo, Error, TransformFooService> }

Rewrite a use case in Effect

At this point we can start using our newly generated Effect companion objects to rewrite the transformFooUseCase use case in Effect. Note that we voluntarily leave the createFooUseCase use case as is to simulate a migration that is ongoing, as opposed to a “big-bang” migration where we would convert all of our use cases to Effect in one go (much harder and riskier).

// usecases.ts
export const transformFooUseCase = (id: string) =>
  pipe(
    F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶F̶p̶t̶s̶.̶g̶e̶t̶B̶y̶I̶d̶(̶i̶d̶)̶,̶
    r̶t̶e̶.̶f̶l̶a̶t̶M̶a̶p̶(̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶F̶p̶t̶s̶.̶t̶r̶a̶n̶s̶f̶o̶r̶m̶)̶,̶
    r̶t̶e̶.̶f̶l̶a̶t̶M̶a̶p̶(̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶F̶p̶t̶s̶.̶s̶t̶o̶r̶e̶)̶
    FooRepository.getById(id),
    Effect.flatMap(TransformFooService.transform),
    Effect.flatMap(FooRepository.store)
  ); // Effect<void, Error, TransformFooService | FooRepository>

Since we don’t want to impact our main program yet, we must maintain an fp-ts version of this use case, for backward compatibility. We can generate it from the Effect version thanks to the functionToFpts helper function:

// usecases.ts
export const transformFooUseCaseFpts = functionToFpts(transformFooUseCase, {
  fooRepository: FooRepositoryTag,
  transformFooService: TransformFooServiceTag,
}); // RTE<TransformFooServiceAccess & FooRepositoryAccess, Error, void>

// index.ts
const main = async () => {
  const fooRepository = await makeFooRepository();
  const transformFooService = await makeTransformFooService();
  await createFooUseCase("my-foo-id")({
   transformFooService,
    fooRepository,
  })();
  a̶w̶a̶i̶t̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶U̶s̶e̶C̶a̶s̶e̶(̶"̶m̶y̶-̶f̶o̶o̶-̶i̶d̶"̶)̶(̶{̶
  await transformFooUseCaseFpts("my-foo-id")({
    transformFooService,
    fooRepository,
  })();
};
main();

Convert ports to Effect

Next we convert our FooRepository port to Effect directly:

// FooRepository.ts
export interface FooRepository {
  g̶e̶t̶B̶y̶I̶d̶:̶ ̶(̶i̶d̶:̶ ̶s̶t̶r̶i̶n̶g̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶u̶n̶k̶n̶o̶w̶n̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶F̶o̶o̶>̶;̶
  s̶t̶o̶r̶e̶:̶ ̶(̶f̶o̶o̶:̶ ̶F̶o̶o̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶u̶n̶k̶n̶o̶w̶n̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶v̶o̶i̶d̶>̶;̶
  getById: (id: string) => Effect.Effect<Foo, Error>;
  store: (foo: Foo) => Effect.Effect<void, Error>;
}

We can now generate the Effect companion object using Effect.serviceFunctions:

// FooRepository.ts
e̶x̶p̶o̶r̶t̶ ̶c̶o̶n̶s̶t̶ ̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶ ̶=̶ ̶p̶o̶r̶t̶T̶o̶E̶f̶f̶e̶c̶t̶(̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶F̶p̶t̶s̶,̶ ̶{̶
̶ ̶ ̶f̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶:̶ ̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶T̶a̶g̶,̶
̶}̶)̶;̶
export const FooRepository = Effect.serviceFunctions(FooRepositoryTag);

Finally, for backward compatibility, we must maintain the fp-ts companion object. We can generate it using the portToFpts helper function:

// FooRepository.ts
e̶x̶p̶o̶r̶t̶ ̶d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶F̶p̶t̶s̶:̶ ̶{̶
̶ ̶ ̶g̶e̶t̶B̶y̶I̶d̶:̶ ̶(̶i̶d̶:̶ ̶s̶t̶r̶i̶n̶g̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶A̶c̶c̶e̶s̶s̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶F̶o̶o̶>̶;̶
̶ ̶ ̶s̶t̶o̶r̶e̶:̶ ̶(̶f̶o̶o̶:̶ ̶F̶o̶o̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶A̶c̶c̶e̶s̶s̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶v̶o̶i̶d̶>̶;̶
̶}̶;̶

export const FooRepositoryFpts = portToFpts(FooRepository, {
  fooRepository: FooRepositoryTag,
}); // { getById: (id: string) => RTE<FooRepositoryAccess, Error, Foo>; ... }

We do the same for the TransformFooService port:

// TransformFooService.ts
export interface TransformFooService {
  t̶r̶a̶n̶s̶f̶o̶r̶m̶:̶ ̶(̶f̶o̶o̶:̶ ̶F̶o̶o̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶u̶n̶k̶n̶o̶w̶n̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶F̶o̶o̶>̶;̶
  transform: (foo: Foo) => Effect<Foo, Error>;
}

e̶x̶p̶o̶r̶t̶ ̶c̶o̶n̶s̶t̶ ̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶ ̶=̶ ̶p̶o̶r̶t̶T̶o̶E̶f̶f̶e̶c̶t̶(̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶F̶p̶t̶s̶,̶ ̶{̶
̶ ̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶:̶ ̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶T̶a̶g̶,̶
̶}̶)̶;̶
export const TransformFooService = Effect.serviceFunctions(
 TransformFooServiceTag
);

e̶x̶p̶o̶r̶t̶ ̶d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶F̶p̶t̶s̶:̶ ̶{̶
̶ ̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶:̶ ̶(̶f̶o̶o̶:̶ ̶F̶o̶o̶)̶ ̶=̶>̶ ̶R̶T̶E̶<̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶A̶c̶c̶e̶s̶s̶,̶ ̶E̶r̶r̶o̶r̶,̶ ̶F̶o̶o̶>̶;̶
̶}̶;̶
export const TransformFooServiceFpts = portToFpts(TransformFooService, {
  fooRepository: FooRepositoryTag,
}); // { transform: (foo: Foo) => RTE<unknown, Error, Foo>; }

Note that we have not changed our main in this step and it can still be run without a problem.

Use ManagedRuntime to run Effect usecases

In order to run the transformFooUseCase as an Effect, we must be able to provide our ports via Layers:

// FooRepository.ts
e̶x̶p̶o̶r̶t̶ ̶d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶m̶a̶k̶e̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶:̶ ̶(̶)̶ ̶=̶>̶ ̶P̶r̶o̶m̶i̶s̶e̶<̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶>̶;̶
export declare const FooRepositoryLive: Layer.Layer<FooRepository>;

// TransformFooService.ts
d̶e̶c̶l̶a̶r̶e̶ ̶c̶o̶n̶s̶t̶ ̶m̶a̶k̶e̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶:̶ ̶(̶)̶ ̶=̶>̶ ̶P̶r̶o̶m̶i̶s̶e̶<̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶>̶;̶
declare const TransformFooServiceLive: Layer.Layer<TransformFooService>;

Next we can create a ManagedRuntime and extract all the ports from the runtime context using the contextToFpts helper:

// index.ts
const main = async () => {
  c̶o̶n̶s̶t̶ ̶f̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶ ̶=̶ ̶a̶w̶a̶i̶t̶ ̶m̶a̶k̶e̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶(̶)̶;̶
  c̶o̶n̶s̶t̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶ ̶=̶ ̶a̶w̶a̶i̶t̶ ̶m̶a̶k̶e̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶(̶)̶;̶
  const runtime = ManagedRuntime.make(
    Layer.mergeAll(FooRepositoryLive, TransformFooServiceLive)
  );
  const { context } = await runtime.runtime();
  const { fooRepository, transformFooService } = contextToFpts(context, {
    fooRepository: FooRepositoryTag,
    transformFooService: TransformFooServiceTag,
  });
  await createFooUseCase("my-foo-id")({
   transformFooService,
    fooRepository,
  })();
  await transformFooUseCaseFpts("my-foo-id")({
    transformFooService,
    fooRepository,
  })();
};
main();

Finally, we can use the runtime to run the Effect transformFooUseCase:

// index.ts
const main = async () => {
  const runtime = ManagedRuntime.make(
    Layer.mergeAll(FooRepositoryLive, TransformFooServiceLive)
  );
  const { context } = await runtime.runtime();
  const { fooRepository, transformFooService } = contextToFpts(context, {
    fooRepository: FooRepositoryTag,
    transformFooService: TransformFooServiceTag,
  });
  await createFooUseCase("my-foo-id")({
   transformFooService,
    fooRepository,
  })();
  a̶w̶a̶i̶t̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶U̶s̶e̶C̶a̶s̶e̶F̶p̶t̶s̶(̶"̶m̶y̶-̶f̶o̶o̶-̶i̶d̶"̶)̶(̶{̶
   ̶ ̶t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶,̶
   ̶ ̶f̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶,̶
  }̶)̶(̶)̶;̶
  await runtime.runPromise(transformFooUseCase("my-foo-id"));
};
main();

Note that, once again, we left the createFooUseCase use case as is to show that we can be in a hybrid state where only part of the use cases have been migrated to Effect.

Bonus: simplify fp-ts ↔ effect tag mapping management

All of the helpers we have used throughout this migration require a mapping object to go from the key name of the fp-ts port Access interface (eg transformFooService of TransformFooServiceAccess) to the Tag of the corresponding Effect port. For example:

contextToFpts(context, {
  fooRepository: FooRepositoryTag,
  transformFooService: TransformFooServiceTag,
});

This mapping is essential for all the helpers to work correctly. It is not ideal to have to craft them like that all the time. To help us with that, we introduce:

const FptsConvertibleId = Symbol();
interface FptsConvertible<T extends string> { 
  [FptsConvertibleId]: T;
}

We can now embed this conversion information at the type level of our ports:

// FooRepository.ts
export interface FooRepository extends FptsConvertible<"fooRepository"> {
  getById: (id: string) => Effect.Effect<Foo, Error>;
  store: (foo: Foo) => Effect.Effect<void, Error>;
}

// TransformFooService.ts
export interface TransformFooService 
  extends FptsConvertible<"transformFooService"> {
  transform: (foo: Foo) => Effect<Foo, Error>;
}

The first thing we can do with this is to simplify the definition of Access interfaces using a type helper FptsAccess:

// FooRepository.ts
export interface FooRepositoryAccess extends FptsAccess<FooRepository> {}

// TransformFooService.ts
export interface TransformFooServiceAccess 
  extends FptsAccess<TransformFooService> {}

And we can also define smaller atomic mapping objects using a new helper getFptsMapping:

// FooRepository.ts
const FooRepositoryFptsMapping = getFptsMapping(
  FooRepositoryTag,
  "fooRepository"
); // { fooRepository: FooRepositoryTag }

// TransformFooService.ts
const TransformFooServiceFptsMapping = getFptsMapping(
  TransformFooServiceTag,
  "transformFooService"
); // { transformFooService: TransformFooServiceTag }

Note: It looks like we are once again typing the key "fooRepository" or "transformFooService" but in fact, the function getFptsMapping is type-safe so that given FooRepositoryTag as first argument, only the string "fooRepository" is valid as second argument. So your code editor will autocomplete it for you. Moreover, the compiler will break if you change the definition in the FptsConvertible so it is not really an additional burden.

We can now combine these two mapping objects when calling contextToFpts or any other helper:

contextToFpts(context, {
  f̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶:̶ ̶F̶o̶o̶R̶e̶p̶o̶s̶i̶t̶o̶r̶y̶T̶a̶g̶,̶
  t̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶:̶ ̶T̶r̶a̶n̶s̶f̶o̶r̶m̶F̶o̶o̶S̶e̶r̶v̶i̶c̶e̶T̶a̶g̶,̶
  ...FooRepositoryFptsMapping,
  ...TransformFooServiceFptsMapping,
});

Conclusion

Our objective of being able to write any new use case or port using Effect was accomplished in 2 months (working around 10% of our time on it)!

Teamwork was definitely a big part of this success: first, we have to mention Stephane Ledorze as he migrated all our repositories single-handedly and gave us great advice on how to define our migration strategy. We handled the rest with the whole team during dedicated “tech sessions” that we do every Wednesday afternoon at Inato: during those sessions, we stop delivering features to be able to focus on purely tech subjects, which was a great occasion to migrate the many ports we had to handle and onboard the team on Effect.

As we’re writing this article, we have around 150 full Effect use cases. The rest of the existing use cases will be migrated on the go whenever we need to update them!

We’re already seeing great improvements: for example, implementing rate limiting with just a few lines of code with Effect, whereas we needed a big amount of code to do it with fp-ts. We’re eager to leverage even more the Effect ecosystem now that we have officially migrated to it!

We hope this article motivated you to take the leap from fp-ts to Effect, don’t hesitate to comment if you have any questions or comments!

This article was written by Jeremie Dayan and Laure Retru-Chavastel

How we migrated our codebase from fp-ts to Effect was originally published in inato on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recently, I was trying to improve the performance of a page in our React app, and I quickly suspected unnecessary re-renders to be the main issue there. After some investigations, I stumbled upon the React DevTools profiler plugin:

This plugin uses React’s experimental Profiler API to collect timing information about each component that’s rendered in order to identify performance bottlenecks in React applications. (source)

More particularly, I discovered a kind of hidden option of this plugin: being able to know why a component rendered. I found this feature so useful that I decided to create this article to share how to use it!

Prerequisites:

• Use React ≥ 16.5 in DEV mode
• Have React DevTools as an extension for your browser (Chrome or Firefox)

Improve a basic app with React Profiler

First of all, here is how to enable this option in your DevTools:

Now, let’s take the example of this simple application where you can see your goals of the day and set them as done. Let’s use React profiler to generate a profile and check what happens in our application after we click on the “Set as done” button.

“The color of a bar indicates how long the component (and its children) took to render in the selected commit. Yellow components took more time, blue components took less time, and gray components did not render at all during this commit.” (source)

We can see that all the “Goal” sub-components were re-rendered. Ideally, as we only updated the Goal “Read 30 min” we would expect that only this Goal component would be re-rendered, not the other Goal components.

How can we fix this? Now is the time to take advantage of the option we selected earlier to know why a component rendered! Let’s hover the first Goal component in the profiling data:

So the cause is that the property setAsDone has changed. Let’s fix this quickly:

Thanks to that change, the method setAsDone won’t change at each render of the Goals component. Unfortunately, that won’t be enough here to prevent the re-render of the other Goal components. Here is what the profiling data tell us now:

So Goal components re-rendered because Goals component re-rendered. And Goals component re-rendered because “Hook 1 changed”, which is not very explicit right?

To know what this “Hook 1” is, let’s switch to the “Components” tab and the DevTools will smartly show you the right component:

We can now see that "Hook 1 changed” actually means "the state goalsReached changed” which is indeed the case as we added the goal “Read 30 min” in this state.

In this specific case, we could use React.memo to fix our issue (it might not be the best solution in every case of course, but let’s keep things simple for this example).

“If your component renders the same result given the same props, you can wrap it in a call to React.memo for a performance boost in some cases by memoizing the result. This means that React will skip rendering the component, and reuse the last rendered result.” (source)

Let’s profile our app one more time:

We can now see that only the right Goal component was re-rendered, while the others were not 🥳

That should help you in some of your investigations to improve the performance of your React app! Feel free to ask your questions or share your experience and tips with React Profiler in the comments!

Improve your React app performance using React Profiler was originally published in inato on Medium, where people are continuing the conversation by highlighting and responding to this story.

You may be thinking “but Google SQL database instances already provide automated and on-demand backups, why would you need to use terraform to deal with it?”. Well, the issue is that if your database instance is deleted, for instance after a manual mistake or a Google Cloud SQL outage, so are your backups!

This is why we decided to save those backups in a google storage bucket, and you will soon discover that this process is really easy to do with Terraform.

Prerequisites :

• Have a project in Google Cloud Platform (GCP)
• Have a Cloud SQL database defined in GCP
• Install Terraform CLI on your computer (see documentation here, or simply run brew install terraform if you are a mac user)
• Install gcloud CLI

Here are the steps we are going to follow in this article :

1. Setup the terraform config for your Google Cloud project
2. Import your database into terraform
3. Create a bucket to store your backups
4. Create a cloud function to store a compressed dump of your database in this bucket
5. Create a cloud scheduler to trigger a daily backup of your database

1. Setup the terraform config for your Google Cloud project

• Run gcloud init and gcloud auth application-default login to allow terraform to benefit from your access rights when you run terraform commands locally
• From your console, create a new bucket (its name has to be unique among all the buckets that exist in GCP so we suffix it with the project id for this purpose) :

PROJECT_ID="your-project-id"
BUCKET_NAME="gs://terraform-state-$PROJECT_ID"
gsutil mb -p $PROJECT_ID $BUCKET_NAME
gsutil versioning set on $BUCKET_NAME

• Go in GCP and check that a bucket terraform-state-your-project-id has been created
• In your project repository, create a folder terraform with the following architecture :

terraform/
  production/ // I named it after the name of a specific environment
              // but you can name it as you like

• Create a file production/main.tf :

terraform {
 backend "gcs" {
  bucket = "terraform-state-your-project-id"
  prefix = "terraform/state"
 }
}

provider "google" {
 region = "europe-west1"
 zone = "europe-west1-d"
}

You may be wondering “but what is this terraform state”? This state is a sort of screenshot of your resources configuration that you are about to link with Terraform. It will use this state to make changes to your infrastructure.

An interesting advantage of storing this state in a bucket is that everyone will share the same configuration. Moreover, if you are editing the state, Terraform will lock it so that no one else would be able to edit it at the same time.

• Navigate in your console into your production terraform folder and run terraform init
• Go in GCP and check that the bucket has a file terraform.tfstate which contains the terraform state

2. Import your database into Terraform

Now that Terraform is ready, we can add your database in its state.

• Create a production/variables.tf file :

variable "project_id" {}

• Create a production/terraform.tfvars file :

project_id = "your-project-id"

• In production/main.tf, add the database resource config :

resource "google_sql_database_instance" "your-database" {
  project          = var.project_id
  name             = "your-database"
  database_version = "POSTGRES_9_6"
  region           = "europe-west1"

  settings {
    tier = "db-custom-16-106496"
  }
}

Here, I specified all the required fields of google_sql_database_instance (see documentation) but also the project field, as it is a good practice to be sure that your resource is going to be linked to the right project.

If you don’t know if these values are adapted to your database, don’t worry, Terraform won’t change the properties of your database without your consent!

• In your terminal, navigate to the folder containing your files and import your database config into Terraform:

terraform import google_sql_database_instance.your-database projects/your-project-id/instances/your-sql-database-instance-name

This command will tell Terraform “the resource called google_sql_database_instance.your-database that I have defined in main.tf is linked to the real resource that you can find here: projects/your-project-id/instances/your-sql-database-instance-name”.

• Let’s run terraform plan to check if there are any differences between the configuration we defined and your real database configuration
• If there are, edit your configuration in production/main.tf. For instance, if the region of your database is in fact “us-east1” instead of “europe-west1”, change the value of the region field in your configuration.
• Repeat the two previous steps until the command terraform plan displays: “No changes. Infrastructure is up-to-date.”

3. Create a GCP bucket to store your backups

• In the terraform folder, create a new folder modules/db-backups , so your architecture should look like this :

terraform/
  modules/
    db-backups/
  production/
    main.tf
    variables.tf
    terraform.tfvars

• In this db-backups folder, create a file main.tf :

resource "google_storage_bucket" "db-backups" {
  project       = var.project_id
  name          = "db-backups-${var.project_id}"
  location      = "EU"
  storage_class = "MULTI_REGIONAL"
}

• Again, we will need a file db-backups/variables.tf :

variable "project_id" {}

• And a file db-backups/terraform.tfvars :

project_id = "your-project-id"

• Our module is now ready to be used! Go back in the production/main.tf and add your module :

module "db_backups" {
  source                = "../modules/db-backups"
  project_id            = var.project_id
}

• In your terminal, navigate to the production folder and run terraform apply (it first does the same as terraform plan and then ask you if you want to apply your changes). You should see that Terraform wants to add a new bucket. Check that nothing else is going to be done, and if so, write “yes” and apply your changes.
• Go in GCP: you should see a new bucket db-backups in your project, it is empty for now but we are soon going to fill it!

4. Create a cloud function to store a compressed dump of your database in this bucket

4.1 Create a zip file containing the code of your cloud function

• First of all, create a new folder inside modules/db-backups/ named backupCloudFunction
• In there, create a new file index.js :

• And also, a package.json file:

{
    "name": "export-database",
    "version": "0.0.1",
    "dependencies": {
      "googleapis": "^39.2.0",
      "google-auth-library": "3.1.2"
    }
}

• Then, let’s create a zip with Terraform from those two files that we will call backup.zip:

data "archive_file" "backupZipFile" {
  type        = "zip"
  output_path = "${path.module}/backupCloudFunction/backup.zip"

source {
    content  = "${file("${path.module}/backupCloudFunction/index.js")}"
    filename = "index.js"
  }

source {
    content  = "${file("${path.module}/backupCloudFunction/package.json")}"
    filename = "package.json"
  }
}

• Run terraform apply and check that the backup.zip file has been created
• We are now going to create a resource to store the code of our cloud function in the db-backups bucket. In the main.tf of db-backups, add a new resource :

resource "google_storage_bucket_object" "archive" {
  name   = "cloudFunctions/backup-${lower(replace(base64encode(data.archive_file.backupZipFile.output_md5), "=", ""))}.zip"
  bucket = google_storage_bucket.db-backups.name
  # Source path is relative to where the module is used : to improve
  source     = data.archive_file.backupZipFile.output_path
  depends_on = [data.archive_file.backupZipFile]
}

You may be surprised by the name attribute of the archive but this is necessary: the cloud function to which we will give this archive will not detect changes made in this file unless its name is changed. This is why we chose a name for this file that will change only when its content has been modified.

• Run terraform apply to create this archive. In the Storage menu of GCP, in the db-backups bucket, you should now see a folder cloudFunctions containing a zip file

4.2 Authorize your database to write in the db-backups bucket

• Add a new resource in db-backups/main.tf :

resource "google_storage_bucket_iam_member" "editor" {
  bucket = google_storage_bucket.db-backups.name
  role   = "roles/storage.objectCreator"
  member = "serviceAccount:${var.service_account_email}"
}

• Add a new variable in db-backups/variables.tf :

variable "service_account_email" {}

• Then, in production/main.tf :

module "db_backups" {
  source                = "../modules/db-backups"
  project_id            = var.project_id
  service_account_email = google_sql_database_instance.your-database.service_account_email_address
}

• Run terraform apply

Your database is now allowed to create files in your db-backups bucket

4.3 Create the cloud function resource

• Before doing anything, you need to enable the Cloud SQL Admin API or your cloud function won’t work
• In db-backups/main.tf, add a new resource :

resource "google_cloudfunctions_function" "backupFunction" {
  name        = "backup"
  description = "Performs a backup of the specified database in the db-backups bucket"
  runtime     = "nodejs10"
  project     = var.project_id

  available_memory_mb   = 128
  source_archive_bucket = google_storage_bucket.db-backups.name
  source_archive_object = google_storage_bucket_object.archive.name
  trigger_http          = true
  entry_point           = "backup"

environment_variables = {
    PROJECT_ID        = var.project_id,
    DB_INSTANCE_NAME  = var.db_instance_name,
    BUCKET_NAME       = google_storage_bucket.db-backups.name
    DB_NAME_TO_EXPORT = var.db_name_to_export
  }
}

• As you can see, some more variables are needed : db_instance_name and db_name_to_export. Let’s add them in db-backups/variables.tf :

variable "app_db_instance_name" {}
variable "db_name_to_export" {}

• Then, in production/main.tf :

module "db_backups" {
  source                = "../modules/db-backups"
  project_id            = var.project_id
  service_account_email = google_sql_database_instance.your-database.service_account_email_address
  app_db_instance_name  = google_sql_database_instance.your-database.name
  db_name_to_export     = "name-of-the-database-to-export"
}

• After running terraform apply, go in the Cloud Functions menu of GCP and you should see your cloud function named “backup” there
• You can already test it (in the “Actions” column, select “Test function”, and then click on “Test the function” without adding anything in the “Triggering event”). Depending on the size of your database, the backup could take long. But ultimately, you should see a backup appears in your db-backups bucket!

4.4 Deal with the “Error waiting for Creating CloudFunctions Function”

When trying to create the cloud function with terraform apply, you may encounter this error :

[...] googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the cloudfunctions.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. [...]

Terraform does not allow end-users to create cloud functions, only service accounts are authorized to do so. So here is a trick to be able to create your cloud function from your terminal anyway :

• In db-backups/main.tf, create a service account that will have the right permissions (💡 if you already have one with the right permissions, you can skip this creation step and use it for the following steps)

resource "google_service_account" "cloudFunction" {
  project      = var.project_id
  account_id   = "cloud-function"
  display_name = "cloud function"
  description  = "Used to create cloud functions"
}

resource "google_project_iam_member" "cloudFunctionEditor" {
  project = var.project_id
  role    = "roles/editor"
  member  = "serviceAccount:${google_service_account.cloudFunction.email}"
}
resource "google_project_iam_member" "cloudFunctionStorageAdmin" {
  project = var.project_id
  role    = "roles/storage.admin"
  member  = "serviceAccount:${google_service_account.cloudFunction.email}"
}

• In the Service Accounts menu of GCP, create a key for this service account
• If you have jq, you can use this command to get a well-formatted output of the key (if not, remove all spaces and line breaks except for those in BEGIN PRIVATE KEY and END PRIVATE KEY) :

cat <your_file_path>.json | jq --compact-output

• Copy the output and then run the following command (don’t forget the ' around the google credentials) :

GOOGLE_CREDENTIALS='<paste_key_output_here>' terraform apply

5. Create a cloud scheduler to trigger a daily backup of your database

5.1 Configure an app engine

• We now have a working cloud function to trigger a backup of our database. Let’s say we want to make a daily backup at 4 a.m in the french timezone.
• First of all, enable the Cloud Scheduler API in your project
• Then, to be able to use a cloud scheduler, your project must contain an App Engine. If you don’t, you need to create one.
• In production/main.tf, add a new resource :

resource "google_app_engine_application" "app_engine" {
  project     = var.project_id
  location_id = "europe-west"
}

⚠️ once an App Engine is created, you can not delete it or edit its location_id. The only way to do so would be to delete the entire project so define it carefully!

• Run terraform plan : if you did not already have an app engine, Terraform will ask to create it. If you are sure about your configuration, do it. If you already did have one, Terraform will throw you an error saying that there is already an existing app engine.
• (Do this step only if you already have an app engine) :

terraform import google_app_engine_application.app_engine your-project-id

5.2 Create the cloud scheduler

• Add a new resource in db-backups/main.tf :

resource "google_cloud_scheduler_job" "backupJob" {
  project     = var.project_id
  region      = var.scheduler_region
  name        = "backup-job"
  description = "Create a backup of the database"
  schedule    = "0 4 * * *"
  time_zone   = "Europe/Paris"

http_target {
    http_method = "POST"
    uri         = google_cloudfunctions_function.backupFunction.https_trigger_url
  }
}

• A new variable scheduler_region is needed, because this value should be compatible with the location you set for your app engine. In db-backups/variables.tf, add :

variable "scheduler_region" {}

• Then, in production/main.tf :

module "db_backups" {
  source                = "../modules/db-backups"
  project_id            = var.project_id
  service_account_email = google_sql_database_instance.your-database.service_account_email_address
  app_db_instance_name  = google_sql_database_instance.your-database.name
  scheduler_region      = "europe-west1"
}

⚠️ in this example, the app engine has a location_id = "europe-west" that’s why I defined scheduler_region = "europe-west1". If your app engine has another location_id, find the compatible region to set your scheduler, or Terraform will throw an error

• Run terraform apply
• Go in the Cloud Scheduler menu of GCP: you should see your scheduler ready to work!
• You can test it right away by clicking the “Test now” button: a backup should then appear in your db-backups bucket!

Congratulations! Daily backups are now scheduled for your database.

Feel free to share your comments or questions below!

Drug discovery is a challenging, intellectually complex, and rewarding endeavor: we help develop effective and safe cures to diseases affecting millions of people. If you’re looking to have a massive impact, join us! inato.com/careers

How to use Terraform to schedule backups for your Google Cloud SQL database was originally published in inato on Medium, where people are continuing the conversation by highlighting and responding to this story.