Asking for required annotations

The next release of Flow, version 0.85.0, includes an important bug fix that will likely add many new “missing annotation” errors to your existing code. This fix does not represent a new annotation requirement from Flow. Instead, it is fixing a hole in Flow’s analysis that caused us to miss truly required and missing annotations.

This change not only fixes a hole in the type checker where code becomes untyped, but it also unblocks multiple projects which the Flow team is actively working on to improve type checker precision and performance.

Since we expect this change to be fairly disruptive, I’ve written up a bit of background on why we ask for annotations in the first place, what this bug fix is all about, and the various ways you can provide the necessary annotations.

Why are annotations required?

Flow has very few limitations on its type inference, but there is one big one: input positions reachable from exports must be annotated. An inferred type in such a position is insufficient, as it would not account for a value that might “flow in” from a dependent module. Indeed, these types are often never constrained by the defining module.

In order to infer a type, Flow needs to see all the values that might reach a given position. In order to infer types in input positions which are exported from a module, Flow would need to analyze files together with all of their transitive dependents. This is called “global” type inference, and while theoretically possible, this would be impossible to scale to the size of our codebase.

Instead of global type inference, Flow makes a humble request: annotate the input positions of your modules and we can infer the rest. This makes it possible to infer modules in parallel in dependency order. (Fun fact: Flow did not always work this way. Before this commit which landed in version 0.16.0, Flow did perform global type inference and that change was a huge perf improvement.)

Note that Flow already requires these annotations and most cases are already an error. Let’s look at some examples of where Flow gets things right before the change.

FUNCTION ARGUMENTS

If a module exports a function, its parameters are inputs. This is the simplest case. In order to infer a type for x, Flow would need to analyze every call in the dependent modules.

export function f(x) {}

Running Flow on this file results in the following error:

1: export function f(x) {}
                     ^ Missing type annotation for `x`.

[Try Flow]

WRITEABLE FIELDS

If a module exports a class instance, its writeable fields are inputs.

class C { p; }
export default new C;

Running Flow on this file results in the following error:

1: class C { p; }
             ^ Missing type annotation for property `p`.

[Try Flow]

We can fix this error by either annotating the field (Try Flow) or by marking the field as covariant which makes this no longer an input position (Try Flow).

IMPLICIT INSTANTIATIONS

In type systems jargon: A generic function or class is implicitly instantiated when called or constructed without explicit type arguments. The concrete type corresponding to this instantiation is inferred based on uses within the program.

If an implicitly instantiated type parameter appears in an input position reachable from a module’s exports, then Flow will also ask for an annotation.

Don’t worry if the above is confusing. It’s describing something that is fairly intuitive. In the example below, think about the type parameter T and how it’s type might be inferred.

function ref<T>(): { value: T|null } {
  return { value: null };
}
module.exports = ref();

Running Flow on this file results in the following error:

4: module.exports = ref();
                    ^ Missing type annotation for `T`. `T` is a type parameter declared in function [1] and was implicitly instantiated at call of `ref` [2].
References:
1: function ref<T>(): { value: T|null } {
   ^ [1]
4: module.exports = ref();
                    ^ [2]

[Try Flow]

We can fix this error by annotating the return type of the function call (Try Flow) or by providing an explicit type argument to the function call (Try Flow).

What was broken?

Since Flow is already designed to ask for annotations in input positions, why does this change add so many new errors? In order to find these errors, Flow walks the structure of types starting with the exports object in a polarity sensitive way. If it sees a type variable (i.e., an inferred type) in an input position, we add an error.

The issue is that we performed this walk too early during type checking. Specifically, before merging in type information from dependencies. It is possible for the exports of a dependent module to include an input position that arises based on the type of its dependency (see example below), but Flow was unable to see these cases.

First let’s look at a small example, then look at some of the examples that this version complains about. Say we have a simple generic utility for containing values:

// ref.js
class Ref<T> {
  x: T | null = null;
  get(): T {
    if (this.x == null) {
      throw new Error("unset");
    }
    return this.x;
  }
  set(x: T) { this.x = x }
}
module.exports = Ref;

Now we might return an instance of this class from a dependent module:

// foo.js
const Ref = require("./ref");
module.exports = new Ref();

And finally, we import our foo ref and use it to store and retrieve values:

// bar.js
const foo = require("./foo");
foo.set(0);
var x: string = foo.get(); // uh oh!

Before this fix, Flow would miss the type error in the assignment in bar.js. After this fix, Flow will ask for an annotation in foo.js:

Missing type annotation for T. T is a type parameter declared in Ref [1] and was implicitly instantiated at new Ref [2].
     foo.js
     1│ /* @flow */
     2│ const Ref = require("./ref");
 [2] 3│ module.exports = new Ref();
     4│
     ref.js
 [1] 2│ class Ref<T> {

If we provide the annotation, by either annotating the return type or using an explicit type argument to the constructor, then Flow will error on the assignment in bar.js.

In this example, when we walk the exports type of foo.js looking for missing annotations, we start with the type corresponding to new Ref(). Before this change, when Flow did the walk before merging in dependencies, we didn’t yet know what Ref was, so we couldn’t see the structure of the resulting instance.

Because the fix moves this analysis after the point where dependencies are merged in, we can now see that Ref is a generic type and its T type parameter appears in an input position in the set() method of the resulting instance.

Without an annotation, Flow relies on type inference, but observe that in foo.js the type of T is never constrained by any use, so no reasonable type can be inferred. This is why the foo ref has untyped behavior in bar.js.

Real world examples

So, what kind of missing annotation errors are you likely to see in your code after the deploy?

IMMUTABLE.MAP

The most common case, which is very similar to the Ref example above, is commonly used data structures like Immutable.Map. Instances of these types often reach the exports of a module, either directly or indirectly.

For example, you might have a module that exports data including a freshly constructed immutable data structure, like so:

module.exports = {
  defaults: {
    number_of_stages: 0,
    questions: Immutable.OrderedMap(),
    question_templates: {
      key: '',
      templates: {},
    },
  },
};

The simplest solution here would be to use an explicit type argument to the OrderedMap() constructor, but any annotation between the exports and the map itself would also work.

Note that builtin generic types like Map, Set, and Promise are not affected. Builtin types are merged early in the analysis, so Flow already complains about missing annotations when their instances reach exports.

REACTREDUX.CONNECT

The connect function has between 3 and 5 type parameters, depending on which overloaded signature is selected. I spent a few days understanding the ins and outs of this little function, and concluded that many of these type parameters appear in input positions and are becoming untyped when used in dependent modules.

Let’s look at the simplified, non-overloaded definition:

type Connector<S, D, OP, SP, DP> = (
  WrappedComponent: React$ComponentType<$Spread<OP, SP, DP>>,
): ConnectedComponent<S, D, OP>;
declare export function connect<S, D, OP, SP, DP>(
  mapStateToProps: (state: S, ownProps: OP) => SP,
  mapDispatchToProps: (dispatch: D, ownProps: OP) => DP,
): Connector<S, D, OP, SP, DP>;

Now let’s look at an example usage:

type WrappedProps = {
  foo: string,
  bar: () => void,
  baz: string,
};
function WrappedComponent(props: WrappedProps) {
  /* ... */
};
module.exports = connect(
  (state, ownProps) => ({ foo: state.foo }),
  (dispatch, ownProps) => ({ bar() { dispatch(anAction) } }),
)(WrappedComponent);

There are a number of issues here that require annotations.

The “state” (S) and “dispatch” (D) types are never constrained, because that only happens when the connected component is actually used — which means that these types effectively appear in input positions in the returned type. We require explicit type annotations for these two type parameters.

The “own props” (OP) type is the props type of the connected component, which is combined with “state props” (SP) and “dispatch props” (DP) to produce the props of the wrapped component. It’s alluring to think we can infer this type from the wrapped props, state props, and dispatch props, but observe that SP and DP themselves can depend on OP (via the ownProps argument to mapStateToProps and mapDispatchToProps) which creates an inference loop. We also require explicit type annotations for the OP type parameter.

On the bright side, the “state props” (SP) and “dispatch props” (DP) types do not appear in the connected component. These types are internal and fully determined by the mapStateToProps and mapDispatchToProps functions which are defined locally. These type parameters can be inferred and do not need to be explicitly provided.

My code has missing annotation errors. What can I do?

Thank you for asking. We added support for explicit type arguments to function and constructor calls to make this transition easier. Most of the time, the simplest way to fix a missing annotation error will be to use an explicit type argument.

In other cases, annotating the return value will be simpler. In fact, it’s enough to annotate any position “between” the exports and the reported missing annotation. Remember that Flow walks the structure of types starting from exports, but once we see an annotation we can stop.

As a last resort, you can always use tool at the root of the Flow repo to suppress all errors in your project. Run ./tool add-comments --help for more information.

Closing remarks

I want to stress that we are not making this change lightly.

In fact, we have been working to land this fix for months. During that time, we have investigated a huge number of errors and landed fixes to both product code and Flow itself in preparation for deploying this change internally.

We also worked on language features, like the explicit type arguments feature, specifically with this change in mind to make it easier to deal with the annotation requirements. We are also in the process of adding a convenient syntax to supply some type arguments explicitly while leaving others inferred.

I am encouraged that this is the right move to make and the right time to make it for a few reasons:

• While these are new type errors, they do not necessarily reflect the loss of coverage. Before this change, Flow would still often lose type coverage, except there was no warning.
• I am working on a long-term project to improve object types, which is currently blocked by this issue. Landing this fix makes makes it possible to continue that project.
• The Flow team is investigating the possibility of leveraging even more annotation requirements to greatly improve type checking performance. Their investigation is confounded by this issue and any implementation would require this fix to be in place.

I am confident that this change will ultimately help Flow feel more understandable and predictable. We have a number of projects in the works which will continue to make changes toward this goal, so stay tuned for updates.