Mapping values in Ballerina: maps and records

Photo by Scott Graham on Unsplash

This article is based on Ballerina Swan Lake Update 8.

This article is meant to be an overview of the different concepts and features related to mapping values and types in Ballerina and also the different features of the mapping constructor expression used to create mapping values (and therefore, this may be a bit of a lengthy one!).

A mapping value is a container of unordered members in which each member is uniquely identified by a key, which is a string.

If instead, you require a container with non-string keys, you could consider Ballerina tables.

A key-value pair together is known as a field of a mapping value. There are two types of mappings: maps and records.

A mapping value can be created using a mapping constructor expression.

{one: 1, two: 2, three: 3}

As an example, you can create a map of three integers as follows.

map<int> digits = {
   one: 1,
   two: 2,
   three: 3
};

Maps​

As you may have noticed in the previous example, the map<T> syntax is used to create a map of T. All of the values in a mapping of map type map<T> have to belong to the type T.

import ballerina/io;

public function main() {
   map<int> digits = {
       one: 1,
       two: 2,
       three: 3
   };
   io:println(digits); // {"one":1,"two":2,"three":3}

   int? v = digits["two"];
   io:println(v); // 2

   string key = "six";
   io:println(digits[key] is ()); // true

   digits["four"] = 4;
   io:println(digits); // {"one":1,"two":2,"three":3,"four":4}
}

Member access m[k] can be used to access and update fields of a map. k can be any expression that is of type string. For example, it can be a string literal, variable reference, function call, etc.

Since a map m of type map<T> may or may not contain a field with key k, the static type of member access m[k] is the union of T and nil - T? - where nil indicates the absence of the field.

Member access can also be used to update a map. The static type required of the right hand side of the assignment would then be the type of the values expected by the map type. For example, int in the example above.

Alternatively, we can use the get function defined in the lang.map lang library to access fields of a map. Unlike with member access, if a field with the specified key is not present in the map, the get function will panic. Therefore, unlike with member access, the type will just be the type of the members in the map (i.e., won’t include nil).

import ballerina/io;

public function main() {
   map<int> digits = {
       one: 1,
       two: 2,
       three: 3
   };

   int v = digits.get("one");
   io:println(v); // 1

   int w = digits.get("seven"); // panics
}

The map lang library contains numerous other functions that can be used with both maps and records.

import ballerina/io;

public function main() {
   map<int> digits = {
       one: 1,
       two: 2,
       three: 3
   };
   int length = digits.length();
   io:println(length); // 3

   boolean hasKey = digits.hasKey("four");
   io:println(hasKey); // false
   io:println(digits.hasKey("one")); // true

   map<int> filtered = digits.filter(digit => digit >= 2);
   io:println(filtered); // {"two":2,"three":3}
}

A map type map<S> is a subtype of map<T> if S is a subtype of T.

import ballerina/io;

public function main() {
   map<int> m = {one: 1, two: 2};
   map<int|string> n = m;

   int|string? v = n["one"];
   io:println(v); // 1

   n["three"] = 3;
   io:println(m); // {"one":1,"two":2,"three":3}

   // panics because the inherent type of the map value
   // only allows int values
   n["four"] = "four";
}

Since int is a subtype of int|string, map<int> is a subtype of map<int|string>, which allows the assignment of m to n, a variable of type map<int|string>. A map read will never fail due to a type incompatibility, since the values in the actual map will always belong to the member type of the map type it is used as (e.g., the actual mapping value here can only have int values and they belong to int|string). But, a write may fail since the inherent map type of the map value may not allow the specific value as a field depending on the type. Note the panic when trying to add a string value to n since the underlying map only allows ints.

Records​

While maps are mappings consisting of fields with values all belonging to a particular type, records allow defining mappings in which different fields can have values of different types. A record also allows specifying additional attributes of mappings such as required vs optional fields, mapping values with exactly the fields specified in the type descriptor vs mapping values that may have fields other than those explicitly specified in the type descriptor, etc.

type Employee record {|
   int id;
   string name;
|};

Employee john = {id: 1123, name: "John"};

Here, Employee is defined using a record type descriptor that specifies exactly two fields named id and name, where id is required to be an integer and name is required to be a string. Note how in contrast with a map of type map<int|string>, we can specify the exact type for each field, to ensure that id is always an int and name is always a string.

Both the id and the name fields are required here, meaning when a value of type Employee is created, values have to be specified for each of these fields. Not specifying a value for such a required field will result in a compilation error.

// error: missing non-defaultable required record field 'name'
Employee emp = {id: 1211};

Fields with default values​

You could also specify a default value for a field in the record type descriptor.

import ballerina/io;

type Employee record {|
   int id;
   string name;
   boolean manager = false;
|};

public function main() {
   Employee e1 = {id: 1211, name: "John"};
   Employee e2 = {id: 1212, name: "Joy", manager: true};
   io:println(e1); // {"id":1211,"name":"John","manager":false}
   io:println(e2); // {"id":1212,"name":"Joy","manager":true}
}

Unlike with the id and name fields, specifying a value for manager (which has a default value) is not mandatory. If a value is not specified for the manager field in the mapping constructor, the default value false is used. This also means that an Employee value will always have the three fields id, name, and manager.

Optional fields​

A record type descriptor can also specify a field as an optional field. This is done by adding ? at the end of the field, before the semi-colon.

The department field in the following example is an optional field. Given that it is optional, here again, a value may or may not be specified in the mapping constructor for the department field. But unlike with fields with a default value, where the field is always present in the constructed record value, with optional fields, if a value was not specified for the field the constructed record will not have a field with the specific key.

import ballerina/io;

type Employee record {|
   int id;
   string name;
   boolean manager = false;
   string department?;
|};

public function main() {
   Employee e1 = {id: 1211, name: "John", department: "legal"};
   Employee e2 = {id: 1212, name: "Joy", manager: true};
   io:println(e1); // {"id":1211,"name":"John","manager":false,"department":"legal"}
   io:println(e2); // {"id":1212,"name":"Joy","manager":true}
}

Note that this does not mean that e2 will never have a department field. If the value is mutable, it is still possible to set a value for the department field even after creation, as we will see later on.

Closed vs open records​

All of the records we saw so far are closed records, meaning they allow only the fields explicitly mentioned in the record. This is done by using the exclusive record type descriptor syntax (record {| … |}) and not specifying a rest descriptor. Alternatively, we can define open records, where in addition to the explicitly defined fields, we can have other fields of any other name. This can be done in two ways.

The first way is by explicitly specifying a record rest descriptor in an exclusive record type descriptor. The type used will specify the type for all the additional fields.

import ballerina/io;

type Employee record {|
    int id;
    string name;
    boolean manager = false;
    string department?;
    (string|int)...;
|};

public function main() {
    Employee john = {id: 1211, name: "John", "year": 2, "eCode": "E1211"};
    io:println(john);
}

The Employee record now has a rest descriptor of type string|int - (string|int)...;. Therefore, in addition to the four explicitly-specified fields, the Employee record now allows other fields of which the values have to be string or integer values. The year and eCode fields in the mapping constructor are now allowed due to the rest descriptor. Note how unlike with required and optional fields that use identifiers as keys, we have used string literals with the rest fields. This is enforced by the compiler in order to avoid errors due to typos with open records and optional fields or fields with default values.

An open record can also be defined using the inclusive record type descriptor syntax (record { }). Here, the rest descriptor is implicitly anydata.

type Employee record {
   int id;
   string name;
   boolean manager = false;
   string department?;
};

// same as
type Employee record {|
   int id;
   string name;
   boolean manager = false;
   string department?;
   anydata...;
|};

Accessing and updating fields​

Fields of a record can be accessed using different kinds of expressions.

import ballerina/io;

type Employee record {|
    int id;
    string name;
    boolean manager = false;
    string department?;
    decimal? salary?;
    (int|string)...;
|};

public function main() {
    Employee e = {
        id: 1124,
        name: "John",
        salary: 1200,
        "eCode": "E1124"
    };

    // Field access - type is int since the field is guaranteed to be
    // present and be an integer.
    int id = e.id;
    io:println(id); // 1124

    // Field access - type contains nil to allow for the absence of the field.
    string? department = e.department;
    io:println(department is string); // false

    // Optional field access - value will be nil if either the field is
    // not present or the value is nil.
    decimal? salary = e?.salary;
    io:println(salary); // 1200

    // Member access - type contains nil to allow for the absence of the field.
    int|string? eCode = e["eCode"];
    io:println(eCode); // E1124

    string key = "eCode";
    int|string|boolean|decimal? value = e[key];
    io:println(value); // E1124
}

Field access (a.b) can be used to access required fields (with or without a default value) and optional fields of a type that does not contain nil. Optional field access (a?.b) can be used to access required or optional fields, including optional fields that are of a type that contain nil. Member access (a[str]) can be used to access required, optional, or rest fields. The static types of accesses contain nil when accessing fields that may not be present in the record, to indicate the absence of the value. Note how in order to access a field with a variable key we can use member access.

Records can be updated similar to maps using member access expressions. Alternatively, it is also possible to use field access to update required and/or optional fields of a record.

import ballerina/io;

type Employee record {|
   int id;
   string name;
   boolean manager = false;
   string department?;
   decimal? salary?;
   (int|string)...;
|};

public function main() {
   Employee e = {
       id: 1124,
       name: "John",
       salary: 1200,
       "year": "2"
   };

   e.id = 112400;
   e.salary = 1400;
   e["manager"] = true;
   io:println(e); // {"id":112400,"name":"John","manager":true,"salary":1400,"year":"2"}
   
   string key = "id";
   // panics since `id` requires an integer
   e[key] = "E112400";
}

Field, optional field, and member access are typed against the record type descriptor which allow for compile-time validations of reads and updates.

import ballerina/io;

type Employee record {|
   int id;
   string name;
   boolean manager = false;
   string department?;
   decimal? salary?;
   (int|string)...;
|};

public function main() {
   Employee e = {
       id: 1124,
       name: "John",
       salary: 1200,
       "year": "2"
   };

   string id = e.id; // compile-time error since `id` is an `int` value
   e.salary = "1400"; // compile-time error since `salary` requires a `decimal` value
   string key = "base";
   e[key] = 1000f; // compile-time error since none of the fields allow float values
}

Working with optional fields​

Ballerina also supports a few features that make it easier to work with optional fields that are of a type that doesn’t include nil.

import ballerina/io;

type Person record {
   string name;
   int id?;
};

public function main() {
   int? idNil = ();
   Person p1 = {name: "Jo", id: idNil}; // valid - effectively one field
   io:println(p1); // {"name":"Jo"}

   int? idNum = 1234;
   Person p2 = {name: "Joy", id: idNum};
   io:println(p2); // {"name":"Joy","id":1234}

   p2.id = (); // removes the `id` field if present
   io:println(p2); // {"name":"Joy"}
}

Here, the id field is an optional field of type int and does not allow nil. We could use an expression of type int? for that field in the mapping constructor. If the value is nil, it is effectively the same as not setting a value for the field. Similarly, you could use nil in an assignment statement with field access on the left hand side to remove the field if it is present in the record value.

Lang library functions​

The lang.map lang library, which defines functions on mapping values, can also be used with records.

import ballerina/io;

type Employee record {|
   int id;
   string name;
   boolean manager = false;
   string department?;
|};

public function main() {
   Employee e = {
       id: 1124,
       name: "John"
   };

   boolean hasDepartment = e.hasKey("department");
   io:println(hasDepartment); // false
   io:println(e.keys()); // ["id","name","manager"]
}

Typing​

Typing is somewhat similar to maps, but the type compatibility checks happen at field-level. Record concepts such as openness and optionality of fields are also taken into consideration.

type E1 record {|
   int id;
   string name;
|};

type E2 record {|
   string id;
   string name;
|};

In the above example, E1 is not a subtype of E2 and similarly E2 is not a subtype of E1 since the id fields are of incompatible types.

type E1 record {|
   int id;
   string name;
|};

type E2 record {|
   string|int id;
   string name;
|};

E1 e1 = {name: "Jo", id: 1120};
E2 e2 = {name: "Joy", id: "3040"};
E2 a = e1; // OK because `E1` is a subtype of `E2`
E1 b = e2; // error because `E2` is not a subtype of `E1`

But if the type of the id field in E2 is string|int, E1 will now be a subtype of E2, because int is a subtype of string|int. However, E2 is not a subtype of E1.

type Employee record {|
   int id;
   string name;
|};

type Person record {
   string name;
};

type Manager record {|
   string name;
   string...;
|};

Employee employee = {name: "Jo", id: 1120};
Person person = {name: "Joy", "id": "3040"};
Person a = employee; // OK because `Employee` is a subtype of `Person`
Employee b = person; // error because `Person` is not a subtype of `Employee`
Manager m = employee; // error because `Employee` is not a subtype of `Manager`

Here, Employee is a closed record whereas Person is an open record. Employee is a subtype of Person because the types of the explicitly-defined fields meet the requirements for subtyping and any additional fields (e.g., id) are of types compatible with the rest descriptor type (which is implicitly anydata here). On the other hand, Person is not a subtype of Employee because it may or may not have an id field and even if present the id field may not have an int value. Moreover, since Person is an open record it may also have additional fields which are not allowed by Employee.

Similarly, Employee is not a subtype of Manager since the rest descriptor type does not allow int fields (which is the type of the id field in Employee).

Let’s now look at typing with optional fields.

type Employee record {|
   int id?;
   string name;
|};

type Manager record {|
   int id;
   string name;
|};

Employee employee = {name: "Jo"};
Manager manager = {name: "Joy", id: 3040};
Employee a = manager; // OK because `Manager` is a subtype of `Employee`
Manager b = employee; // error because `Employee` is not a subtype of `Manager`

Employee represents mapping values that have a string name field and optionally an int id field. Manager represents mapping values that have a string name field and an int id field. Given that all the other subtyping conditions are met, since Manager represents mapping values that will always have the id field, the set of values that belong to Manager is a subset of the set of values that belong to Employee, and therefore, Manager is a subtype of Employee. On the other hand, since Manager always requires an id field, Employee (which may or may not have an id field) is not a subtype of Manager.

Now that we understand records, let’s look at a few advanced concepts.

Sometimes, we may want to specify that a particular field will never be present in the record. This is done using the never type: the type to which no values belong to.

type Person record {
   never id?;
   string name;
};

Note how we have defined an optional field of type never for id. Since it is an optional field, the record value may or may not contain it. But, combined with the fact that no value belongs to never, this definition effectively means the record can never have an id field.

type Person record {
   never id?;
   string name;
};

type Employee record {
   int id?;
   string name;
};

Person person = {name: "Jo"};
Employee employee = person;

While both Person and Employee are open records here, by specifying id to be an optional field of the never type we guarantee that it will never have a field named id. But what that also means is that it implicitly meets the requirement for id in Employee, which is to be int if present but it may also be absent. Therefore, Person is a subtype of Employee.

Immutable mapping values​

As with other values, a record value can also be made immutable by using an intersection with readonly or using the cloneReadOnly function from lang.value lang library.

type Person record {
   int id;
   string name;
};

public function main() {
   Person & readonly person = {name: "Jo", id: 12121};
   person.id = 12112; // compile-time error since an immutable value cannot be updated
}

Immutable record fields​

Alternatively, there could be a scenario where you want to make only certain fields of a record immutable.

type Person record {
   readonly string[] name;
   readonly & string[] address;
   boolean employed;
};

public function main() {
   Person person = {
       name: ["Maya", "Silva"],
       address: ["Colombo", "Sri Lanka"],
       employed: true
   };
  
   person.name = ["maya", "silva"]; // compile-time error
   person.name[0] = "maya"; // compile-time error
   person.address = ["Colombo", "SL"]; // valid
   person.address[1] = "SL"; // compile-time error
}

This can be done using readonly fields. Note how the name field is defined as a readonly field. This effectively means two things: 1. the field is effectively final meaning a new value cannot be set to this field once the record value is created. 2. the value required by the field is an immutable value: the type is effectively immutable string[] (readonly & string[]) here. So not only can we not set a new array to the name field we also can't update the array already set to the name field.

In contrast, the address field is not a readonly field, but its type is an intersection with readonly, which requires the value to be an immutable string array. Therefore, while the array set to the address field cannot be updated, it is possible to set a new array value to the address field since it is not a readonly field.

Record type inclusion​

Ballerina also supports record type inclusion.

type Person record {
   readonly string name;
   int id?;
   string[]|string address;
   string country;
};

type Employee record {
   *Person;
   int id;
   string address;
};

Employee employee = {
   name: "Maya",
   id: 1232,
   country: "Colombo, Sri Lanka",
   address: "LKA"
};

With the *Person syntax, Employee includes Person. What inclusion does is that it effectively copies all the fields from the included record (Person) to the including record (Employee). This way you would not have to manually copy all the fields from an included record. When creating a record value of type Employee it will be mandatory to specify the required fields included from Person such as name.

The including record can also override the types of the fields as long as the overriding type is a subtype of the overridden type. Note how we’ve done this for the address field: while Person allows the address field to be a string or a string array, by overriding it in Employee we require the address field to be a string in Employee. Similarly, we’ve made id a required field in Employee, even though it is optional in Person.

FillMember Operation​

Updating a mapping value in Ballerina also supports the FillMember operation where possible.

import ballerina/io;

type Settings record {|
    string name?;
    boolean enabled = false;
    int priority = 0;
|};

type Config record {|
    string name;
    Settings settings?;
|};

public function main() {
    Config config = {name: "basic"};
    io:println(config); // {"name":"basic"}

    config.settings.enabled = true;
    io:println(config); // {"name":"basic","settings":{"enabled":true,"priority":0}}
}

Here, we have an optional field named settings for which we do not specify a value when creating config. Note how we can still use config.settings.enabled to assign a value to the enabled field of settings, and how we now have a settings field with the default value for priority from the record type descriptor. What happens here is that Ballerina adds in the settings fields if it is not specified. This is possible because all of the fields that are required in the Settings record have a default value that can be used to construct a value of type Settings. This is similar to explicitly using an empty mapping constructor for the settings field and then updating the enabled field.

config.settings = {};
config.settings.enabled = true;

But if there are fields that cannot be filled, such an update attempt can result in an error.

type Settings record {|
    string name?;
    boolean enabled = false;
    int priority;
|};

type Config record {|
    string name;
    Settings settings?;
|};

public function main() {
    Config config = {name: "basic"};
    config.settings.enabled = true; // error: {ballerina/lang.map}KeyNotFound {"message":"cannot find key 'settings'"}
}

Here, Ballerina cannot fill in the settings field since priority is now a required field. Attempting to set a value to config.settings.enabled will now result in a runtime panic.

Other features of mapping constructors​

Let us now have a quick look at a few features available with mapping constructor expressions which are used to create mapping values.

Variable name fields​

type Person record {
   string name;
   int id?;
};

function getPerson(string name, string id) returns Person|error => {
   name: name,
   id: check int:fromString(id) 
};

// can be written as
function getPerson(string name, string id) returns Person|error => {
   name,
   id: check int:fromString(id) 
};

If a mapping constructor contains a key-value pair where the value is variable reference and the name of the variable is the same as the key, instead of writing name: name we can just use name and the compiler will handle it as key value pair for us.

Spread fields​

Mapping constructors also support spread fields, where we can use another mapping value in a spread field to spread out its members as key value pairs in the new mapping value being constructed.

import ballerina/io;

type Employee record {
   string name;
   int id;
   string[] address;
};

public function main() {
   record {
       string name;
       string[] address;
       never id?;
   } person = {
       name: "Maya",
       address: ["Colombo", "Sri Lanka"]
   };
   Employee employee = {id: 1234, ...person};
   io:println(employee); // {"name":"Maya","id":1234,"address":["Colombo","Sri Lanka"]}
}

Note how the values from person get copied into the Employee value being created. Also note how we’ve used an optional field of type never to specify that the person value will never have an id field, which is specified explicitly in the mapping constructor.

Computed name fields​

Sometimes you would want to evaluate and use an expression that results in a string as the key (e.g., reference to a constant, function parameter, etc.) rather than specifying the key directly in the mapping constructor.

import ballerina/io;

const NOT_FOUND_ERR = "NotFound";
const INVALID_TYPE_ERR = "InvalidType";

function getErrorCodeMapWithDefaults(string id, int code) returns map<int> {
    map<int> errorCodes = {
        [NOT_FOUND_ERR] : 1100,
        [INVALID_TYPE_ERR] : 2230,
        [id] : code,
        default: 1000
    };
    return errorCodes;
}

public function main() {
    // {"default":1000,"NotFound":1100,"InvalidType":2230,"identifier":10200}
    io:println(getErrorCodeMapWithDefaults("identifier", 10200));
}

We can do this using computed name fields, where the key is an expression within square brackets, which tells Ballerina to evaluate the expression to get the key. The expression is expected to be of static type string, since keys are required to be strings. While in the example we have parameter and constant references, this can be any expression including variable reference and function call expressions, as long as they evaluate to strings.

readonly mapping constructor fields​

Previously we saw how a record type descriptor can define readonly fields. A mapping constructor can also specify fields as readonly fields.

type Person record {
   string name;
   int id;
   boolean employed;
};

public function main() {
   Person person = {
       readonly name: "John",
       id: 1123,
       employed: false
   };
   person.name = "john"; // panics
}

Here, even though the Person record does not specify the name field to be a readonly field, the mapping constructor does so, making the name field in the mapping value a readonly field. The inherent type is therefore decided based on both the expected type (Person here) and the mapping constructor, and any attempt to update the name field will panic similar to when the name field is a readonly field in Person.

Maps as records and records as maps​

Given that both map and record types represent mapping values, a mapping value that has a record type as the inherent type can be used where a map type is expected and vice versa, as long as the values (and types) are compatible similar to map to map and record to record compatibility.

map<int> m = {};
record {| int...; |} n = m; // OK
map<int> o = n; // OK

A map<int> value is a mapping value that can contain any field that has an int value. The equivalent record type for this would be record {| int...; |}, which also represents the same set of mapping values. So map<int> is a subtype of record {| int...; |} and record {| int...; |} is a subtype of map<int>, which allows the assignments above.

map<int> m = {};
record {| int i; int...; |} n = m; // error, since `m` may not have a field with key `i`
map<int> o = n; // OK

But if the record has required fields, with or without a default value, the map type will not be a subtype of the record type since the mapping value with inherent type map<int> may not have the required fields. On the other hand, as long as the type of the required field is a subtype of the type expected by the map, the record type will be a subtype of the map type.

map<int> m = {};

record {| int i; int j; |} n = m; // error
record {| int i; never j?; int...; |} o = m; // error

map<int> p = n; // OK
map<int> q = o; // OK

Similarly, a map type can never be a subtype of a closed record type (with a finite number of fields) or a record type that has never-typed optional fields.

Hope you found this useful as an overview of some of the core concepts related to mapping values and types in Ballerina.

This article was previously published on StackOverflow.