Golang tips: why pointers to slices are useful and how ignoring them can lead to tricky bugs
Pointers to slices: useless or indispensable?
Doubt
Today while I was working a nice question arose:
Why in many BuiltIn function and libraries it is common to see as arguments pointers to slices, aren’t slices always passed by reference?
For example, in the implementation of the api-machinery of Kubernetes we can see a function with the following signature:
func Convert_Slice_string_To_string(input *[]string, out *string, s conversion.Scope) error;:, and in the example of the priority queues, we can find again something similar:
func (pq *PriorityQueue) Pop() interface{};Aren’t slices already pointers to the underlying data?
Let’s investigate a bit making use of the Go-Playground to test the behaviours of the code reported.
During the whole explanation, I will make use of the same example: a function that initializes a slice, passes it as an argument to a second function, modifies it, and then prints the slice to verify the content.
It is a common belief that slices are passed by reference, in fact, the following example will print [b,b] and [b,b] even if the slice was initialized to [a,a]since it got modified during the execution of the literal function and the change is visible to the main.
func main() {
slice:= []string{"a","a"} func(slice []string){
slice[0]="b";
slice[1]="b";
fmt.Print(slice)
}(slice)
fmt.Print(slice)
}
Making use of pointers leads to the same result, in fact, the following code
func main() {
slice:= []string{"a","a"}
func(slice *[]string){
(*slice)[0]="b";
(*slice)[1]="b";
fmt.Print(*slice)
}(&slice)
fmt.Print(slice)
}prints again [b,b] and [b,b]. Therefore passing it by pointer looks useless and it seems that the slice gets passed by reference anyway and the content is modified in both cases.
So… why those functions have that signature?
Explanation
You can roughly imagine the implementation of the slice as:
type sliceHeader struct {
Length int
Capacity int
ZerothElement *byte
}Passing a slice to a function by value, all the fields are copied and only the data can be modified and accessed from outside through the copy of the pointer.
However, keep in mind that if the pointer is overwritten or modified (due to a copy, an assign, or an append) no change will be visible outside the function, moreover, no change of length or capacity will be visible to the initial function.
The answer to the questions, then, is simple, but hidden inside the implementation of the slice itself:
The pointer to a slice is indispensable when the function is going to modify the structure, the size, or the location in memory of the slice and every change should to be visible to those who call the function.
When we pass a slice to a function as an argument the values of the slice are passed by reference (since we pass a copy of the pointer), but all the metadata describing the slice itself are just copies.
We can modify the data of the slice in the literal function, however if the pointer to the data changes for any reason or the slice metadata is modified, the change can be partially or no visible at all to the outside function.
For example, if the slice gets allocated again, a new location of the memory is used; even if the values are the same, the slice points to a new location and therefore no modification of the values will be visible ouside, since the slices are pointing to two different localtions (the pointer in the slice copy got overwritten).
Therefore the same example, forcing the slice to be allocated again,
func main() {
slice:= []string{"a","a"}
func(slice []string){
slice= append(slice, "a")
slice[0]="b";
slice[1]="b";
fmt.Print(slice)
}(slice)
fmt.Print(slice)
}will print [b,b,a] and [a,a]. Moving the append after the manipulation of the slice, we can notice that the behavior is different since the slice got reallocated after the manipulation of the values and the pointer is still pointing to the initial memory address. You can check it with the following code
func main() {
slice:= []string{"a","a"}
func(slice []string){
slice[0]="b";
slice[1]="b";
slice= append(slice, "a")
fmt.Print(slice)
}(slice)
fmt.Print(slice)
}that prints [b,b,a] and [b,b] for the reasons just explained.
This behavior can lead to tricky bugs to be spotted since the result depends on the size of the initial array, for example, the following code
func main() {
slice:= make([]string, 2, 3)
func(slice []string){
slice= append(slice, "a")
slice[0]="b";
slice[1]="b";
fmt.Print(slice)
}(slice)
fmt.Print(slice)
}prints [b,b,a] and [b,b] since the array is not allocated again and the pointer stays the same.
However appending a string more slice=append(slice, "a", "a"), the array gets allocated again and the result would be [b,b,a,a] and [ ] ( an empty array since it was not initialized).
To spot this kind of bugs in the middle of hundreds or thousands of lines can be quite difficult.
Therefore make sure to keep in mind that you can pass a slice by value if you want to modify merely the values of the elements, not their number or position, otherwise weird bugs will arise from time to time.
Now you are ready to understand which is the result of the following snippet of code, check the answer in the Golang playground or write it in the comments:
func main() {
slice:= make([]string, 1, 3)
func(slice []string){
slice=slice[1:3]
slice[0]="b"
slice[1]="b"
fmt.Print(len(slice))
fmt.Print(slice)
}(slice)
fmt.Print(len(slice))
fmt.Print(slice)
}
