The Go godror driver now supports the Oracle Database 23ai VECTOR data type

Tamás Gulácsi’s excellent godror driver for the Go language now supports the Oracle Database 23ai VECTOR data type, courtesy of a pull request by Sudarshan Soma, a senior member of Oracle Database’s driver development group.

Photo by Bernard Hermant on Unsplash

Oracle Database 23ai introduced a VECTOR data type to aid artificial intelligence and machine learning search operations. Vectors are an homogeneous array of 8-bit signed integers, 8-bit unsigned integers, 32-bit floating-point numbers, or 64-bit floating-point numbers. You can optionally define the number of dimensions for the data. Vectors can be “dense” (the default), or “sparse” when data is mostly zeroes.

For example to create a table with two VECTOR columns, one being “dense” containing 20 dimensions of 64-bit floating point numbers, and the other column being a sparse vector of 35 8-bit signed integers:

create table vector_table (
    v64 vector(20, float64),
    v8  vector(35, int8, sparse)
)

Oracle Database 23ai supports a number of advanced operations such as similarity searches on vector embeddings stored as the VECTOR data type. See the Oracle AI Vector Search User’s Guide for all the details.

Here is a basic example in Go that uses godror 0.48 (or later). You also need Oracle Database 23.7 (or later). The code simply inserts and fetches vectors to show the godror side of working with them.

package main

import (
  "context"
  "database/sql"
  "fmt"
  "github.com/godror/godror"
  "log"
  "math/rand/v2"
  "strconv"
  "time"
)

// Generates a slice of random float32 numbers
func randomFloat32Slice(size int) []float32 {
  slice := make([]float32, size)
  for i := range slice {
    slice[i] = rand.Float32() * 10
  }
  return slice
}

func main() {

  db, err := sql.Open("godror", ` user="scott" password="tiger" connectString="localhost/orclpdb1" `)
  if err != nil {
    log.Fatalf("Failed to connect to database: %v", err)
  }
  defer db.Close()

  ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
  defer cancel()

  dropTable := `DROP TABLE IF EXISTS text_search`
  _, err = db.ExecContext(ctx, dropTable)

  // Create a table with VECTOR columns
  dimensions := 5
  createTable := `CREATE TABLE text_search (
      id NUMBER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
      text_column CLOB NOT NULL,
      dense_embedding VECTOR(` + strconv.Itoa(dimensions) + `),
      sparse_embedding VECTOR(` + strconv.Itoa(dimensions) + `, FLOAT32, SPARSE))`
  _, err = db.ExecContext(ctx, createTable)
  if err != nil {
    log.Fatalf("Error creating table: %v", err)
  }
  fmt.Println("Table created successfully.")

  conn, err := db.Conn(ctx)
  if err != nil {
    log.Fatal(err)
  }
  defer conn.Close()

  stmt, err := conn.PrepareContext(ctx, `INSERT INTO text_search (text_column, dense_embedding, sparse_embedding)
      VALUES (:1,:2,:3)`)
  if err != nil {
    log.Fatal(err)
  }
  defer stmt.Close()

  // Insert some vector data
  float32Vector := godror.Vector{Values: randomFloat32Slice(dimensions)}
  sparseFloat32Vector := godror.Vector{Dimensions: uint32(dimensions), Indices: []uint32{0, 2, 4}, Values: randomFloat32Slice(3)}
  _, err = stmt.ExecContext(ctx, "SAMPLE TEXT1", &float32Vector, &sparseFloat32Vector)
  if err != nil {
    log.Fatal(err)
  }

  float32Vector = godror.Vector{Values: randomFloat32Slice(dimensions)}
  sparseFloat32Vector = godror.Vector{Dimensions: uint32(dimensions), Indices: []uint32{1, 3, 4}, Values: randomFloat32Slice(3)}
  _, err = stmt.ExecContext(ctx, "SAMPLE TEXT2", &float32Vector, &sparseFloat32Vector)
  if err != nil {
    log.Fatal(err)
  }
  fmt.Println("Inserted rows successfully.")

  // Query the vectors
  rows, err := conn.QueryContext(ctx, fmt.Sprintf(`
    SELECT id, text_column, dense_embedding, sparse_embedding from text_search`))
  if err != nil {
    log.Fatal("QueryContext failed: %v", err)
  }
  defer rows.Close()

  fmt.Println("Reading rows:")
  for rows.Next() {
    var outDenseEmbedding godror.Vector
    var outSparseEmbedding godror.Vector
    var outValue string
    var id godror.Number
    if err := rows.Scan(&id, &outValue, &outDenseEmbedding, &outSparseEmbedding); err != nil {
      log.Fatal("Scan failed: %v", err)
    }
    fmt.Printf("\tID:[%s], Text:[%s], Dense Embedding:[%v] and Sparse Embedding:[%v]\n",
      id, outValue, outDenseEmbedding.Values, outSparseEmbedding)
  }
}

You can see it makes use of the new godror.Vector() struct.

The output will be:

Table created successfully.
Inserted rows successfully.
Reading rows:
  ID:[1], Text:[SAMPLE TEXT1], Dense Embedding:[[7.2318554 8.494286 2.4187808 5.6896343 1.2094277]] and Sparse Embedding:[{5 [0 2 4] [3.5823603 3.5613031 3.4197946] true}]
  ID:[2], Text:[SAMPLE TEXT2], Dense Embedding:[[8.704301 2.638142 5.376678 7.2791424 1.3518274]] and Sparse Embedding:[{5 [1 3 4] [6.7112656 5.6008286 0.21929502] true}]

You can easily extend the example to use the power of Oracle Database 23ai similarity searches.

Let us know how you use vectors in Go.