Godot + Python = ❤

Godot is one of the most popular open source game engines. Although a lot of work is being put into its development, there are still some features missing that I want to use. One of these missing features is an easy way to access a camera stream and process images. As of Godot 4.2 only macOS and iOS are supported [1]. Python on the other hand has a very simple interface to capture a camera stream and process the frames using OpenCV for python [2].

In this project, I want to combine the power of Python and Godot by creating a communication pipeline between them. The idea is to run a UDP server in godot using the builtin UDPServer and then connecting to it in python and start sending data from there. In this project, we will only implement the basic communication structure and send a greeting between Godot and the Python application. In the next series, I investigate the video streaming and image processing part more.

Project Structure

The project will be structured into two parts, the python code and the godot project. I created an empty godot project and an folder with the python gitignore [3], each in a separate subfolder.

Godot Server

Let’s start with the UDP server in Godot. Create an empty Node2D for the scene root and save the scene. Add a child node for the server and attach a gd script, called upd_server.

Create a variable for the UDP server object and instantiate it in the ready function of the script. I am going to use the port 4242, but you can use any free port.

extends Node

var server: UDPServer

func _ready() -> void:
 server = UDPServer.new()
 server.listen(4242)

For an initial test, we are going to listen for a new connection, receive a single message and reply with a greeting from godot. To check if new connections are available, we first have to poll the server.

...

func _process(_delta: float) -> void:
 server.poll()
 if server.is_connection_available():
  # TODO: receive message
  # TODO: send greeting

Now once a connection is available, we can accept the connection and store the udp peer with the take_connection function. This object contains the ip address of the client and we can use it to receive packets from this client. To get a message from the client, we use the get_packet function. This will return the raw byte array of the sent data.

func _process(_delta: float) -> void:
 server.poll()
 if server.is_connection_available():
  var peer: PacketPeerUDP = server.take_connection()
  var packet = peer.get_packet()
  print("Received: '%s' %s:%s" % [packet.get_string_from_utf8(), peer.get_packet_ip(), peer.get_packet_port()])
  
  # TODO: send greeting

Finally we can send our greeting message. Here we need to encode the string message into bytes, we cannot directly send a string by itself. We will use the UTF8 encoding, this is also the default in python. Now the full script should look like this:

extends Node

var server: UDPServer

func _ready() -> void:
 server = UDPServer.new()
 server.listen(4242)

func _process(_delta: float) -> void:
 server.poll()
 if server.is_connection_available():
  var peer: PacketPeerUDP = server.take_connection()
  var packet = peer.get_packet()
  print("Received: '%s' %s:%s" % [packet.get_string_from_utf8(), peer.get_packet_ip(), peer.get_packet_port()])
  
  peer.put_packet("Hello from Godot!".to_utf8_buffer())

Python Client

Now let’s move on to the python client. Let’s create a client.py script in our python directory. We can use the builtin socket module to create a udp socket by specifying the AF_INET family and the SOCK_DGRAM type.

NOTE: It’s a good idea to create a virtual environment for each project, however we won’t need any packages and therefore I won’t go into detail on how to do this here.

import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

Using the sendto method, we can send a greeting message from our python code. We need to specify the IP address and the por tof the server. Since we are communicating locally, we can use the localhost address (127.0.0.1). Make sure that the port matches the one you set earlier for the Godot server, in my case this is 4242.

ip = "127.0.0.1"
port = 4242

client_socket.sendto("Hello from Python!".encode(), (ip, port))

Now finally we can receive the response to our greeting. Here we have to specify a buffer size. For a short greeting a small buffer size of 1024 bytes is sufficient.

data, (recv_ip, recv_port) = client_socket.recvfrom(1024)
print(f"Received: '{data.decode()}' {recv_ip}:{recv_port}")

Note that this call is synchronous, so the current thread is blocked until a message is received, or if there were a timeout until it runs out. To define a timeout, we can use the settimeout method on our socket. The full code for the python cloud should look now something like this:

import socket

ip = "127.0.0.1"
port = 4242

client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
client_socket.settimeout(1)

client_socket.sendto("Hello from Python!".encode(), (ip, port))

data, (recv_ip, recv_port) = client_socket.recvfrom(1024)
print(f"Received: '{data.decode()}' {recv_ip}:{recv_port}")

Test Time

Now you can start testing. First run the godot project to start the server. Then you can run the python client from the command line. You should now be able to see the greetings from python in the Godot console and from Godot in the terminal of the python client.

Outlook

This demo shows how a communication pipeline between a Python and a Godot application can be created. This opens up some interesting doors to camera streaming, image processing but also machine learning as there is an abundance of libraries and tutorials on how these work with Python. If you build something interesting with this let me know, I already have some ideas I want to try as well.

Source Code

The full source code for this project is available on GitHub:

GitHub - trflorian/godot-python-comm: Demo of communication between a Godot project and a Python…

References

[1] https://docs.godotengine.org/en/stable/classes/class_cameraserver.html#class-cameraserver
[2] https://pypi.org/project/opencv-python/
[3] https://github.com/github/gitignore/blob/main/Python.gitignore