Chatting With ChatGPT On a Sony PSP

Say hello to Chat-GPSP, a Rust application for PSP — Sony’s first handheld released in 2005 — to chat with ChatGPT.

Photo by Dennis Cortés on Unsplash

Foreword

Much like those Guinness World Records one wins just because they were the first to attempt such a weird thing, I wanted to be the first to create an application capable of making you chat with the arguably most disruptive technology in years — OpenAI’s GPT — from a Sony PlayStation Portable, my most beloved handheld console to date.

Rust on PSP

Rust on PSP is possible thanks to the rust-psp crate [5], a wonderful library you can install by simply running cargo install cargo-psp, setting rust version to nightly (rustup override set nightly, in the root of your repository), and adding psp = { version = "0.3.7" } to your Cargo.toml.

The crate is an SDK for PSP with no dependency on the famous PSPSDK [6] in C++, although the function signatures are almost always the same, making it easier to convert code between the two.

The SDK is relatively easy to use, and has support for core and alloc, although there is no support for the standard library. The lack of support for std limits you to use only crates that supports no_std, which, fortunately, are not few since it is a widespread constraint in embedded systems, but is still one of the main limitations of the crate.

Project Structure

To achieve my goal, I split the problem in the following sub-problems

• Reading input
• Setting up networking and DNS
• Interacting With OpenAI APIs.

The GUI leave much to be desired, and is just terminal-like, but it was outside the scope of the project.

Reading from OSK — On-Screen Keyboard

In the PSP development world, the virtual keyboard is known as the OSK, which stands for On-Screen Keyboard.

Setting up a working OSK was one of the tasks that took me the longest time, even though it was possible to find some examples on the internet. The main problem, was that I used, for speed of development, the PPSSPP emulator, and only from time to time the actual device. The two devices behaves slightly differently, and that caused me many headaches, and I had to rework the code many times.

Moreover, rust-psp OSK state management enum was not quite correct (it contained a variant too much), which the code had to account for.
I created a PR fixing this that was merged, so versions newer than 0.3.7 should not have the problem.

Coming to the code, I created a separate module for the OSK, which implements a struct OskState that wraps around SceUtilityOskState — this is done to help cast an i32 to a SceUtilityOskState. Other than that, and some helper functions, the read_from_osk function is the core function to be used to read a string from the OSK.

pub fn read_from_osk(params: &mut SceUtilityOskParams) -> Option<String> {
    let mut done = false;
    let mut osk_state = OskState::new();

    unsafe {
        sceDisplayWaitVblankStart();
        sceGuSwapBuffers();
        while !done {
            sys::sceGuStart(
                sys::GuContextType::Direct,
                &mut LIST as *mut _ as *mut c_void,
            );
            sys::sceGuClear(ClearBuffer::COLOR_BUFFER_BIT | ClearBuffer::DEPTH_BUFFER_BIT);
            sys::sceGuFinish();
            sys::sceGuSync(GuSyncMode::Finish, sys::GuSyncBehavior::Wait);
            sceGuClear(ClearBuffer::COLOR_BUFFER_BIT);
            
            match osk_state.get() {
                // TODO: switch to PspUtilityDialogState when it's implemented
                SceUtilityOskState::None => done = true,
                // This is the visible state, but the 0.3.7 enum has a variant too much
                SceUtilityOskState::Initialized => {
                    if sceUtilityOskUpdate(1).is_negative() {
                        panic!("cannot update osk");
                    }
                }
                // same as above, it is the Quit state
                SceUtilityOskState::Visible => {
                    if sceUtilityOskShutdownStart().is_negative() {
                        panic!("cannot shutdown osk");
                    }
                }
                _ => (),
            }
            sceDisplayWaitVblankStart();
            sceGuSwapBuffers();
        }
    }

    // Convert the read text to a string, if any
    let osk_data: &SceUtilityOskData = unsafe { params.data.as_ref().unwrap() };
    match osk_data.result {
        sys::SceUtilityOskResult::Cancelled => None,
        _ => {
            let out_text = unsafe {
                mut_ptr_u16_to_vec_char(osk_data.outtext, osk_data.outtextlength as usize)
            };
            let out_text = String::from_iter(out_text);
            Some(out_text)
        }
    }
}

Code for the OSK was inspired by CrossCraft [1].

Networking, TLS, and DNS

PSP sockets closely resembles Linux ones, thus implementing a UDP and TCP socket abstraction was not too difficult (implementation can be found in the repo).

TLS

One of the major problems with the PSP is that it does not come with TLS support, so I had to find a Rust-only implementation of it, and luckily that was offered by the embedded-tls crate. In order to make the sockets compatible with it, I had to implement the traits embedded-io::Read and embedded-io::Write from the crate embedded-io. Finally, I used another great crate, embedded-nal, to have a handy way to manage socket addresses and IPs representations.

Using embedded-tls I was able to make TLS work on the PSP. For the implementation, I started from here [2], and then adapted the implementation to use more up-to-date crates.

lazy_static::lazy_static! {
    static ref REGEX: Regex = Regex::new("\r|\0").unwrap();
}

pub struct TlsSocket<'a> {
    tls_connection: TlsConnection<'a, TcpSocket, Aes128GcmSha256>,
    tls_config: TlsConfig<'a, Aes128GcmSha256>,
}

impl<'a> TlsSocket<'a> {
    pub fn new(
        socket: TcpSocket,
        record_read_buf: &'a mut [u8],
        record_write_buf: &'a mut [u8],
        server_name: &'a str,
        cert: Option<&'a [u8]>,
    ) -> Self {
        let tls_config: TlsConfig<'_, Aes128GcmSha256> = match cert {
            Some(cert) => TlsConfig::new()
                .with_server_name(server_name)
                .with_cert(Certificate::RawPublicKey(cert))
                .enable_rsa_signatures(),
            None => TlsConfig::new()
                .with_server_name(server_name)
                .enable_rsa_signatures(),
        };

        let tls_connection: TlsConnection<TcpSocket, Aes128GcmSha256> =
            TlsConnection::new(socket, record_read_buf, record_write_buf);

        TlsSocket {
            tls_connection,
            tls_config,
        }
    }

    pub fn open(&mut self, seed: u64) -> Result<(), embedded_tls::TlsError> {
        let mut rng = ChaCha20Rng::seed_from_u64(seed);
        let tls_context = TlsContext::new(&self.tls_config, &mut rng);
        self.tls_connection
            .open::<ChaCha20Rng, NoVerify>(tls_context)
    }

    pub fn write(&mut self, buf: &[u8]) -> Result<usize, embedded_tls::TlsError> {
        self.tls_connection.write(buf)
    }

    pub fn write_all(&mut self, buf: &[u8]) -> Result<(), embedded_tls::TlsError> {
        self.tls_connection.write_all(buf)
    }

    pub fn read(&mut self, buf: &mut [u8]) -> Result<usize, embedded_tls::TlsError> {
        self.tls_connection.read(buf)
    }

    pub fn read_string(&mut self) -> Result<String, embedded_tls::TlsError> {
        let mut buf = Self::new_buffer();
        let _ = self.read(&mut buf)?;

        let text = String::from_utf8_lossy(&buf);
        let text = REGEX.replace_all(&text, "");
        Ok(text.into_owned())
    }

    pub fn flush(&mut self) -> Result<(), embedded_tls::TlsError> {
        self.tls_connection.flush()
    }
}

DNS

Although the PSP has native DNS, I was not able to make it work on my PSP-3000 (it can as well be that I was using it wrong), thus I had to find a no_std crate I could use to implement my own DNS. Luckily, that was not difficult to find, the dns-protocol crate offers all you need just for this purpose.

I created a DnsResolver trait (well, to be fair that was implemented in a following iteration, but for storytelling purposes it’s not relevant) as the composition of two other traits ResolveHostname, and ResolveAddr. Types implementing ResolveHostname can resolve a hostname to an IP, types implementing ResolveAddr the opposite. Types implementing both, can be “complete” DNS Resolvers.

pub trait ResolveHostname {
    type Error: Debug;
    fn resolve_hostname(&mut self, hostname: &str) -> Result<SocketAddr, Self::Error>;
}

pub trait ResolveAddr {
    type Error: Debug;
    fn resolve_addr(&mut self, addr: in_addr) -> Result<String, Self::Error>;
}

pub trait DnsResolver: ResolveHostname + ResolveAddr {}

Then I created a type implementing those traits (well, the ResolveAddr is still a todo!() TBH, but it is not needed for this application, I might implement it later on).

/// Create a DNS query for an A record
pub fn create_a_type_query(domain: &str) -> Question {
    Question::new(domain, dns_protocol::ResourceType::A, 1)
}

/// A DNS resolver
pub struct DnsResolver {
    udp_socket: UdpSocket,
    dns: SocketAddr,
}

impl DnsResolver {
    /// Create a new DNS resolver
    pub fn new(dns: SocketAddr) -> Result<Self, ()> {
        let mut udp_socket = UdpSocket::open().map_err(|_| ())?;
        udp_socket.bind(Some(dns)).map_err(|_| ())?;

        Ok(DnsResolver { udp_socket, dns })
    }

    /// Create a new DNS resolver with default settings.
    /// The default settings are to use Google's DNS server at `8.8.8.8:53`
    pub fn default() -> Result<Self, ()> {
        let dns = GOOGLE_DNS_HOST.clone();
        let mut udp_socket = UdpSocket::open().map_err(|_| ())?;
        udp_socket.bind(None).map_err(|_| ())?;

        Ok(DnsResolver { udp_socket, dns })
    }

    pub fn resolve(&mut self, host: &str) -> Result<in_addr, ()> {
        // connect to the DNS server, if not already
        if self.udp_socket.get_socket_state() != UdpSocketState::Connected {
            self.udp_socket.connect(self.dns).map_err(|_| ())?;
        }

        // create a new query
        let mut questions = [super::dns::create_a_type_query(host)];
        let query = dns_protocol::Message::new(
            0x42,
            Flags::standard_query(),
            &mut questions,
            &mut [],
            &mut [],
            &mut [],
        );

        // create a new buffer with the size of the message
        let mut tx_buf = a_vec![0u8; query.space_needed()];
        // serialize the message into the buffer
        query.write(&mut tx_buf).map_err(|_| ())?;

        // send the message to the DNS server
        let _ = self.udp_socket.write(&tx_buf).map_err(|_| ())?;

        let mut rx_buf = [0u8; 1024];

        // receive the response from the DNS server
        let data_len = self.udp_socket.read(&mut rx_buf).map_err(|_| ())?;

        if data_len == 0 {
            return Err(());
        }

        // parse the response
        let mut answers = [ResourceRecord::default(); 16];
        let mut authority = [ResourceRecord::default(); 16];
        let mut additional = [ResourceRecord::default(); 16];
        let message = dns_protocol::Message::read(
            &rx_buf[..data_len],
            &mut questions,
            &mut answers,
            &mut authority,
            &mut additional,
        )
        .map_err(|_| ())?;

        if message.answers().is_empty() {
            return Err(());
        }
        let answer = message.answers()[0];

        match answer.data().len() {
            4 => {
                let mut octets = [0u8; 4];
                octets.copy_from_slice(answer.data());
                let addr = in_addr(u32::from_be_bytes(octets));
                Ok(addr)
            }
            _ => Err(()),
        }
    }
}

impl traits::dns::ResolveHostname for DnsResolver {
    type Error = ();

    fn resolve_hostname(&mut self, hostname: &str) -> Result<SocketAddr, ()> {
        self.resolve(hostname).map(|addr| addr.to_socket_addr())
    }
}

impl traits::dns::ResolveAddr for DnsResolver {
    type Error = ();

    fn resolve_addr(&mut self, _addr: in_addr) -> Result<String, ()> {
        todo!("resolve_addr")
    }
}

impl traits::dns::DnsResolver for DnsResolver {}

Chatting With GPT

Now that we have implemented a way to prompt the user for input and read it, and have a way to send messages on the network, as well as a way to resolve a hostname to an IP address, all we’re missing is a way to interact with OpenAI APIs, then we will be able to put all the pieces together.

Serde

OpenAI APIs require the client to keep track of the conversation history, so I implemented a couple types to take care of that

#[derive(Debug, Clone, PartialEq)]
pub struct Message {
    role: String,
    pub content: String,
}

impl Message {
    pub fn new_user(content: String) -> Self {
        Self {
            role: "user".to_owned(),
            content,
        }
    }
    pub fn new_assistant(content: String) -> Self {
        Self {
            role: "assistant".to_owned(),
            content,
        }
    }
}

impl Display for Message {
    // ...
}

impl ChatHistory {
    pub fn new(model: String, temperature: f32) -> Self {
        Self {
            model,
            messages: Vec::new(),
            temperature,
        }
    }

    #[inline]
    pub fn new_gpt3(temperature: f32) -> Self {
        Self::new(GPT3_MODEL.to_owned(), temperature)
    }

    pub fn clear(&mut self) {
        self.messages.clear();
    }

    pub fn add_user_message(&mut self, content: String) {
        self.messages.push(Message::new_user(content));
    }

    pub fn add_assistant_message(&mut self, content: String) {
        self.messages.push(Message::new_assistant(content));
    }

    pub fn to_string_with_content_length(&self) -> (String, usize) {
        let string = self.to_string();
        let len = string.len();
        (string, len)
    }
}

impl Display for ChatHistory {
    // ...
}

Now that we have a way to store our chat (Display implementation is used to serialize the content to valid JSON, so that we can send it in our request), we need a way to parse OpenAI’s responses. Since we don’t have the std crate on our side, we need to use serde with default-features off, and serde-json-core. Also, we need heapless crate for String values deserialisation (because we need to set a max string length a priori).

#[derive(Debug, Deserialize)]
pub struct Usage {
    pub prompt_tokens: i32,
    pub completion_tokens: i32,
    pub total_tokens: i32,
}

#[derive(Debug, Deserialize)]
pub struct ResponseMessage {
    pub role: heapless::String<32>,
    pub content: heapless::String<1024>,
}

#[derive(Debug, Deserialize)]
/// # Note
/// This struct does not support the [`Self::logprobs`] field different from `None` yet.
pub struct CompletionChoice {
    pub message: ResponseMessage,
    pub index: i64,
    pub finish_reason: heapless::String<32>,
    pub logprobs: Option<LogprobResult>,
}

#[derive(Debug, Deserialize)]
/// This is a dummy struct, it's not actually used.
pub struct LogprobResult {}

#[derive(Debug, Deserialize)]
/// Completion response from OpenAI.
pub struct CompletionResponse<'a> {
    pub id: &'a str,
    pub object: &'a str,
    pub created: i64,
    pub model: &'a str,
    pub choices: heapless::Vec<CompletionChoice, 3>,
    pub usage: Usage,
}

A lot could be improved on the (de)serialisation side, but this working well enough for the project.

OpenAI Interaction

To interact with OpenAI APIs, I created two structs, OpenAiContext and OpenAi, the former is used to set up the context in which the latter — the one actually interacting with the APIs — operates.

Skipping many of the details of the code that you can find in the repo, the main method of the OpenAi struct is ask_gpt which takes a prompt to ask to GPT, adds it to the current history, and send the request to the APIs, parsing the response.

One of the problems I encountered along the way was that I would often find the connection closed whenever it took me a few seconds too long to type the prompt (which, writing on PSP’s OSK, is not unlikely to happen for messages longer than “hello”). To work around this, I took the decision to open a new connection for each call to ask_gpt, which is probably not a good decision from an engineering POV, but gave me the possibility to keep the use of the struct very simple (the caller do not need to worry about managing the connection).

Another problem, encountered quite often, was that the first time I called read on the socket, only a partial response was read. To solve this, I implemented a retry mechanism which attempt to read for MAX_RETRIES times (3 by default) a complete response before giving up. I still don’t know what is causing this problem, if it is my code, the crate, the APIs, or else.

    pub fn ask_gpt(&mut self, prompt: &str) -> Result<String, OpenAiError> {
        self.history.add_user_message(prompt.to_owned());

        let (request_body, content_length) = self.history.to_string_with_content_length();

        let mut read_buf = Self::create_new_buf();
        let mut write_buf = Self::create_new_buf();

        let mut tls_socket = Self::open_tls_socket(&mut read_buf, &mut write_buf, self.remote)?;

        let request = format!(
            "POST {} HTTP/1.1\nHost: {}\nAuthorization: Bearer {}\nContent-Type: application/json\nContent-Length: {}\nUser-Agent: Sony PSP\n\n{}\n",
            POST_PATH,
            OPENAI_API_HOST,
            self.api_key,
            content_length,
            request_body,
        );
        let request_bytes = request.as_bytes();

        let mut response_string = String::new();

        for _ in 1..=MAX_RETRIES {
            tls_socket
                .write_all(request_bytes)
                .map_err(|e| log_error(&e))?;
            tls_socket.flush().map_err(|e| log_error(&e))?;

            let response_buf = &mut [0u8; 16_384];
            tls_socket.read(response_buf).map_err(|e| log_error(&e))?;

            let text = String::from_utf8_lossy(response_buf)
                .to_string().replace(['\r', '\0'], "");

            response_string += &text;

            let res_code = response_string
                .split('\n')
                .next()
                .ok_or(OpenAiError::UnparsableResponseCode)?
                .split(' ')
                .nth(1)
                .ok_or(OpenAiError::UnparsableResponseCode)?;
            if res_code != "200" {
                return Err(OpenAiError::ResponseCodeNotOk);
            }

            if response_string.ends_with("}\n") {
                break;
            }
        }

        // find for double newline, get the body
        let res_body =
            response_string
                .split("\n\n")
                .nth(1)
                .ok_or(OpenAiError::UnparsableResponseBody(
                    "Body not found".to_owned(),
                ))?;

        let completion_response: CompletionResponse = serde_json_core::from_str(res_body)
            .map_err(|e| OpenAiError::UnparsableResponseBody(e.to_string()))?
            .0;

        let assistant_message = completion_response.choices[0].message.content.trim();
        self.history
            .add_assistant_message(assistant_message.to_owned());

        Ok(assistant_message.to_owned())
    }

The checks if the response is complete or not are “empirical” and not very robust, but they seem to work fine in my (limited) testing.

Putting All Together

Finally, we have all the pieces, and we just need to put them together in the main, and run cargo psp --release.

For performance reasons, I was advised to always build in release mode.

To run it yourself, you have to create an OpenAI API key, and export it as an environment variable in the terminal you use to build the program (if you use VS Code as an IDE, I advise you to have it exported already in the terminal when you run it with code .).

export OPENAI_API_KEY=my-api-key

In the main, which in rust-psp is called psp_main (and you need to add the #![no_main] directive at the top of your main.rs) I need some boilerplate to set up the environment, and then I run a loop of asking the user to write a prompt, asking GPT, printing the results.

const OPENAI_API_KEY: &str = core::env!("OPENAI_API_KEY");

fn psp_main() {
    psp::enable_home_button();

    unsafe {
        // setup network
        net::utils::load_net_modules();
        psp::dprintln!("Initializing network...");
        net::utils::net_init();

        psp::sys::sceNetApctlConnect(1);
        loop {
            let mut state: psp::sys::ApctlState = core::mem::zeroed();
            psp::sys::sceNetApctlGetState(&mut state);
            if let psp::sys::ApctlState::GotIp = state {
                break;
            }
            psp::sys::sceKernelDelayThread(50_000);
        }

        // setup controls
        psp::sys::sceCtrlSetSamplingCycle(0);
        psp::sys::sceCtrlSetSamplingMode(psp::sys::CtrlMode::Analog);
    }

    psp::dprintln!("Connected to network!");

    let mut resolver = DnsResolver::default().expect("failed to create resolver");

    let openai_context =
        OpenAiContext::new(&mut resolver, OPENAI_API_KEY).expect("failed to create openai context");

    let mut input_handler = InputHandler::default();

    psp::dprintln!("Press X to start asking GPT-3.5, any other button to exit.");
    if !input_handler.choose_continue() {
        unsafe {
            psp::dprintln!("Exiting...");
            sceGuTerm();
            sceKernelExitGame();
        };
    }

    setup_gu();

    loop {
        let read_text = unsafe {
            sceKernelDcacheWritebackAll();

            let mut out_text: Vec<u16> = Vec::with_capacity(CHAT_MAX_LENGTH_USIZE);
            let out_capacity: i32 = out_text.capacity() as i32;

            let description = str_to_u16_mut_ptr("Ask GPT\0");
            let mut osk_data = default_osk_data(description, out_capacity, out_text.as_mut_ptr());

            let params = &mut default_osk_params(&mut osk_data);

            start_osk(params).expect("failed to start osk");

            read_from_osk(params).unwrap_or_default()
        };
        let read_text = read_text.replace('\0', "");

        psp::dprintln!("User: {}\n", read_text);

        let mut openai = OpenAi::new(&openai_context).expect("failed to create openai");

        let answer = openai.ask_gpt(read_text.as_str());

        if answer.is_err() {
            psp::dprintln!("failed to get answer from openai");
            psp::dprintln!("Got error: {:?}\n", answer.err().unwrap());
        } else {
            psp::dprintln!("GPT: {}\n", answer.unwrap());
        }

        psp::dprintln!("Press X to ask again, any other button to exit.");
        if !input_handler.choose_continue() {
            break;
        }
    }

    unsafe {
        sceGuTerm();
        sceKernelExitGame();
    };
}

Testing And Final Words

Testing

The PSP cannot connect to the new password-protected Wi-Fi. The simplest way to get it connected is to create a password-free hotspot with your phone, and connect to it.

Note, the program as is will try to connect to the saved connection in first position. Thus, beware of having it already set up correctly before you run it.

To test it on real hardware, copy the EBOOT.PBP in the GAME directory of your PSP.

I was not able to run the program successfully on the popular PPSSPP emulator, the OSK and networking in particular were giving me troubles.

Chat-GPSP in action. The text in the photo looks kinda blurry, but I wasn’t able to capture a better photo.

As you can see in the picture above, the UI is not very sophisticated, just a simple terminal-like one.

To see the complete code, you can have a look at the repo here [3].

Final Words

Completing this project wasn’t easy — partly because while I was working on it, I completed my master’s degree, was working full-time, and relocated to a different country, partly because the documentation on the topic is limited. I was lucky to find and join the PSP Homebrew Community on Discord [4], they were really helpful whenever I needed, and are a great community in general. I would not have made it without their support, so a big thanks goes to them.

References

1. CrossCraft OSK example — https://github.com/CrossCraft/CrossCraft-Classic/blob/ce932e73468c2868932c3136dfd590473b57a474/src/Player/PlayerInput.cpp#L629
2. TLS example — https://github.com/sajattack/rust-psp/blob/tls/examples/tls-test/src/main.rs
3. Chat-GPSP repo — https://github.com/lorenzofelletti/chat_gpsp
4. PSP Homebrew Community Discord — https://discord.gg/tJ8NXFzDfY
5. Rust PSP — https://github.com/overdrivenpotato/rust-psp
6. PSPSDK — https://pspdev.github.io/pspsdk/index.html