Creating your own Programming Language with Python.

Photo by Radowan Nakif Rehan on Unsplash

So you want to create your own Programming Language, but you don’t know where to start? Well here is how I created my own programming language called TEN for Father’s Day.

P.S — You can too!

To get started, you are going to need to make sure you have Python installed. You may also want to update it to Python 3.X. I would recommend 3.9 since it is the latest one as of this article.

To install Python, simply go to www.python.org, open the download section in the navbar, and click the Python 3.9.5 button:

Go to Python’s website (Screenshot outdated)

Run the installer and follow the steps that pop up.

Now we need to install some python packages which will allow us to convert our python files to exe files. The steps are taken from this Stack Overflow answer.

Open a new command prompt window, and type in the following command:

pip install cx_Freez

Then, type in this next command:

pip install idna

These 2 libraries are needed for converting .py files to .exe files.

Now that that is out of the way, we need to start planning out our custom language. For simplicity, we will be building a stack-based programming language.

A stack-based programming language is one that runs on a stack. First, you input some values, then an operation on those values. For example, to add 2 numbers together, you would type the following code:

number1 number2 +

number1 adds the first number to the stack, then number2 adds the second number to the stack. So, our stack now contains number1 and number2. Then, by typing +, we add the 2 numbers to the stack. So, the end result of this is the value of number1+number2 in the stack.

As the name suggests, the entire language runs on a stack, so we need to first define a variable containing our stack. To do so, import the deque (pronounced deck) package from the collections library:

from collections import deque
q = deque()

The next step is — Well there isn’t much you can do now, since you haven’t really thought about the syntax of your language. The real next step is to pull yourself away from Python, and open up a black text file. You need to start creating the syntax of your programming language.

Think about what you want it to look like. How will you perform basic operations on numbers? How will you add items to the stack? How would you define strings?

If you thought about it long enough, you should have at least 10 different commands, not including basic number arithmetic. Otherwise, you are basically just building a calculator.

Now that you have the syntax sort of thought out (and trust me, you haven’t fully thought it out yet, because you can bet the syntax will change over the period of time you make this language), we can begin our real work with python.

For clarity, let’s call the language we are going to create as CUSTOM.

We need to open up the file containing our CUSTOM script. But, before we open the file, we need the file name. If you have ever compiled a python script, you know the command you type is python <filename>.py. What we are actually doing is passing in the string <filename.py> to python.exe, and then python.exe is compiling that file. Python actually allows us to compile python files with additional arguments passed in: python <filename>.py arg1 arg2 …. arg1, arg2, … are then passed into our python script, which can use those arguments. To access those arguments, we use sys.argv. sys.argv is a list containing each additional argument. For the time being, we will just have 1 argument passed in, the file name of the CUSTOM script:

from collections import deque
filename = sys.argv[1]
q = deque()

Then, we read all the contents of the file to a variable called contents.

import sys
from collections import deque
filename = sys.argv[1]
file = open(filename, 'r')
contents = file.read() # Reads the entire file as a single string
q = deque()

I am assuming that your programming language will not have variables, as variable access can be a real pain. You can check out my TEN Github Page (linked above) to see how I managed variable assignment and access.

Now, pretty much all of our variables are done. Now we need to iterate character by character over the code. I made each of my commands a single character so that I could iterate character by character. You could use a string tokenizer, but that’s really not needed. (Plus, making an obfuscated programming language is all the rage)

First, the code which iterates over each character:

i = 0
while (i < len(contents):
    # Some code here

Let’s start simple, with checking if the user is defining a string. Suppose the user opens the string with “, and closes with ‘. Then we would write the following:

if (contents[i] == "\""):
    j = i
    while (contents[j] != "\'"):
        j += 1
    q.appendleft(contents[i+1:j])
    i = j

So just some basic things like that make the code much better!

Here is a sample, which only allows users to add strings and print them. Using the same string definition as above, and letting ; be the command to print the top of the stack:

import sys
from collections import deque
filename = sys.argv[1]
file = open(filename, "r")
contents = file.read()
i = 0
while i < len(contents):
    if (contents[i] == "\""):
        j = i
        while (contents[j] != "\'"):
            j += 1
        q.appendleft(contents[i+1:j])
        i = j
    elif (contents[i] == ";"):
        print(q[0])
    i+=1

And thats it! The script is done! Now, we just need to turn it into an EXE file. To do that, create another python file, and call it setup.py. Inside the file, paste the following code:

from cx_Freeze import setup, Executable
base = None
executables = [Executable("CUSTOM.py", base=base)]
packages = ["idna"]
options = {
    'build_exe': {
        'packages':packages,
    },
}
setup(
    name = "CUSTOM",
    options = options,
    version = "0.2",
    description = '',
    executables = executables
)

Don’t forget to replace CUSTOM with the name of your python file. Just run setup.py like so:

python setup.py build

And that is IT! The file will be saved inside the build folder. Make sure the other folder (lib) and python3.dll, python3X.dll (X is your version) are all in the same directory as the actual exe file.

Now, create a text file with the following:

"Hello World!';

Run it with the command:

CUSTOM sample.txt

And, “Hello World!” should be outputted! Feel free to add more to the script as you wish. You can see a sample of my work in my GitHub, as mentioned earlier.