Introduction
Designing the Scripting Language
Lexical Analysis (Tokenization)
Conclusion

Creating a custom scripting language for your game engine can give you flexibility and control over your game's behavior without recompiling the entire engine. This guide will walk you through the steps to design, parse, and execute a simple scripting language in C++.

Introduction

In this guide, we'll create a basic scripting language for our game engine. The scripting language will support simple operations like variable assignment, arithmetic operations, and printing values to the console.

If you're unfamilier with a game engine, read creating a game engine with c++ and OpenGL.

Designing the Scripting Language

First, we need to define the syntax and features of our scripting language. Let's keep it simple:

Variables: var x = 10
Arithmetic: x = x + 5
Print: print(x)

Lexical Analysis (Tokenization)

We'll start by breaking the script into tokens. Each token represents a keyword, identifier, operator, or literal value.

Tokenizer Implementation

Create a new file tokenizer.h:

#ifndef TOKENIZER_H
#define TOKENIZER_H
 
#include <string>
#include <vector>
 
enum class TokenType {
    VAR,
    IDENTIFIER,
    NUMBER,
    ASSIGN,
    PLUS,
    PRINT,
    END,
    UNKNOWN
};
 
struct Token {
    TokenType type;
    std::string value;
};
 
std::vector<Token> tokenize(const std::string& code);
 
#endif // TOKENIZER_H

Now, implement the tokenizer in tokenizer.cpp:

#include "tokenizer.h"
#include <cctype>
#include <sstream>
 
std::vector<Token> tokenize(const std::string& code) {
    std::vector<Token> tokens;
    std::istringstream stream(code);
    std::string word;
 
    while (stream >> word) {
        if (word == "var") {
            tokens.push_back({ TokenType::VAR, word });
        } else if (word == "print") {
            tokens.push_back({ TokenType::PRINT, word });
        } else if (word == "=") {
            tokens.push_back({ TokenType::ASSIGN, word });
        } else if (word == "+") {
            tokens.push_back({ TokenType::PLUS, word });
        } else if (std::isdigit(word[0])) {
            tokens.push_back({ TokenType::NUMBER, word });
        } else {
            tokens.push_back({ TokenType::IDENTIFIER, word });
        }
    }
 
    tokens.push_back({ TokenType::END, "" });
    return tokens;
}

Parsing the Script

Next, we'll parse the tokens into an abstract syntax tree (AST).

Parser Implementation
Create a new file parser.h:

#ifndef PARSER_H
#define PARSER_H
 
#include "tokenizer.h"
#include <memory>
#include <unordered_map>
 
class ASTNode {
public:
    virtual ~ASTNode() = default;
    virtual void execute(std::unordered_map<std::string, int>& variables) const = 0;
};
 
using ASTNodePtr = std::unique_ptr<ASTNode>;
 
ASTNodePtr parse(const std::vector<Token>& tokens);
 
#endif // PARSER_H

Implement the parser in parser.cpp:

#include "parser.h"
#include <stdexcept>
 
class VarNode : public ASTNode {
    std::string name;
    int value;
public:
    VarNode(const std::string& name, int value) : name(name), value(value) {}
    void execute(std::unordered_map<std::string, int>& variables) const override {
        variables[name] = value;
    }
};
 
class PrintNode : public ASTNode {
    std::string name;
public:
    PrintNode(const std::string& name) : name(name) {}
    void execute(std::unordered_map<std::string, int>& variables) const override {
        std::cout << variables[name] << std::endl;
    }
};
 
class AssignNode : public ASTNode {
    std::string name;
    int value;
public:
    AssignNode(const std::string& name, int value) : name(name), value(value) {}
    void execute(std::unordered_map<std::string, int>& variables) const override {
        variables[name] = value;
    }
};
 
ASTNodePtr parse(const std::vector<Token>& tokens) {
    auto it = tokens.begin();
    if (it->type == TokenType::VAR) {
        ++it;
        std::string name = it->value;
        ++it;
        ++it; // Skip '='
        int value = std::stoi(it->value);
        return std::make_unique<VarNode>(name, value);
    } else if (it->type == TokenType::PRINT) {
        ++it;
        return std::make_unique<PrintNode>(it->value);
    } else if (it->type == TokenType::IDENTIFIER) {
        std::string name = it->value;
        ++it;
        ++it; // Skip '='
        int value = std::stoi(it->value);
        return std::make_unique<AssignNode>(name, value);
    }
    throw std::runtime_error("Unknown statement");
}

Executing the Script

Finally, we'll execute the AST nodes.

** Main Implementation **
Create a new file main.cpp:

#include "tokenizer.h"
#include "parser.h"
#include <iostream>
#include <unordered_map>
 
int main() {
    std::string code = "var x = 10\nprint(x)\nx = x + 5\nprint(x)";
    
    auto tokens = tokenize(code);
    std::unordered_map<std::string, int> variables;
    
    for (const auto& token : tokens) {
        if (token.type == TokenType::END) break;
        auto node = parse(tokens);
        node->execute(variables);
    }
    
    return 0;
}

Putting It All Together

To compile and run your script, use a Makefile:

all: main
 
main: main.o tokenizer.o parser.o
    g++ -o main main.o tokenizer.o parser.o
 
main.o: main.cpp tokenizer.h parser.h
    g++ -c main.cpp
 
tokenizer.o: tokenizer.cpp tokenizer.h
    g++ -c tokenizer.cpp
 
parser.o: parser.cpp parser.h tokenizer.h
    g++ -c parser.cpp
 
clean:
    rm -f *.o main

Run the following commands to build and execute the program:

make
./main

Conclusion

In this guide, we developed a simple scripting language for a game engine in C++. While the language and interpreter are basic, they serve as a foundation for more complex features. You can extend this project by adding more operations, control structures (like loops and conditionals), and improving error handling.

Happy coding!

Developing a Custom Scripting Language for Your Game Engine in C++

Table of contents