Use Any Language to Control mGBA

I made mGBA-http, a cross-platform HTTP wrapper around mGBA I wrote. If you can send an HTTP request, you can control mGBA.

GitHub - nikouu/mGBA-http: An HTTP interface for mGBA scripting. Connect to mGBA with your favourite language using an HTTP REST API! Easily create your own cross platform “Twitch Plays”, Gameboy playing bot, or more.

An HTTP interface for mGBA scripting. Connect to mGBA with your favourite language using an HTTP REST API! Easily create your own cross platform "Twitch Plays", Gameboy playing bot, or mo…

GitHubnikouu

How to use

In short:

1. Ensure you have mGBA
2. Download mGBA-http and mGBASocketServer.lua from the Releases section
3. Run mGBA-http
4. In mGBA, go to Tools > Scripting, then File > Load script and load in mGBASocketServer.lua

Once a ROM is loaded, you are now ready to start using mGBA-http.

For a more in-depth guide with pictures, see the Full Guide.

Once running you quickly begin prototyping as it comes with SwaggerUI out of the box. Meaning you can use your browser to send requests to mGBA. When running mGBA-http, the default endpoint for this is http://localhost:5000/index.html.

And since it's Swagger, you can grab the swagger.json spec file and import it into Postman or your request software of choice. The default for this is http://localhost:5000/swagger/v1/swagger.json.

This is also available on GitHub as the raw swagger.json or as a readable markdown file via API documentation.

Once you've finished prototyping, pick anything that can send an HTTP message, and you're good to go. If you need some examples, check out below.

Example usage

Let's check out a couple of examples taken from the repository examples page where you'll find examples using SwaggerUI, Postman or languages such as C#, JavaScript (Node.js), Python, and uh, PowerShell.

Moving and interacting

If you want a running C# code example, the test console project, mGBAHttpServer.TestClient, that's in the mGBA-http solution. In this example we're able to move the main character around the overworld and interact with Pikachu in Pokémon Yellow. This is the same video from the top of the post.

Reading what nature a wild Pokémon has

Knowing what's ahead, or what secret things the game doesn't explicitly tell you is powerful. In this instance, we're using the in-built SwaggerUI functionality of mGBA-http to determine what nature the opposing wild Pokémon has in Pokémon Sapphire.

To get the nature of a wild Pokémon in Sapphire from memory:

1. Read the first dword (32 bits) starting from address 0x030045C0. In this case we get decimal: 3121544013
2. Then do dword % 25. Here we get 3121544013 % 25 = 13
3. See where the number corresponds on the list of natures. Here, 13 corresponds to Jolly

We can validate this by checking the nature of the Pokémon just caught. In this case, the Wurmple is Jolly in nature.

Loading a savestate

Automating loading a savestate is easy. Perhaps your bot needs to reset the game to a known state. The following example is using Node.js to load a save file for Oracle of Seasons.

const http = require('http');

// url encoded path of: C:\mgba-http\ZELDA DIN.ss1
const options = {
  hostname: 'localhost',
  port: 5000,
  path: '/core/loadstatefile?path=C%3A%5Cmgba-http%5CZELDA%20DIN.ss1&flags=2',
  method: 'POST'
};

const req = http.request(options, response => {
  response.on('data', chunk => {
    console.log(chunk);
  });
});

req.on('error', error => {
  console.error(error);
});

req.end();

What can I create?

Originally the goal was to make it as easy as possible to create a "Twitch Plays X" style of program. However, you can also:

• Write an AI bot. Send movement commands through http-mGBA. Perhaps for vision, use the screenshot API and feed that into your bot after each movement step. This is similar to how this great video, Training AI to Play Pokémon with Reinforcement Learning by Peter Whidden, does it.
• Creating a dashboard about your current game state. You have access to the memory read APIs meaning you can grab any piece of information you need in order to present it. Check out Pokélink and Ironmon-Tracker.
• Manipulating your current game. With the memory write APIs you can change inventory quantities, stat values, remove limitations, walk through walls, etc. Think along the lines of GameShark or Action Replay.

You aren't limited by just the above, go wild with your imagination.

Limitations

Watch out for these when creating. mGBA-http will suit most use cases, but not all.

No frame perfect calls

There is network latency between your application to mGBA-http and again latency between mGBA-http and mGBA. This will not be accurate for frame perfect manipulation and is meant for more general usage such as for "Twitch plays", AI playing bot, or other non frame specific application. For high accuracy manipulation see Bizhawk which is used for TASBots.

Or write purely in Lua against the mGBA scripting API.

Not all scripting calls implemented

Most, but not all calls are implemented. Though mostly this is around calls that return complex objects which would be difficult to send back to mGBA-http. However, a vast majority of the calls are implemented.

Sending requests quickly

Fixed in version 0.2.0.

As of the original mGBA-http release, when sending many requests per second, the requests may queue and be actioned long after the input requests stop. Or in other words, no key inputs are eaten.

For example, holding down on the d-pad with original hardware didn't trigger the down action long after d-pad release.

You may want to write a rate limiter in your code to prevent a large backlog of requests from backing up when calling mGBA-http.

Cross platform-ish

While I've built the binaries for OSX and Linux, I don't have the hardware capable of testing the binaries.

Technical notes

If you've made it this far, you might be interested in more than just the setup. Let's talk about some of the programming designs behind the first release of version 0.1.0. Many points below are outlined in the design document on GitHub.

Below outlines essentially "any language that does HTTP can talk to mGBA via mGBA-http":

The C# side

The C# code itself is designed to be simple enough for a regular C# developer or an experienced developer from another language. It really just uses two layers:

1. .NET minimal API with OpenAPI
2. The socket connection to mGBA

Each endpoint is simply mapped to a file that corresponds to the mGBA scripting API endpoints. We can take an example with the getGameTitle endpoint:

group.MapGet("/getgametitle", async (SocketService socket) =>
{
    return await socket.SendMessageAsync(new MessageModel("core.getGameTitle"));
}).WithOpenApi(o =>
{
    o.Summary = "Get internal title of the game.";
    o.Description = "Get internal title of the game from the ROM header.";
    return o;
});

I miss how clean it looked before I added WithOpenApi() but it really helps prototyping. Continuing with how I want to keep this project simple, I decided not to abstract it away because I want to look at an endpoint and say "I know exactly what this is doing".

Breaking down the code:

1. group.MapGet("/getgametitle", async (SocketService socket) =>
   1. The endpoints are split up into one group per file, which is the group object. I.e. all the core APIs are in CoreAPI.cs and are in the /core group.
   2. With MapGet() we say this is an HTTP GET call.
   3. "/getgametitle" is the path, which gets added onto the group. Meaning we'll end up with /core/getgametitle.
   4. The final part is essentially the function that runs (here as a delegate), with SocketService being injected as an input parameter. We use SocketService to send and receive socket messages. SocketService is our code in the repo, and it's registered for dependency injection in program.cs as builder.Services.AddSingleton<SocketService>();
2. return await socket.SendMessageAsync(new MessageModel("core.getGameTitle"));
   1. We return the value from mGBA straight back to the caller by returning the value of SendMessageAsync()
   2. SendMessageAsync() takes in a MessageModel. This is essentially a little DTO that takes in the call identifier (core.getGameTitle) and any other parameters the call might need - in this case, no other parameters are needed.
3. WithOpenApi() adds the annotations you see in Swagger.

Mentioned previously, Program.cs sets up and runs the code. I want to talk about a not-as-nice code piece for logging filtering. My goal is that the user does not need to have appsettings.json in order to keep mGBA-http a single, clean, single binary. Meaning I wanted to set some logging defaults which could be overwritten if the appsettings.json is present in the same directory, however it ended up being an ugly chunk of code to set up defaults:

// Allow log filtering if the configs aren't present with prod exe
if (!builder.Environment.IsDevelopment() && !loggingSection.Exists())
{
    builder.Logging.AddFilter((provider, category, logLevel) =>
    {
        if (category.Contains("Microsoft.AspNetCore.Hosting.Diagnostics") && logLevel <= LogLevel.Error)
        {
            return false;
        }
        else if (category.Contains("Microsoft.AspNetCore.Mvc.Infrastructure.DefaultActionDescriptorCollectionProvider")
            && logLevel <= LogLevel.Error)
        {
            return false;
        }
        else if (category.Contains("Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware")
            && logLevel <= LogLevel.Error)
        {
            return false;
        }
        else if (logLevel >= LogLevel.Information)
        {
            return true;
        }
        else
        {
            return false;
        }
    });
}

I wouldn't be surprised if I come back to this in a later version to clean up.

More notes to the C# side can be found in the references documentation.

The Lua side

My first actual Lua project. A lot of code is from (and attributed in one way or another) the mGBA repo or from the mGBA Discord. It's mostly two pieces:

1. Socket management.
2. A big if/else for each of the call types. Which I understand doesn't follow the usual Lua programming style (which switch statements end up as a lookup table) however, with not being experienced at all, this helps me read it for now.

Why?

I had a few goals when getting into making mGBA-http:

1. Have fun
2. Allow any language to interact with mGBA
3. Be as equally cross-platform as mGBA
4. Create my most professional repository to date
5. Didn't know Lua and wanted to use C# as much as I could

Have fun

"Have fun" is the biggest factor. I've written about why I write this blog and having fun little personal projects. This is no exception. If it's useful to 100 or zero people, I still had fun.

It's as if I were a salaried artist: I get paid for doing large art pieces or murals (corporate software work) but I really enjoy drawing silly little dogs farting (personal projects). 🐕💨

I was heavily inspired by the original Twitch Plays Pokémon. Having software that I wrote to enable me to do something fun is a great feeling. I look forward to making something like that in the future.

Allow any language to interact with mGBA

On the back of having fun, I enjoy C#. It's a language I know pretty well and can easily manoeuvre in, so of course I'd like to make a Twitch Plays Pokémon clone in C#.

After having done some investigation, it turns out a lot of emulator controlling software uses the Windows SendKey API (or similar abstraction) whether it's C#, Java, Python, etc. Not helpful if there has to be some nasty Windows COM work to shill in for other languages just to send controlling commands to an emulator.

However, HTTP requests are pretty universal and mGBA has a socket API. So writing a thin wrapper around the socket API would allow any language to interact with mGBA as HTTP is easier to work with compared to the raw underlying socket.

Be as equally cross-platform as mGBA

mGBA is cross-platform and the Windows SendKey event isn't. Good news, HTTP is cross-platform. Good(er) news, .NET is also cross-platform. Perfect.

In addition to this, not everyone has .NET installed on their machines. However, with the great work with AOT (Ahead of Time Compilation) in .NET (that I've written about too) we can bring all the necessary pieces of .NET with the binary - reducing the barrier of using mGBA-http by not forcing users to install .NET. This is why in the releases you'll see:

1. mGBA-http-0.1.0-win-x64.exe
2. mGBA-http-0.1.0-win-x64-self-contained.exe

With the first being just the code and assuming you have .NET installed to run against and thus a small file size (a couple of megabytes). Then the latter being larger as it brings all the .NET pieces it needs with it, so the user does not need to do a separate .NET runtime install.

Create my most professional repository to date

While most of my repos are silly one offs like procedural-pokemon-buildings, I have tried making nicer repositories before:

1. Adafruit-Feather-CO2-Meter
2. TinyWordle
3. GameboyColour-Decolouriser

And I see mGBA-http as the next step of what makes a repo great to read and use. Lessons learned from previous attempts are around better:

1. Documentation
2. Build and releases
3. Versioning
4. Repo layout

Which I feel I've achieved compared to my previous attempts.

Didn't know Lua and wanted to use C# as much as I could

This one is a little ironic because I had to write Lua in order to use C#. It do be like that. But now I can write a C# program to interact with mGBA instead.

To Conclude

If you made it this far, thank you for following my little journey for my fun little project. I hope you find it useful, or at least inspirational. I look forward to moving beyond version 0.1.0 and adding some new features and cleaning up some older ones that I wanted to add. But hey, you gotta release at some point!

Links

• mGBA-http
• mGBA Website
• mGBA GitHub
• mGBA Discord