379 replies to this topic

[Medo42]

What's this about?
The Faucet Networking extension aims to provide a sane low-level networking API for multiplayer games. You could probably summarize it as "a better 39dll". Not in the sense that it can do more (not at first anyway,) but in the sense that it's easier to use and easier to learn.

[Rebound is] my first foray into online programming. I am making the game in Game Maker using the Faucet Networking Extension by the guys that made Gang Garrison. It has been wonderful to use and has been really fun. -- Sean A.

What's cool about it?

• Easy to learn and use
• Completely nonblocking
• IPv6 support
• Detailed documentation of all functions
• Support for little-endian as well as big-endian byte order
• Very permissive license

Current Status
The extension now supports TCP and UDP in the latest version.

If you need a feature for your project that this extension doesn't provide, feel free to ask and I'll try to figure out the most sensible way to make it possible. Keep in mind though that the focus of this extension is on low-level networking, so please do not ask for things like functions for HTTP or IRC.

Most recent release
You can download the most recent release here (V1.4.2, released on 2012-11-14).
The most recent source code is always available in the repository at Github.
You might find more up-to-date information at the ganggarrison.com forum thread

Did this extension help you and you want to give something in return? If yes, you can Flattr me.

Tutorials
TheCrazyGameMaker has created a screencast that walks you through creating a small TCP networking game with Faucet Net.

The main download also comes with two example programs, and a help file which explains every function of the extension in detail.

What's wrong with 39dll?
(Alternative title: A combined 39dll rant and short history of Faucet Networking)

For many years, 39dll has been the most popular networking library for Game Maker by quite a comfortable margin, and it has enabled many projects to implement their networking code. However, that does not mean it is a good library. It isn't. Maybe 39dll became popular because it was the only real option for a while, but that is not true anymore today. Using a more modern library like Faucet Networking will spare you some unnecessary headaches. But you don't have to take my word for that:

I don't know what Faucenet is (my guess, a new and much better networking lib) but I definitely recommend using it over [39dll], which I wrote when I literally just started programming in c++ />.
I'm always surprised to see that it is still popular, as it is very poorly coded.

So what is the actual problem with 39dll, and why did I start writing Faucet Networking? To better appreciate the motivation behind this project, you should know that I have used 39dll in Gang Garrison 2 for a long time, and finally grew tired of the hoops I had to jump through to perform the most basic tasks. In many cases, a task would seem easy at first, but then in some corner cases the easy solution would make my game freeze or misbehave, so I had to add code to deal with those corner cases. Suddenly, the solution wasn't so easy anymore, and I had spent hours on tracking down unexpected networking problems that could have been spent on inproving the gameplay.

A simple example is sending some bytes of data from a game server to a client, over an existing TCP connection. You will (and should!) probably expect that this is easy to do in any networking library. Naturally, you want all the bytes to eventually arrive at the client (unless the connection breaks). But you don't want to wait until the data is sent - Your game server needs to run (e.g.) 30 steps per second, and there is no way to tell how long the sending will take. Usually it is immediate, but it can take seconds in some cases, and you don't want your server to freeze for that long (or at all).

However, to do this properly with 39dll, you need to manually store the data that can't be sent immediately, and try resending it every step. You also need to handle specific return codes of 39dll functions which are completely undocumented, so you basically need to read the 39dll source code and cross-reference with the Winsock reference. All this adds up to a rather large block of GML code you have to write.

Faucet Networking on the other hand just accepts all your bytes without blocking and sends them in the background, so your GML code can do more interesting things. If you want details on this example, read on here - I didn't want to clobber the first post with too many details.

Receiving data has a similar problem. In the usual case where you know exactly how many bytes you need next, you have to take precautions for the case that some data is available, but not as much as you asked for.

Gang Garrison had a couple of scripts to provide a more convenient way of doing common tasks, but not every problem could be hidden this way, and some things - like the ever more important IPv6 support - obviously can't be done in GML alone. So I decided to take a deeper look under the hood of 39dll and add some tweaks. That was the initial plan at least. Working through the 39dll source code though, I realized there were more problems than just a clunky interface. 39dll has bugs. It leaks memory every time you delete a buffer or socket. It has hidden limits: if you try to read more than 20000 characters with readchars, there's an internal buffer overflow. It relies on undefined compiler behaviour: if you recompile it with gcc, simply calling writebyte(1000, buffer) will crash your game.

I realized that instead of trying to fix the problems, it would be more profitable to develop something new, starting with the question what a game programmer actually needs, based on my own experience with the development of Gang Garrison 2. The focus was to make the API as easy to use as possible, even if that meant the library would be more complicated to develop. I tried designing the API functions in a way that made them intuitive, and robust against errors and wrong usage.

When I had developed the extension far enough that it was able to replace 39dll for Gang Garrison, I did exactly that, and it was satisfying to see how much it simplified the code of the game in some places. However, I found that it still required too much work to do some common tasks, so I went back to the drawing board and revised several decisions to iron out the creases.

The result is a library that is easy to learn and use, and that gets common tasks done with a minimum of code and effort. I hope you will find it useful.

Edited by Medo42, 16 February 2013 - 04:57 PM.

[Primoz128]

Umm... any examples to show it's power and reliability and that it actually works... ?

[Medo42]

Umm... any examples to show it's power and reliability and that it actually works... ?

A Pong example game is included in the download. Also, the current development version of Gang Garrison 2 uses this library and it does work (the release version you can download on the homepage still uses 39dll). As for reliability, I don't know of any bugs, but it has not been extensively tested yet. That's why I am still calling this an alpha version. If you come across a bug I will be happy to fix it though.

[Schyler]

TCP in 39dll, you have two options: You can wait until all data is accepted (blocking mode), but this will essentially freeze your game for as long as it takes. Or you can write as much data as possible without blocking, but then you have to take precautions for the case that not everything is written.

Why don't you put some example code in the topic showing why yours doesn't do that.

The only way I can think of is that you are using GMAPI, and if you are, you didn't include credit.

[Medo42]

I wasn't aware of GMAPI, that's an interesting project. No, I don't use that at the moment.

So, how does Faucet Networking avoid the problem? I did put it in the original post:

In Faucet Networking, writing to a socket always accepts all the data and returns immediately.

This is achieved by adding everything to an internal buffer and constantly sending the data from that buffer in the background.

Blocking send with 39dll
So let's compare the code needed. The reason why I didn't do that in the first post is that it's rather long and complicated. Using a blocking socket with 39dll is simple, and the only real complaint with it is that it can freeze your game, which makes it useless in server code of a realtime game:

if(sendmessage(socket, 0, 0, buffer) < 0) {
    // handle send error
}

Nonblocking send with 39dll
Here's the nonblocking 39dll example. This is actually taken directly from the old Gang Garrison 2 source. It is a script which is called on all send buffers in every step:

// This function attempts to send a buffer.
// All bytes successfully written will be cleared from the buffer.
// If the buffer is sent completely, it returns 0
// If the buffer is sent partially, it returns 1
// If the socket had to be closed due to an error or loss of connection, it returns 2

// argument 0: Socket
// argument 1: Buffer

var size;

setsync(argument0, 1);

size = sendmessage(argument0, 0, 0, argument1);
if(size == buffsize(argument1)) {
    clearbuffer(argument1);
    return 0;
} else if (size < 0) {
    switch(size) {
        case WSAENOBUFS:
        case WSAEWOULDBLOCK:
            if(buffsize(argument1)>100000) {
                closesocket(argument0);
                return 2;
            } else {
                return 1;   
            }
            
        default:
            closesocket(argument0);
            return 2;
    }
} else if(size < buffsize(argument1)) {
    if(buffsize(argument1)-size>100000) {
        closesocket(argument0);
        return 2;
    } else {
        clearbuffer(global.tempBuffer);
        copybuffer2(global.tempBuffer, size, buffsize(argument1)-size, argument1);
        clearbuffer(argument1);
        copybuffer(argument1, global.tempBuffer);
        return 1;
    }
}

Do you think this is excessive? Well, let me explain why it does what it does.
When the socket is in nonblocking mode, sendmessage might not send everything. If it does (first if-block), everything is fine and we're done.
If the operating system has no buffer space available at all, it might not send anything and return an error. But that specific error (WSAEWOULDBLOCK) does not mean the connection is broken, just that we can't send anything right now, so we have to check the actual error code and act depending on that (switch-block).
What can also happen is that sendmessage just sends less than we asked for. In that case you have to remove the sent bytes from the buffer, so that you can send the rest next time. However, 39dll doesn't offer a function to cut bytes from the start of a buffer, so you have to use a temp buffer and copy the remaining data back and forth (last else if-block).
In some places there are checks to prevent the buffer from growing too large (e.g. if(buffsize(argument1)>100000)), which would happen on bad connections where the server generates data faster than it is transmitted.
I won't go into detail on what you'd have to do to use message format 0 or 1 correctly with a nonblocking socket, but basically you'd have to remember that a message was interrupted midway and send the rest in message format 2 to avoid a new header from being inserted by 39dll.

Nonblocking send with Faucet Networking
Complicated enough? Now here's the nonblocking code with Faucet Networking:

write_buffer(socket, buffer);
socket_send(socket);

To check if there was an error you can add the following, but you'll usually only check a socket for errors once per step:

if(socket_has_error(socket)) {
    // handle error, e.g. disconnect player
}

If you want to add a limit to the internal buffer to prevent slow connections from hogging your memory (say, 100kb as in the nonblocking 39dll example above):

socket_sendbuffer_limit(socket, 100000)

The socket will automatically cut the connection if the internal send buffer grows larger than that limit.

Edited by Medo42, 18 February 2011 - 08:46 PM.

[Medo42]

V1.0b1 has been released.

Download: here

Changes:

• Bugfix: Acceptors were not being destroyed properly
• Bugfix: Avoided potential conflict with dual stack sockets under Windows Vista

As far as I can tell, this is relatively stable now, so I'm calling this the first beta version. Keep in mind though that it didn't get much real-world testing yet, that's your part

Edited by Medo42, 01 March 2011 - 07:55 PM.

[nutsaq]

I've started using this and I like it. I have not run into any bugs yet. Looking forward to full UDP functionality!

[SimplySeth]

This looks pretty impressive. Downloading now

[Medo42]

V1.0b2 is out now.
Download here

Changes:
- Fixed hang on shutdown when there was still a running operation (e.g. a connection attempt)

Edited by Medo42, 23 March 2011 - 11:32 AM.

[Medo42]

V1.0 is out now.
Download here

Changes:
- Added a function socket_remote_ip to find out which IP a socket is connected to
- Changed the name to Faucet Networking (from Faucet Networking Extension - it was a bit redundand)

Because of the name change, you will have to install the new version first, then change your game to use the new version and finally uninstall the old one.

[tie372]

Looks good. Will use it once UDP is added in. GG2 is all TCP based?

[Medo42]

Looks good. Will use it once UDP is added in. GG2 is all TCP based?

Yes, it's a somewhat unconventional choice, but when we started out I did not have experience with network programming and we had a deadline to work towards. Switching to UDP now would mean rewriting rather large parts of the game, and nobody wants to invest the time right now. TCP is a workable solution as you can see in GG2, and most of the latency felt in the game is not because of that choice of protocol, but rather because we don't do any input prediction. That said, for the best results with realtime games you should use something UDP based to avoid the latency spike that will happen when a TCP packet is lost.

UDP will make it into the extension, but unfortunately have other priorities at the moment. I hope get around to by the end of May.

[sabriath]

I am using this post to tag your topic....think of it as a free bump

Any plans you wish to share with your "rival"? teehee

[Medo42]

I am using this post to tag your topic....think of it as a free bump

Any plans you wish to share with your "rival"? teehee

Yes: I plan to stick with having this extension do one thing, and do it well. In fact, if the "shared buffers" project takes off I'll even remove my own buffer code.

Congratulations on your 3000th post, which might appear below

[tie372]

Played around with this some more and it's basically amazing.

What's the approach on sending multiple variable length strings in one message? Custom delimiters (read one character at a time via read_string() until you reach the delimiter)? Some sort of size heading for each string?

[Medo42]

Played around with this some more and it's basically amazing.

What's the approach on sending multiple variable length strings in one message? Custom delimiters (read one character at a time via read_string() until you reach the delimiter)? Some sort of size heading for each string?

At the moment it's far easier to send a size header in advance, but I plan to add functions that you can use to receive / read data delimited by any string.

[scorpeti]

I started to use your extension's newest version, but I ran into a problem. In the first game window, I created an acceptor (acceptor=1) as a server, then I connected itself as a client (socket_server=2) and accepted itself as a server (socket[0]=3), everything's perfect. After that I opened an other game window. In this new game I connected to the existing server (socket_server=1) and in the first game, the server accepted this one and saved it's socket (socket[1]=4). So I have two game windows with one server and two clients. After that I wanted to disconnect a client by clicking close button on the second game window, so I did it, but unfortunately the server didn't notice anything. If I close the first game window like this and shut down the server, the client should notice this, but nothing happened. In every game, I'm checking the sockets in begin step event by socket_has_error() function, but it says no error: returns 0. If I try to connect to a non existing server, this error code works perfectly. It's probably my mistake, but I cannot find out what's wrong. I ran the games in debug mode many times and just this socket_has_error code goes wrong. If I close the socket by socket_destroy function, same problem appears.

[Medo42]

I started to use your extension's newest version, but I ran into a problem. In the first game window, I created an acceptor (acceptor=1) as a server, then I connected itself as a client (socket_server=2) and accepted itself as a server (socket[0]=3), everything's perfect. After that I opened an other game window. In this new game I connected to the existing server (socket_server=1) and in the first game, the server accepted this one and saved it's socket (socket[1]=4). So I have two game windows with one server and two clients. After that I wanted to disconnect a client by clicking close button on the second game window, so I did it, but unfortunately the server didn't notice anything. If I close the first game window like this and shut down the server, the client should notice this, but nothing happened. In every game, I'm checking the sockets in begin step event by socket_has_error() function, but it says no error: returns 0. If I try to connect to a non existing server, this error code works perfectly. It's probably my mistake, but I cannot find out what's wrong. I ran the games in debug mode many times and just this socket_has_error code goes wrong. If I close the socket by socket_destroy function, same problem appears.

There are several reasons why the sockets might not show an error. The first one is simple: socket_has_error is implemented in a passive way. At the moment, a socket will only show an error if some command (e.g. sending or receiving data) actually failed, so if you don't do anything except check for errors, you will never see that the connection is no longer working. This behavior is consistent with the documentation but a bit unintuitive, so I could change the function to actively check if the socket is still good.

Even with that change you might not get an error though, at least for a while, because there simply is no error if the other side closed the connection cleanly - in fact, in that case the connection is still half-open until you either close the other side as well or some timeout occurs. Disconnecting is a surprisingly complicated subject in TCP

You can find out if the connection was closed with the help of tcp_eof() though - it will return true in that case, unless there is still data left to be read. Another solution is to actually use the sockets for something: If you try reading data from the connection with tcp_receive() after the connection is closed, you will get an error the way you expected.

[scorpeti]

Thank you very much. The problem was your first guess: I didn't do anything but checking for errors. Now it's solved, extension works fine.

[thaddeus_maximus]

I've been a user of 39dll for a long time. However, I can't wait to try this out. I wish you the best of luck in your development.