Jump to content


Photo

Faucet Networking


  • Please log in to reply
651 replies to this topic

#1 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 02 February 2011 - 05:09 PM

What's this about?
The Faucet Networking extension aims to provide a sane low-level networking API for multiplayer games. You could probably summarize it as "a better 39dll". Not in the sense that it can do more (not at first anyway,) but in the sense that it's easier to use and easier to learn.
 

 

[Rebound is] my first foray into online programming. I am making the game in Game Maker using the Faucet Networking Extension by the guys that made Gang Garrison. It has been wonderful to use and has been really fun. -- Sean A.


What's cool about it?

  • Easy to learn and use
  • Completely nonblocking
  • IPv6 support
  • Support for UDP broadcasting (local server discovery)
  • Detailed documentation of all functions
  • Support for little-endian as well as big-endian byte order
  • Very permissive license

Current Status
The extension now supports TCP and UDP in the latest version.

If you need a feature for your project that this extension doesn't provide, feel free to ask and I'll try to figure out the most sensible way to make it possible. Keep in mind though that the focus of this extension is on low-level networking, so please do not ask for things like functions for HTTP or IRC.

Most recent release
You can always download the most recent release at Github (currently V1.6.0).

The most recent source code is also available there: Github repository.
You might find more up-to-date information at the ganggarrison.com forum thread

Did this extension help you and you want to give something in return? If yes, you can Flattr me.

Tutorials
TheCrazyGameMaker has created a screencast that walks you through creating a small TCP networking game with Faucet Net.

The main download also comes with two example programs, and a help file which explains every function of the extension in detail.

What's wrong with 39dll?
(Alternative title: A combined 39dll rant and short history of Faucet Networking)

For many years, 39dll has been the most popular networking library for Game Maker by quite a comfortable margin, and it has enabled many projects to implement their networking code. However, that does not mean it is a good library. It isn't. Maybe 39dll became popular because it was the only real option for a while, but that is not true anymore today. Using a more modern library like Faucet Networking will spare you some unnecessary headaches. But you don't have to take my word for that:
 

 


I don't know what Faucenet is (my guess, a new and much better networking lib) but I definitely recommend using it over [39dll], which I wrote when I literally just started programming in c++ smile.gif/>.
I'm always surprised to see that it is still popular, as it is very poorly coded.


So what is the actual problem with 39dll, and why did I start writing Faucet Networking? To better appreciate the motivation behind this project, you should know that I have used 39dll in Gang Garrison 2 for a long time, and finally grew tired of the hoops I had to jump through to perform the most basic tasks. In many cases, a task would seem easy at first, but then in some corner cases the easy solution would make my game freeze or misbehave, so I had to add code to deal with those corner cases. Suddenly, the solution wasn't so easy anymore, and I had spent hours on tracking down unexpected networking problems that could have been spent on inproving the gameplay.

A simple example is sending some bytes of data from a game server to a client, over an existing TCP connection. You will (and should!) probably expect that this is easy to do in any networking library. Naturally, you want all the bytes to eventually arrive at the client (unless the connection breaks). But you don't want to wait until the data is sent - Your game server needs to run (e.g.) 30 steps per second, and there is no way to tell how long the sending will take. Usually it is immediate, but it can take seconds in some cases, and you don't want your server to freeze for that long (or at all).

However, to do this properly with 39dll, you need to manually store the data that can't be sent immediately, and try resending it every step. You also need to handle specific return codes of 39dll functions which are completely undocumented, so you basically need to read the 39dll source code and cross-reference with the Winsock reference. All this adds up to a rather large block of GML code you have to write.

Faucet Networking on the other hand just accepts all your bytes without blocking and sends them in the background, so your GML code can do more interesting things. If you want details on this example, read on here - I didn't want to clobber the first post with too many details.

Receiving data has a similar problem. In the usual case where you know exactly how many bytes you need next, you have to take precautions for the case that some data is available, but not as much as you asked for.

Gang Garrison had a couple of scripts to provide a more convenient way of doing common tasks, but not every problem could be hidden this way, and some things - like the ever more important IPv6 support - obviously can't be done in GML alone. So I decided to take a deeper look under the hood of 39dll and add some tweaks. That was the initial plan at least. Working through the 39dll source code though, I realized there were more problems than just a clunky interface. 39dll has bugs. It leaks memory every time you delete a buffer or socket. It has hidden limits: if you try to read more than 20000 characters with readchars, there's an internal buffer overflow. It relies on undefined compiler behaviour: if you recompile it with gcc, simply calling writebyte(1000, buffer) will crash your game.

I realized that instead of trying to fix the problems, it would be more profitable to develop something new, starting with the question what a game programmer actually needs, based on my own experience with the development of Gang Garrison 2. The focus was to make the API as easy to use as possible, even if that meant the library would be more complicated to develop. I tried designing the API functions in a way that made them intuitive, and robust against errors and wrong usage.

When I had developed the extension far enough that it was able to replace 39dll for Gang Garrison, I did exactly that, and it was satisfying to see how much it simplified the code of the game in some places. However, I found that it still required too much work to do some common tasks, so I went back to the drawing board and revised several decisions to iron out the creases.

The result is a library that is easy to learn and use, and that gets common tasks done with a minimum of code and effort. I hope you will find it useful.


Edited by Medo42, 19 February 2014 - 10:19 PM.

  • 27

#2 Primoz128

Primoz128

    GMC Member

  • GMC Member
  • 313 posts
  • Version:GM:Studio

Posted 17 February 2011 - 08:56 PM

Umm... any examples to show it's power and reliability and that it actually works... ?
  • -1

#3 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 17 February 2011 - 09:38 PM

Umm... any examples to show it's power and reliability and that it actually works... ?

A Pong example game is included in the download. Also, the current development version of Gang Garrison 2 uses this library and it does work (the release version you can download on the homepage still uses 39dll). As for reliability, I don't know of any bugs, but it has not been extensively tested yet. That's why I am still calling this an alpha version. If you come across a bug I will be happy to fix it though.
  • 0

#4 Schyler

Schyler

    Noskcirderf Derf

  • GMC Member
  • 2679 posts
  • Version:GM:Studio

Posted 18 February 2011 - 03:46 AM

TCP in 39dll, you have two options: You can wait until all data is accepted (blocking mode), but this will essentially freeze your game for as long as it takes. Or you can write as much data as possible without blocking, but then you have to take precautions for the case that not everything is written.

Why don't you put some example code in the topic showing why yours doesn't do that.

The only way I can think of is that you are using GMAPI, and if you are, you didn't include credit.
  • 0

#5 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 18 February 2011 - 10:37 AM

I wasn't aware of GMAPI, that's an interesting project. No, I don't use that at the moment.

So, how does Faucet Networking avoid the problem? I did put it in the original post:

In Faucet Networking, writing to a socket always accepts all the data and returns immediately.

This is achieved by adding everything to an internal buffer and constantly sending the data from that buffer in the background.

Blocking send with 39dll
So let's compare the code needed. The reason why I didn't do that in the first post is that it's rather long and complicated. Using a blocking socket with 39dll is simple, and the only real complaint with it is that it can freeze your game, which makes it useless in server code of a realtime game:
if(sendmessage(socket, 0, 0, buffer) < 0) {
    // handle send error
}

Nonblocking send with 39dll
Here's the nonblocking 39dll example. This is actually taken directly from the old Gang Garrison 2 source. It is a script which is called on all send buffers in every step:
// This function attempts to send a buffer.
// All bytes successfully written will be cleared from the buffer.
// If the buffer is sent completely, it returns 0
// If the buffer is sent partially, it returns 1
// If the socket had to be closed due to an error or loss of connection, it returns 2

// argument 0: Socket
// argument 1: Buffer

var size;

setsync(argument0, 1);

size = sendmessage(argument0, 0, 0, argument1);
if(size == buffsize(argument1)) {
    clearbuffer(argument1);
    return 0;
} else if (size < 0) {
    switch(size) {
        case WSAENOBUFS:
        case WSAEWOULDBLOCK:
            if(buffsize(argument1)>100000) {
                closesocket(argument0);
                return 2;
            } else {
                return 1;   
            }
            
        default:
            closesocket(argument0);
            return 2;
    }
} else if(size < buffsize(argument1)) {
    if(buffsize(argument1)-size>100000) {
        closesocket(argument0);
        return 2;
    } else {
        clearbuffer(global.tempBuffer);
        copybuffer2(global.tempBuffer, size, buffsize(argument1)-size, argument1);
        clearbuffer(argument1);
        copybuffer(argument1, global.tempBuffer);
        return 1;
    }
}
Do you think this is excessive? Well, let me explain why it does what it does.
When the socket is in nonblocking mode, sendmessage might not send everything. If it does (first if-block), everything is fine and we're done.
If the operating system has no buffer space available at all, it might not send anything and return an error. But that specific error (WSAEWOULDBLOCK) does not mean the connection is broken, just that we can't send anything right now, so we have to check the actual error code and act depending on that (switch-block).
What can also happen is that sendmessage just sends less than we asked for. In that case you have to remove the sent bytes from the buffer, so that you can send the rest next time. However, 39dll doesn't offer a function to cut bytes from the start of a buffer, so you have to use a temp buffer and copy the remaining data back and forth (last else if-block).
In some places there are checks to prevent the buffer from growing too large (e.g. if(buffsize(argument1)>100000)), which would happen on bad connections where the server generates data faster than it is transmitted.
I won't go into detail on what you'd have to do to use message format 0 or 1 correctly with a nonblocking socket, but basically you'd have to remember that a message was interrupted midway and send the rest in message format 2 to avoid a new header from being inserted by 39dll.

Nonblocking send with Faucet Networking
Complicated enough? Now here's the nonblocking code with Faucet Networking:
write_buffer(socket, buffer);
socket_send(socket);

To check if there was an error you can add the following, but you'll usually only check a socket for errors once per step:
if(socket_has_error(socket)) {
    // handle error, e.g. disconnect player
}

If you want to add a limit to the internal buffer to prevent slow connections from hogging your memory (say, 100kb as in the nonblocking 39dll example above):
socket_sendbuffer_limit(socket, 100000)
The socket will automatically cut the connection if the internal send buffer grows larger than that limit.

Edited by Medo42, 18 February 2011 - 08:46 PM.

  • 1

#6 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 01 March 2011 - 07:04 PM

V1.0b1 has been released.

Download: here

Changes:
  • Bugfix: Acceptors were not being destroyed properly
  • Bugfix: Avoided potential conflict with dual stack sockets under Windows Vista

As far as I can tell, this is relatively stable now, so I'm calling this the first beta version. Keep in mind though that it didn't get much real-world testing yet, that's your part :)

Edited by Medo42, 01 March 2011 - 07:55 PM.

  • 1

#7 nutsaq

nutsaq

    GMC Member

  • New Member
  • 15 posts

Posted 04 March 2011 - 12:48 PM

I've started using this and I like it. I have not run into any bugs yet. Looking forward to full UDP functionality!
  • 0

#8 SimplySeth

SimplySeth

    GMC Member

  • GMC Member
  • 79 posts
  • Version:GM:Studio

Posted 06 March 2011 - 10:51 PM

This looks pretty impressive. Downloading now :D
  • 0

Posted Image
"It's a big Universe, everything happens somewhere," --The Doctor


#9 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 23 March 2011 - 11:30 AM

V1.0b2 is out now.
Download here

Changes:
- Fixed hang on shutdown when there was still a running operation (e.g. a connection attempt)

Edited by Medo42, 23 March 2011 - 11:32 AM.

  • 0

#10 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 07 April 2011 - 12:10 AM

V1.0 is out now.
Download here

Changes:
- Added a function socket_remote_ip to find out which IP a socket is connected to
- Changed the name to Faucet Networking (from Faucet Networking Extension - it was a bit redundand)

Because of the name change, you will have to install the new version first, then change your game to use the new version and finally uninstall the old one.
  • 0

#11 tie372

tie372

    Bassist of Death

  • New Member
  • 1038 posts
  • Version:Unknown

Posted 19 April 2011 - 05:16 AM

Looks good. Will use it once UDP is added in. GG2 is all TCP based?
  • 0
Posted Image
Posted ImagePosted ImagePosted Image

#12 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 19 April 2011 - 09:29 AM

Looks good. Will use it once UDP is added in. GG2 is all TCP based?

Yes, it's a somewhat unconventional choice, but when we started out I did not have experience with network programming and we had a deadline to work towards. Switching to UDP now would mean rewriting rather large parts of the game, and nobody wants to invest the time right now. TCP is a workable solution as you can see in GG2, and most of the latency felt in the game is not because of that choice of protocol, but rather because we don't do any input prediction. That said, for the best results with realtime games you should use something UDP based to avoid the latency spike that will happen when a TCP packet is lost.

UDP will make it into the extension, but unfortunately have other priorities at the moment. I hope get around to by the end of May.
  • 0

#13 sabriath

sabriath

    12013

  • GMC Member
  • 3192 posts

Posted 19 April 2011 - 01:46 PM

I am using this post to tag your topic....think of it as a free bump :)

Any plans you wish to share with your "rival"? teehee
  • 0

Tutorials of Interest:
* Multiplayer: mine or True Valhalla's

Projects:
* Net39
* My 39dll scripts
* My 39dll lib
* Multiplayer Engine
* Artificial Chemistry

Posted Image

#14 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 19 April 2011 - 02:40 PM

I am using this post to tag your topic....think of it as a free bump :)

Any plans you wish to share with your "rival"? teehee

Yes: I plan to stick with having this extension do one thing, and do it well. In fact, if the "shared buffers" project takes off I'll even remove my own buffer code.

Congratulations on your 3000th post, which might appear below :P
  • 0

#15 tie372

tie372

    Bassist of Death

  • New Member
  • 1038 posts
  • Version:Unknown

Posted 27 April 2011 - 05:41 AM

Played around with this some more and it's basically amazing.

What's the approach on sending multiple variable length strings in one message? Custom delimiters (read one character at a time via read_string() until you reach the delimiter)? Some sort of size heading for each string?
  • 0
Posted Image
Posted ImagePosted ImagePosted Image

#16 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 27 April 2011 - 08:03 AM

Played around with this some more and it's basically amazing.

What's the approach on sending multiple variable length strings in one message? Custom delimiters (read one character at a time via read_string() until you reach the delimiter)? Some sort of size heading for each string?

At the moment it's far easier to send a size header in advance, but I plan to add functions that you can use to receive / read data delimited by any string.
  • 0

#17 scorpeti

scorpeti

    GMC Member

  • New Member
  • 15 posts

Posted 03 May 2011 - 10:30 AM

I started to use your extension's newest version, but I ran into a problem. In the first game window, I created an acceptor (acceptor=1) as a server, then I connected itself as a client (socket_server=2) and accepted itself as a server (socket[0]=3), everything's perfect. After that I opened an other game window. In this new game I connected to the existing server (socket_server=1) and in the first game, the server accepted this one and saved it's socket (socket[1]=4). So I have two game windows with one server and two clients. After that I wanted to disconnect a client by clicking close button on the second game window, so I did it, but unfortunately the server didn't notice anything. If I close the first game window like this and shut down the server, the client should notice this, but nothing happened. In every game, I'm checking the sockets in begin step event by socket_has_error() function, but it says no error: returns 0. If I try to connect to a non existing server, this error code works perfectly. It's probably my mistake, but I cannot find out what's wrong. I ran the games in debug mode many times and just this socket_has_error code goes wrong. If I close the socket by socket_destroy function, same problem appears.
  • 0
Posted Image

#18 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 03 May 2011 - 05:32 PM

I started to use your extension's newest version, but I ran into a problem. In the first game window, I created an acceptor (acceptor=1) as a server, then I connected itself as a client (socket_server=2) and accepted itself as a server (socket[0]=3), everything's perfect. After that I opened an other game window. In this new game I connected to the existing server (socket_server=1) and in the first game, the server accepted this one and saved it's socket (socket[1]=4). So I have two game windows with one server and two clients. After that I wanted to disconnect a client by clicking close button on the second game window, so I did it, but unfortunately the server didn't notice anything. If I close the first game window like this and shut down the server, the client should notice this, but nothing happened. In every game, I'm checking the sockets in begin step event by socket_has_error() function, but it says no error: returns 0. If I try to connect to a non existing server, this error code works perfectly. It's probably my mistake, but I cannot find out what's wrong. I ran the games in debug mode many times and just this socket_has_error code goes wrong. If I close the socket by socket_destroy function, same problem appears.


There are several reasons why the sockets might not show an error. The first one is simple: socket_has_error is implemented in a passive way. At the moment, a socket will only show an error if some command (e.g. sending or receiving data) actually failed, so if you don't do anything except check for errors, you will never see that the connection is no longer working. This behavior is consistent with the documentation but a bit unintuitive, so I could change the function to actively check if the socket is still good.

Even with that change you might not get an error though, at least for a while, because there simply is no error if the other side closed the connection cleanly - in fact, in that case the connection is still half-open until you either close the other side as well or some timeout occurs. Disconnecting is a surprisingly complicated subject in TCP :)

You can find out if the connection was closed with the help of tcp_eof() though - it will return true in that case, unless there is still data left to be read. Another solution is to actually use the sockets for something: If you try reading data from the connection with tcp_receive() after the connection is closed, you will get an error the way you expected.
  • 0

#19 scorpeti

scorpeti

    GMC Member

  • New Member
  • 15 posts

Posted 03 May 2011 - 09:38 PM

Thank you very much. The problem was your first guess: I didn't do anything but checking for errors. :) Now it's solved, extension works fine.
  • 0
Posted Image

#20 thaddeus_maximus

thaddeus_maximus

    GMC Member

  • GMC Member
  • 39 posts

Posted 06 May 2011 - 05:56 AM

I've been a user of 39dll for a long time. However, I can't wait to try this out. I wish you the best of luck in your development.
  • 3

#21 DanRedux

DanRedux

    GMC Member

  • GMC Member
  • 1403 posts
  • Version:GM:Studio

Posted 11 May 2011 - 08:31 PM

I have a few notes.

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

Second, don't think Blocking mode is a bad thing. It's not. Blocking mode simplifies actions quite a lot for new users who expect actions to complete instantly.

Here are my thoughts. When you run the script that connects to a server in blocking mode, it enters the game into a Pause loop with a small "Connecting" text somewhere. The script returns False if you couldn't connect and a Socket ID if you could.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

In other words, you're aiming to beat 39dll, but you do this in very few ways, most of which are behind the scenes (which we don't see anyway). If you want a Networking DLL to be actually useful, it needs to maintain the standard that 39dll set, and THEN include extra functionality. Right now, if I replace every 39dll script with it's Faucet equivalent, I get absolutely no benefits, realistically speaking. I also lose a lot of functionality.

About buffers. My view on Buffers is that you should be able to have multiple buffers. The DLL should not create one automatically, you should have to do it yourself. Buffers can be used as shortcuts. In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

This makes the game easy to program. Any time someone logs in, I simply send all the clients Name and Position buffers to the new client. Having the client instances maintain their buffers simplifies the programming considerably.

However, to summarize my thoughts, I AM going to be using BOTH dll's in the same project. 39dll will be connecting to a different port, but both connections will be maintained. This is because while 39DLL is currently the better option, I feel like Faucet could soon become very useful, especially if it incorporated a Database, and I want it to already be thoroughly embedded in my project for when that time arrives.
  • 0

#22 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 11 May 2011 - 10:12 PM

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

Extensions are harder to modify. With 39DLL, one could easily get your encryption keys. Plus, the external_* functions are (pretty much) deprecated.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

Faucet Networking

In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

You only need one buffer for a single-threaded server, you know. Does having all of these buffers increase speed at all (stats)?
  • 1

#23 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 11 May 2011 - 10:32 PM

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

You are welcome to use this extension as a base for a set of scripts that better fit your intended usage. However, I do not see why a dll/scripts combination would be superior to an extension. You can implement extension functions as GML scripts too, if you need to manipulate GM resources or take advantage of features like variadic functions or parameters that can be either strings or numbers.

Second, don't think Blocking mode is a bad thing. It's not. Blocking mode simplifies actions quite a lot for new users who expect actions to complete instantly.

Here are my thoughts. When you run the script that connects to a server in blocking mode, it enters the game into a Pause loop with a small "Connecting" text somewhere. The script returns False if you couldn't connect and a Socket ID if you could.

You can write a connection script like you describe, which would essentially be blocking for the user but not completely stalling the game, but to do that you need a nonblocking way of connecting as a basis. Once you have that, it just takes a few lines of code to build a loop around it that waits for the connection and also redraws the screen now and then to prevent the game from appearing unresponsive. I might add a basic script like that to the extension for convenience, but many people will want to roll their own so they can add in more fancy stuff like a "Connecting..." dialog where you can abort the attempt, and perhaps animation.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

I do not see any benefit of having database functionality in the same extension as networking. The two do not seem to be related in any substantial way.

About buffers. My view on Buffers is that you should be able to have multiple buffers. The DLL should not create one automatically, you should have to do it yourself. Buffers can be used as shortcuts. In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

Perhaps you should read the help file again. You can create as many buffers as you like.

However, to summarize my thoughts, I AM going to be using BOTH dll's in the same project. 39dll will be connecting to a different port, but both connections will be maintained. This is because while 39DLL is currently the better option, I feel like Faucet could soon become very useful, especially if it incorporated a Database, and I want it to already be thoroughly embedded in my project for when that time arrives.

I do not recommend to combine two libraries in this way. It should work, but you will be subject to the bugs and idiosyncrasies of both at the same time. If you rely on features of 39dll and would only include Faucet Networking in the hope that it will see a feature explosion, stick with 39dll throughout.
  • 1

#24 DanRedux

DanRedux

    GMC Member

  • GMC Member
  • 1403 posts
  • Version:GM:Studio

Posted 11 May 2011 - 11:16 PM

My comment on buffers was just generic, not specifically aimed at Faucet, just letting you know to keep that functionality in, as apposed to removing it for "simplicity".

Also, are EXT really more secure than DLL's? I was unaware. When I loaded up Faucet GEX, auto-complete wasn't helping me at all, so I didn't know the arguments. Also, opening a GM81, for me, doesn't auto-load the related GEX, so I have to load the GEX manually. I don't know why. It's installed, just doesn't automatically enable itself.

Also, a DB would be immensely useful, instead of making us do it ourselves. The entire "Account" system in an MMORPG relies on DB, so if you included even just a basic DB system, it would satisfy most of a Servers needs.

Edited by time4dan, 11 May 2011 - 11:17 PM.

  • 0

#25 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 12 May 2011 - 12:14 AM

Also, a DB would be immensely useful, instead of making us do it ourselves. The entire "Account" system in an MMORPG relies on DB, so if you included even just a basic DB system, it would satisfy most of a Servers needs.

http://gmc.yoyogames.com/index.php?showtopic=402456
  • 1

#26 sabriath

sabriath

    12013

  • GMC Member
  • 3192 posts

Posted 13 May 2011 - 06:17 AM

Second, don't think Blocking mode is a bad thing.

It is.

You only need one buffer for a single-threaded server, you know.

lol...not going there.

I do not see any benefit of having database functionality in the same extension as networking. The two do not seem to be related in any substantial way.

Actually they are tied almost on a 1:1 basis, so having DB functions in the networking DLL wouldn't hurt....in fact, I've already planned it for mine.

Also, are EXT really more secure than DLL's?

Not really...not even sure why that was even mentioned. I see extensions as more of a packaging of the DLL into your executable, while the other method relies on script calls which are slower, that alone should make you choose extensions over scripts.

{link 402456}

sql...lol
  • 0

Tutorials of Interest:
* Multiplayer: mine or True Valhalla's

Projects:
* Net39
* My 39dll scripts
* My 39dll lib
* Multiplayer Engine
* Artificial Chemistry

Posted Image

#27 billydoesitbest

billydoesitbest

    GMC Member

  • GMC Member
  • 445 posts

Posted 03 June 2011 - 06:13 AM

Very intrigued by your DLL! I am going to try it out tomorrow. I hope you roll out with the updates rather fast.
  • 0

#28 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 04 June 2011 - 08:58 AM

Here is the current plan for version 2:
- UDP support
- Improved TCP support (In particular, adding a tcp_shutdown or tcp_close_send function to allow half-close operation)
- Switch to using shared buffers. This will change the existing API, sorry. Well, it is a major-version update. I can try to provide a compatibility extension that adds the old function names back.
- Delimited receiving functions, so you can more comfortably receive messages of which you don't know the size in advance, but which end with a fixed string. For example, receiving whole lines that end with \r\n, or 0-terminated strings.

It could still take a while before this is finished, because I don't have as much time as I would want, and that won't change for at least another two months. At the moment, I am working on the shared buffers system, which will provide a common buffer implementation, common GM functions for reading and writing data to/from buffers, and a way to let different extensions share buffers and streams.
  • 0

#29 desolatorXT

desolatorXT

    GMC Member

  • GMC Member
  • 354 posts
  • Version:GM:Studio

Posted 06 June 2011 - 05:35 PM

Is this extention compatible with GM 8.1?
  • 0

- Random Retro Space Shooter: http://gmc.yoyogames...howtopic=605687

 

- Nameless TDS game: http://gmc.yoyogames...howtopic=453982 (Project is on hold)
 


#30 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 06 June 2011 - 07:21 PM

Is this extention compatible with GM 8.1?

As far as I know, yes.
If you do find any problems, please let me know.
  • 0

#31 jon sploder

jon sploder

    GMC Member

  • GMC Member
  • 919 posts

Posted 13 June 2011 - 01:32 AM

Hey Medo42, any ideas as to when version 2, or namely UDP support will come along? Wondering if I should freeze any work in 39dll for this. ETA?
  • 0

#32 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 13 June 2011 - 01:13 PM

Hey Medo42, any ideas as to when version 2, or namely UDP support will come along? Wondering if I should freeze any work in 39dll for this. ETA?

Faucet Networking 2 is the new Duke Nuker Forever :D

More seriously though, the original plan was to work on this starting in May, but I am writing a thesis which is dragging on a bit longer than expected, and getting this finished is rather more important to me right now.

As I wrote above, the plan is to collect a few new features and adapt the API a bit to create a completely new version. However, since things are going very slowly at the moment, maybe I should add preliminary UDP support to version 1 first since this is the most important new feature for several people. UDP support as such is not very difficult, but I might not create the best API on the first try, so I might change it again in verison 2. That means you should be prepared to either stick with version 1 of the extension or to modify your networking code later. Would that be ok? On the upside, it would mean you could give feedback on how the api could be improved for version 2.
  • 1

#33 jon sploder

jon sploder

    GMC Member

  • GMC Member
  • 919 posts

Posted 14 June 2011 - 06:52 AM

I'd be fine to modify my code a bit, and I'm keen to give feedback on features required. I definitely encourage you to add preliminary UDP support!
  • 0

#34 OldSkoolGamer

OldSkoolGamer

    GMC Member

  • GMC Member
  • 128 posts
  • Version:GM:Studio

Posted 16 June 2011 - 06:24 PM

I second the request for adding preliminary UDP support, only if it doesn't take you away from your Thesis too long of course Posted Image
  • 0

#35 Knuked

Knuked

    GMC Member

  • New Member
  • 242 posts
  • Version:GM:HTML5

Posted 18 June 2011 - 12:06 AM

Would that be ok?


Absolutely! I've been lurking in this topic for what seems like forever. I'm very pumped for the UDP support!
  • 0

#36 loopy268

loopy268

    GMC Member

  • GMC Member
  • 242 posts

Posted 19 June 2011 - 07:29 AM

Imma download it tomorrow and edit this post! :D
  • 0
Ok, can someone join this? It only has 2 members on it... this isn't a site that remakes GMC, it's a site just for making games on any type of system... so please join? And if you do I'll give you +1, but you must be active! :)
Thanks! :D
Epic site!

+1?


#37 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 03 July 2011 - 09:13 PM

There are good news and bad news.
Good news, I started work on the preliminary UDP support. It will look quite similar to the UDP support in 39dll, except that UDP sockets will contain their internal send and receive buffers just like TCP sockets do.
Bad news, I'll probably remove IPv6 support for WinXP and below. This has always been a bit of a kludge, and on the upside I can finally allow binding TCP and UDP sockets to a random unused port.
  • 0

#38 death-droid

death-droid

    GMC Member

  • GMC Member
  • 2600 posts

Posted 05 July 2011 - 02:12 PM

Being a user of 39dll for a couple of projects, this is already looking a lot cleaner and simpler to use.
I'll definitely be using this for my upcoming project.

Keep up the good work!!
  • 0

Posted Image


#39 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 08 July 2011 - 01:13 AM

Keeping up the progress reports, I spent the entire day today working on this. Everything is still a bit shaky and unfinished, but UDP sending and receiving generally works now. I didn't remove the IPv6 support for WinXP after all, but under some (relatively unlikely) circumstances the improvised solution can fail and then you get a socket with just IPv4.

You can write data to a UDP socket just like a TCP socket or a buffer, and then send it using udp_send(socket, ip, port). In contrast to tcp_connect, udp_send doesn't take hostnames as parameter. That's because tcp_connect only runs once per socket, so the added weight of the hostname lookup isn't as big, but udp_send it usually called a lot, and attempting to look up a hostname every time could lead to a big queue of lookups that clog the entire socket. Of course, that means a new function is needed to manually resolve hostnames to IPs. This is one of the things which haven't been written yet.

udp_send pushes the current content of the sendbuffer into the send queue, and the packets are generated from that queue in the background worker thread. Similar to TCP buffers, you can set a maximum size for that send queue to prevent it from growing indefinitely if you attempt to send data faster than it can be pushed out to the network. If the queue is full, attempting to send another packet will simply throw that packet away.

On the receiving end, there is another queue where newly received datagrams are placed by the worker thread. Again, in order to prevent memory problems the size is limited, and if too much data is received without being read by the application, packets will be thrown away. The size is fixed to 2MiB at the moment which should be plenty, but I will probably add another function to allow tweaking it.

udp_receive(socket) transfers a datagram from the receive queue into the receive buffer. In addition to reading the data, you can also query the source of the datagram which is currently in the receive buffer by using socket_remote_ip(socket) and socket_remote_port(socket). You can also ask about the port that the socket itself is bound to by using socket_local_port(socket), which is useful if you let the OS choose the port number. socket_remote_ip and socket_remote_port work for both TCP and UDP sockets, socket_local_port works for those as well and additionally for acceptors.

I probably won't be able to work on this in the next few days, so... patience please. I'll finish it. Eventually. :D
  • 2

#40 death-droid

death-droid

    GMC Member

  • GMC Member
  • 2600 posts

Posted 08 July 2011 - 06:03 AM

Sounding really good there man, Im glad you didnt end up reiving IPv6 support for WinXP.

EDIT:
One problem i seem to be having, the send buffer and recieve buffer just seems to grow and grow without anything actually happening

Edited by death-droid, 11 July 2011 - 11:52 AM.

  • 0

Posted Image


#41 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 14 July 2011 - 09:13 PM

Sounding really good there man, Im glad you didnt end up reiving IPv6 support for WinXP.

EDIT:
One problem i seem to be having, the send buffer and recieve buffer just seems to grow and grow without anything actually happening

It doesn't make much sense to me that you say the receive buffer would be growing. TCP sockets in this extension only read data from the Windows socket when you ask them to receive data.
If the sendbuffer grows, this indicates that you are sending data more quickly than it is processed by the receiver, or more quickly than the network can transport.

Some more progress information, I sketched the API for nonblocking hostname lookups today and implemented/documented/wrote unit tests for two functions ip_is_v4 and ip_is_v6 which you can use to find out whether a string represents an IPv4/IPv6 address literal.

Edited by Medo42, 14 July 2011 - 09:27 PM.

  • 0

#42 jobro

jobro

    GMC Member

  • GMC Member
  • 1300 posts
  • Version:GM:Studio

Posted 14 July 2011 - 10:14 PM

I encountered a minimal problem when opening up .gmk files in 8.1, it doesn't find the extension even though I've installed it. No big deal though since one can always add it in after the file is loaded.
  • 0

Marketplace.png


#43 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 14 July 2011 - 10:21 PM

I encountered a minimal problem when opening up .gmk files in 8.1, it doesn't find the extension even though I've installed it. No big deal though since one can always add it in after the file is loaded.

This might be unrelated to GM 8.1 - I renamed the extension from "Faucet Networking Extension" to "Faucet Networking" in version 1.0, and forgot to change the reference in the example .gmk files. You might just be getting the error from that.
  • 0

#44 death-droid

death-droid

    GMC Member

  • GMC Member
  • 2600 posts

Posted 14 July 2011 - 11:53 PM

@Medo42, i must of done something wrong, ive sent you what ive coded, i cant see where ive gone wrong -.-
  • 0

Posted Image


#45 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 15 July 2011 - 12:49 AM

@Medo42, i must of done something wrong, ive sent you what ive coded, i cant see where ive gone wrong -.-

Neither can I because megaupload doesn't want me to download this. Try to put the relevant code on Pastebin instead, or just post it here. However, I won't take a closer look at it if all you can be bothered to say is basically "it doesn't work, please fix it for me". Try to describe your problem in a precise and detailled way and I (or someone else) might be able to help you.
  • 0

#46 death-droid

death-droid

    GMC Member

  • GMC Member
  • 2600 posts

Posted 15 July 2011 - 03:27 AM


@Medo42, i must of done something wrong, ive sent you what ive coded, i cant see where ive gone wrong -.-

Neither can I because megaupload doesn't want me to download this. Try to put the relevant code on Pastebin instead, or just post it here. However, I won't take a closer look at it if all you can be bothered to say is basically "it doesn't work, please fix it for me". Try to describe your problem in a precise and detailled way and I (or someone else) might be able to help you.


XD I explained to you whats wrong with it, the receive buffer just seems to grow larger and larger without any processing actually being done, the server seems to also send in chunks of data and not straight away when socket_send is called. The same method used with 39dll works absolutely fine, btw this is just sending to the localhost (127.0.0.1). I've tryed to find out whats causing it at all, everything should be being processed correctly so im left clueless.

Edited by death-droid, 15 July 2011 - 03:36 AM.

  • 0

Posted Image


#47 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 15 July 2011 - 09:47 AM

I had some more time yesterday because people on the floor below me had a party until quite late, so I stayed up a bit and implemented the planned IP lookup functions. There are seven new functions for this, which seems a bit inflationary, but it should be pretty easy to get the hang of it.

The main problem is that lookups can take time, sometimes several seconds, so the API to use them has to be nonblocking. As a result, IP lookups are now resources of their own that have to be created and destroyed. You can create them with ip_lookup_create(hostname), which returns a lookup handle and starts resolving the name. There are also variants ipv4_lookup_create and ipv6_lookup_create, which will only look for IPv4/IPv6 addresses - the default way is to look for both.

After creating them, you can regularly query whether the lookups are finished by calling ip_lookup_ready(lookup), which unsurprisingly returns true once the results are ready. The thing about the results is that there can be several IPs for a single hostname - take google.com for example, which has multiple IPv4 adresses associated with it. In order to read the results, you use the functions ip_lookup_has_next() and ip_lookup_next_result(). The first one tells you if there is another IP in the results that you didn't read yet, and the second one reads the next IP. This is similar to the concept of an iterator, and makes it convenient to loop over all results. For example, the following code queries all IPs associated with google.com and shows a message for each one:

lookup = ip_lookup_create("google.com");
while(!ip_lookup_ready(lookup)) {
    sleep(1); // Or do something useful, like redraw your screen
}
while(ip_lookup_has_next(lookup)) {
    show_message(ip_lookup_next_result(lookup));
}
ip_lookup_destroy(lookup);

Edited by Medo42, 15 July 2011 - 09:49 AM.

  • 1

#48 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 16 July 2011 - 11:45 PM

More progress: I updated the documentation with the new IP lookup functions and played around with hostname resolution a bit. In the end, I decided to allow using hostnames in UDP send after all, since you sometimes just want to send a few packets where the overhead of the extra lookup isn't a big problem, and it's just a bit more convenient that way. It's also more consistent, since tcp_connect allows hostnames as well.

There are still a few things to do, but we're getting closer :)
  • 0

#49 8-BitTonberry

8-BitTonberry

    GMC Member

  • GMC Member
  • 673 posts
  • Version:GM:Studio

Posted 18 July 2011 - 04:25 AM

Normally, I'm pretty good with these things, but would there be any chance you could give a brief explanation about how to send and receive variables?

Thanks! =]

This is pretty amazing, by the way.
  • 0
8-BitTonberry.png

#50 Medo42

Medo42

    GMC Member

  • GMC Member
  • 317 posts

Posted 18 July 2011 - 06:51 PM

Normally, I'm pretty good with these things, but would there be any chance you could give a brief explanation about how to send and receive variables?

Thanks! =]

This is pretty amazing, by the way.

To send something, you need to write it into the socket's sendbuffer first. You can write the value of a variable there using the write_X functions. If the variable contains a number, you can e.g. write
write_double(socket, variable)
in order to write the value with full precision. If it's a string, you can use
write_string(socket, variable)
- however, in order to receive the string on the other side you need to know how long it is, so it's a good idea to write the length of the string first.
Then you call socket_send(socket) to actually send your data.

Receiving works by calling one of the tcp_receive functions and then using something like
variable = read_double(socket)

  • 0




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users