379 replies to this topic

[DanRedux]

I have a few notes.

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

Second, don't think Blocking mode is a bad thing. It's not. Blocking mode simplifies actions quite a lot for new users who expect actions to complete instantly.

Here are my thoughts. When you run the script that connects to a server in blocking mode, it enters the game into a Pause loop with a small "Connecting" text somewhere. The script returns False if you couldn't connect and a Socket ID if you could.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

In other words, you're aiming to beat 39dll, but you do this in very few ways, most of which are behind the scenes (which we don't see anyway). If you want a Networking DLL to be actually useful, it needs to maintain the standard that 39dll set, and THEN include extra functionality. Right now, if I replace every 39dll script with it's Faucet equivalent, I get absolutely no benefits, realistically speaking. I also lose a lot of functionality.

About buffers. My view on Buffers is that you should be able to have multiple buffers. The DLL should not create one automatically, you should have to do it yourself. Buffers can be used as shortcuts. In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

This makes the game easy to program. Any time someone logs in, I simply send all the clients Name and Position buffers to the new client. Having the client instances maintain their buffers simplifies the programming considerably.

However, to summarize my thoughts, I AM going to be using BOTH dll's in the same project. 39dll will be connecting to a different port, but both connections will be maintained. This is because while 39DLL is currently the better option, I feel like Faucet could soon become very useful, especially if it incorporated a Database, and I want it to already be thoroughly embedded in my project for when that time arrives.

[TheMagicNumber]

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

Extensions are harder to modify. With 39DLL, one could easily get your encryption keys. Plus, the external_* functions are (pretty much) deprecated.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

Faucet Networking

In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

You only need one buffer for a single-threaded server, you know. Does having all of these buffers increase speed at all (stats)?

[Medo42]

First of all, Extensions suck for this kind of thing, at least for me. I'd really appreciate a Script Set, and a DLL, as apposed to an Extension.

You are welcome to use this extension as a base for a set of scripts that better fit your intended usage. However, I do not see why a dll/scripts combination would be superior to an extension. You can implement extension functions as GML scripts too, if you need to manipulate GM resources or take advantage of features like variadic functions or parameters that can be either strings or numbers.

Second, don't think Blocking mode is a bad thing. It's not. Blocking mode simplifies actions quite a lot for new users who expect actions to complete instantly.

Here are my thoughts. When you run the script that connects to a server in blocking mode, it enters the game into a Pause loop with a small "Connecting" text somewhere. The script returns False if you couldn't connect and a Socket ID if you could.

You can write a connection script like you describe, which would essentially be blocking for the user but not completely stalling the game, but to do that you need a nonblocking way of connecting as a basis. Once you have that, it just takes a few lines of code to build a loop around it that waits for the connection and also redraws the screen now and then to prevent the game from appearing unresponsive. I might add a basic script like that to the extension for convenience, but many people will want to roll their own so they can add in more fancy stuff like a "Connecting..." dialog where you can abort the attempt, and perhaps animation.

The other thing a DLL like this requires is basic databasing, IE, an in-memory database with fast searching and inserting, like SQL. It also needs auto-backup functionality so that updates are saved to the in-memory database AND are queued up to be saved to the Hard Drive when the Server has some idle time.

I do not see any benefit of having database functionality in the same extension as networking. The two do not seem to be related in any substantial way.

About buffers. My view on Buffers is that you should be able to have multiple buffers. The DLL should not create one automatically, you should have to do it yourself. Buffers can be used as shortcuts. In my MMO engine, every client instance on the server has a lot of buffers it maintains. One, for example, might be it's name. Another might be it's position. A third might be it's chat.

Perhaps you should read the help file again. You can create as many buffers as you like.

However, to summarize my thoughts, I AM going to be using BOTH dll's in the same project. 39dll will be connecting to a different port, but both connections will be maintained. This is because while 39DLL is currently the better option, I feel like Faucet could soon become very useful, especially if it incorporated a Database, and I want it to already be thoroughly embedded in my project for when that time arrives.

I do not recommend to combine two libraries in this way. It should work, but you will be subject to the bugs and idiosyncrasies of both at the same time. If you rely on features of 39dll and would only include Faucet Networking in the hope that it will see a feature explosion, stick with 39dll throughout.

[DanRedux]

My comment on buffers was just generic, not specifically aimed at Faucet, just letting you know to keep that functionality in, as apposed to removing it for "simplicity".

Also, are EXT really more secure than DLL's? I was unaware. When I loaded up Faucet GEX, auto-complete wasn't helping me at all, so I didn't know the arguments. Also, opening a GM81, for me, doesn't auto-load the related GEX, so I have to load the GEX manually. I don't know why. It's installed, just doesn't automatically enable itself.

Also, a DB would be immensely useful, instead of making us do it ourselves. The entire "Account" system in an MMORPG relies on DB, so if you included even just a basic DB system, it would satisfy most of a Servers needs.

Edited by time4dan, 11 May 2011 - 11:17 PM.

[TheMagicNumber]

Also, a DB would be immensely useful, instead of making us do it ourselves. The entire "Account" system in an MMORPG relies on DB, so if you included even just a basic DB system, it would satisfy most of a Servers needs.

http://gmc.yoyogames.com/index.php?showtopic=402456

[sabriath]

Second, don't think Blocking mode is a bad thing.

It is.

You only need one buffer for a single-threaded server, you know.

lol...not going there.

I do not see any benefit of having database functionality in the same extension as networking. The two do not seem to be related in any substantial way.

Actually they are tied almost on a 1:1 basis, so having DB functions in the networking DLL wouldn't hurt....in fact, I've already planned it for mine.

Also, are EXT really more secure than DLL's?

Not really...not even sure why that was even mentioned. I see extensions as more of a packaging of the DLL into your executable, while the other method relies on script calls which are slower, that alone should make you choose extensions over scripts.

{link 402456}

sql...lol

[billydoesitbest]

Very intrigued by your DLL! I am going to try it out tomorrow. I hope you roll out with the updates rather fast.

[Medo42]

Here is the current plan for version 2:
- UDP support
- Improved TCP support (In particular, adding a tcp_shutdown or tcp_close_send function to allow half-close operation)
- Switch to using shared buffers. This will change the existing API, sorry. Well, it is a major-version update. I can try to provide a compatibility extension that adds the old function names back.
- Delimited receiving functions, so you can more comfortably receive messages of which you don't know the size in advance, but which end with a fixed string. For example, receiving whole lines that end with \r\n, or 0-terminated strings.

It could still take a while before this is finished, because I don't have as much time as I would want, and that won't change for at least another two months. At the moment, I am working on the shared buffers system, which will provide a common buffer implementation, common GM functions for reading and writing data to/from buffers, and a way to let different extensions share buffers and streams.

[desolatorXT]

Is this extention compatible with GM 8.1?

[Medo42]

Is this extention compatible with GM 8.1?

As far as I know, yes.
If you do find any problems, please let me know.

[jon sploder]

Hey Medo42, any ideas as to when version 2, or namely UDP support will come along? Wondering if I should freeze any work in 39dll for this. ETA?

[Medo42]

Hey Medo42, any ideas as to when version 2, or namely UDP support will come along? Wondering if I should freeze any work in 39dll for this. ETA?

Faucet Networking 2 is the new Duke Nuker Forever

More seriously though, the original plan was to work on this starting in May, but I am writing a thesis which is dragging on a bit longer than expected, and getting this finished is rather more important to me right now.

As I wrote above, the plan is to collect a few new features and adapt the API a bit to create a completely new version. However, since things are going very slowly at the moment, maybe I should add preliminary UDP support to version 1 first since this is the most important new feature for several people. UDP support as such is not very difficult, but I might not create the best API on the first try, so I might change it again in verison 2. That means you should be prepared to either stick with version 1 of the extension or to modify your networking code later. Would that be ok? On the upside, it would mean you could give feedback on how the api could be improved for version 2.

[jon sploder]

I'd be fine to modify my code a bit, and I'm keen to give feedback on features required. I definitely encourage you to add preliminary UDP support!

[OldSkoolGamer]

I second the request for adding preliminary UDP support, only if it doesn't take you away from your Thesis too long of course

[Knuked]

Would that be ok?

Absolutely! I've been lurking in this topic for what seems like forever. I'm very pumped for the UDP support!

[loopy268]

Imma download it tomorrow and edit this post!

[Medo42]

There are good news and bad news.
Good news, I started work on the preliminary UDP support. It will look quite similar to the UDP support in 39dll, except that UDP sockets will contain their internal send and receive buffers just like TCP sockets do.
Bad news, I'll probably remove IPv6 support for WinXP and below. This has always been a bit of a kludge, and on the upside I can finally allow binding TCP and UDP sockets to a random unused port.

[death-droid]

Being a user of 39dll for a couple of projects, this is already looking a lot cleaner and simpler to use.
I'll definitely be using this for my upcoming project.

Keep up the good work!!

[Medo42]

Keeping up the progress reports, I spent the entire day today working on this. Everything is still a bit shaky and unfinished, but UDP sending and receiving generally works now. I didn't remove the IPv6 support for WinXP after all, but under some (relatively unlikely) circumstances the improvised solution can fail and then you get a socket with just IPv4.

You can write data to a UDP socket just like a TCP socket or a buffer, and then send it using udp_send(socket, ip, port). In contrast to tcp_connect, udp_send doesn't take hostnames as parameter. That's because tcp_connect only runs once per socket, so the added weight of the hostname lookup isn't as big, but udp_send it usually called a lot, and attempting to look up a hostname every time could lead to a big queue of lookups that clog the entire socket. Of course, that means a new function is needed to manually resolve hostnames to IPs. This is one of the things which haven't been written yet.

udp_send pushes the current content of the sendbuffer into the send queue, and the packets are generated from that queue in the background worker thread. Similar to TCP buffers, you can set a maximum size for that send queue to prevent it from growing indefinitely if you attempt to send data faster than it can be pushed out to the network. If the queue is full, attempting to send another packet will simply throw that packet away.

On the receiving end, there is another queue where newly received datagrams are placed by the worker thread. Again, in order to prevent memory problems the size is limited, and if too much data is received without being read by the application, packets will be thrown away. The size is fixed to 2MiB at the moment which should be plenty, but I will probably add another function to allow tweaking it.

udp_receive(socket) transfers a datagram from the receive queue into the receive buffer. In addition to reading the data, you can also query the source of the datagram which is currently in the receive buffer by using socket_remote_ip(socket) and socket_remote_port(socket). You can also ask about the port that the socket itself is bound to by using socket_local_port(socket), which is useful if you let the OS choose the port number. socket_remote_ip and socket_remote_port work for both TCP and UDP sockets, socket_local_port works for those as well and additionally for acceptors.

I probably won't be able to work on this in the next few days, so... patience please. I'll finish it. Eventually.

[death-droid]

Sounding really good there man, Im glad you didnt end up reiving IPv6 support for WinXP.

EDIT:
One problem i seem to be having, the send buffer and recieve buffer just seems to grow and grow without anything actually happening

Edited by death-droid, 11 July 2011 - 11:52 AM.