379 replies to this topic

[jon sploder]

Excellent, very excited for the new version. Thanks a lot Medo!

EDIT 2: The problem to the below might be due to this: For some reason my client can only send a message in it's create event: right after I use tcp_connect to get a tcpsocket handle. No other messages can be sent outside this event, unless immediately beforehand I reset the tcpsocket handle with another tcp_connect... Any ideas what I'm doing wrong or how to fix it? I'll post the code if you want it but there is a fair bit of junk in it to fill in reminders for myself for later work.

Edit: I have the scripts:

/*
write_string_delim(socket,string);
argument0: socket
argument1: string will be given a delimiter automatically
*/
write_ushort(argument0,string_length(argument1));
write_string(argument0,argument1);

and

/*
read_string_delim(socekt);
argument0: socket
*/
var length, string;
length = read_ushort(argument0);
string = read_string(argument0,length);
return string;

I'm not sure what's going on because the code:
write_string_delim(argument4,mac_addrs());
will not allow the socket to send it's data, but this returns no errors via the extension precaution function... I'm unsure whether it is the mac address function somehow screwing it up or my code, although it must be noted that my write_string_delim works for every other string I've tested. show_message(get_macadr) returns a proper string mac address so I'm stumped.

Edited by jon sploder, 13 February 2012 - 07:58 AM.

[Medo42]

Without further info there is not much I can say about this. It seems unlikely that the mac_addrs() function would cause sockets to misbehave, because the code for mac_addrs() has no interconnection at all to any other code in the library except for being linked into the same dll.

As for what does go wrong, you could use Wireshark to figure out what actually happens between client and server. That should give you a good starting point for narrowing down the problem in the code.

[jon sploder]

I found the problem; I was accepting sockets per step in the server, so if I made it only accept once the connection worked. I remember I did this for 39dll and it worked fine (was required). Where am I meant to put the acceptor handle code? (It's not every step)
Current snippet from the code that didn't work:

tcpsock = socket_accept(acceptor);
        datagram = tcp_receive_available(tcpsock);
        if (datagram)
        {
            //check to see if anyone is connecting from the 'wrong' port
            if (socket_local_port(tcpsock) != global.tcpport)
            {
                socket_destroy_abortive(tcpsock); //ungracefully close the socket;                
                chat_add("___someone connected from the wrong port."+string(socket_remote_port(tcpsock)));
            }
            var msgid;
            msgid = read_ubyte(tcpsock);        
            script_execute(global.tcpscript[msgid]);
        }

Changed to :
if (!tcpsock) {tcpsock = socket_accept(acceptor);}
To work.

Edited by jon sploder, 14 February 2012 - 08:31 AM.

[Medo42]

There are several things wrong with this code:
- You attempt to accept a new socket every step and overwrite your old socket with that. I have no idea what you are trying to do there. Typically, a TCP server uses one socket for every connected client, so if you want to support multiple clients, you need to keep track of the socket for each one seperately.
- The code assumes that tcp_receive_avaliable will give you a complete message of some sort - but that's not guaranteed. It will just give you *some* number of bytes. You should not make assumptions about how and where TCP packets will be split up or combined, otherwise your protocol will break under load and in real use, even if you get your data exactly in the neat packets you sent if you test on a LAN.
- The check whether someone is connecting on/from (seems to be inconsistent) the wrong port is useless. If it is supposed to check whether the local port is correct, that can never fail because sockets can't accept connection attempts to a different port. If it's supposed to check if the remote port is correct, you'll have a hard time using this server with Faucet Networking because it has no way to specify a source port with tcp_connect. It's just something that is almost never needed (in fact I can't think of a good reason why a protocol would need it).

[jon sploder]

Surely I must always look out for new connections, and the way to do it is constantly accept new sockets? My server has one object to do this (handle login and registration) and makes a new object for each new connection, and remembers the 'tcpsock' variable on receiving a messageid for login/reg... sorry my code wasn't very revealing because the making a new object and remembering tcpsock is in the script_execute part.

Would this code be a better server loop for receiving?

while (tcp_receive(tcpsock,bytes))
{
    if (byteisheader)
    {
        bytes = read_ushort(tcpsock);
        byteisheader = 0;
    }
    else
    {
        var msgid;
        msgid = read_ubyte(tcpsock);
        chat_add("___msg debug id "+string(msgid));
        if (variable_global_array_get('tcpscript',msgid))
        {
            script_execute(global.tcpscript[msgid]);
        } else {chat_add("___a message id was sent that the server does not know");} //debug
        byteisheader = 1;
        bytes = 2;
    }
}

(tried to be similar to your example).

I still am not sure where to put the tcpsock acceptor definition though...

EDIT: Nevermind for now, I just saved all previous sockets into a loop. I'm not really confident with anything I write in this though. If, when I'm done, I post the source to my first attempt to use this extension could you have a quick look at try and tell me any ineffeciencies or things that just plain don't work (like before with the remote_port thing... I assumed that was what it was for so I just put it in)? It'd be pretty helpful for me and others, as I'll probably make most of the same errors that other people used to 39dll will make.

Edited by jon sploder, 16 February 2012 - 06:37 AM.

[wnsrn3436]

In my opinion. When you upload your source will be processed quickly.

[jon sploder]

Just checking with you medo, does this seem reasonable:

tcpsock = socket_accept(acceptor);
if (tcpsock >= 0)
{
    sockets += 1;
    socket[sockets] = tcpsock;
    chat_add("new client(socket): "+string(tcpsock)+". total sockets: "+string(sockets));
}
var i;
for (i = 1; i <= sockets; i += 1)
{
    while (tcp_receive(socket[i],bytes))
    {
        chat_add('received a byte');
        if (byteisheader)
        {
            bytes = read_ushort(socket[i]);
            byteisheader = 0;
        }
        else
        {
            var msgid;
            msgid = read_ubyte(socket[i]);
            chat_add("___msg debug id "+string(msgid));
            if (variable_global_array_get('tcpscript',msgid))
            {
                script_execute(global.tcpscript[msgid]);
            } else {chat_add("___a message id was sent that the server does not know");} //debug
            byteisheader = 1;
            bytes = 2;
        }
    }
    //chat_add("for: "+string(i)+" "+string(socket[i]));
}

[Medo42]

Just checking with you medo, does this seem reasonable:

tcpsock = socket_accept(acceptor);
if (tcpsock >= 0)
{
    sockets += 1;
    socket[sockets] = tcpsock;
    chat_add("new client(socket): "+string(tcpsock)+". total sockets: "+string(sockets));
}
var i;
for (i = 1; i <= sockets; i += 1)
{
    while (tcp_receive(socket[i],bytes))
    {
        chat_add('received a byte');
        if (byteisheader)
        {
            bytes = read_ushort(socket[i]);
            byteisheader = 0;
        }
        else
        {
            var msgid;
            msgid = read_ubyte(socket[i]);
            chat_add("___msg debug id "+string(msgid));
            if (variable_global_array_get('tcpscript',msgid))
            {
                script_execute(global.tcpscript[msgid]);
            } else {chat_add("___a message id was sent that the server does not know");} //debug
            byteisheader = 1;
            bytes = 2;
        }
    }
    //chat_add("for: "+string(i)+" "+string(socket[i]));
}

Yes, that looks better, but I still see a problem: byteisheader and bytes are shared between all sockets, so if the last thing read from one socket is the header (again, unlikely on a LAN) the next socket will attempt to receive a message corresponding to the header of the previous socket. To make this work, you need byteisheader and bytes for every socket. In Gang Garrison 2 every client has its own object (Player) which holds the socket and the current protocol state (commandReceiveState and commandReceiveExpectedBytes - basically the same as your byteisheader and bytes). I do realize that it's awkward to manually keep track of this state, and that it is something many games need. Even 39dll has some support for automating this with its message formats, so this is definitely something I want to address some time. No promises on when that will happen though.

Edited by Medo42, 16 February 2012 - 03:12 PM.

[jon sploder]

Thanks.

write_buffer_to_file() doesn't work with sockets as the buffer for me. Can you replicate this? And/Or buffer_size doesn't work with sockets. My code to show this is:

var buffer;
buffer = buffer_create();

write_ubyte(buffer,8);
write_string_delim(buffer,argument1);
chat_add("buffer size: "+string(buffer_size(argument0)));
buffer_clear(argument0);
write_ushort(argument0,buffer_size(buffer));
write_buffer(argument0,buffer);
chat_add("buffer size: "+string(buffer_size(argument0)));
socket_send(argument0);

buffer_destroy(buffer);

Both times it returns a buffer size of 0. See if you can replicate this yourself.

Edited by jon sploder, 19 February 2012 - 02:49 AM.

[wnsrn3436]

Thanks.

write_buffer_to_file() doesn't work with sockets as the buffer for me. Can you replicate this? And/Or buffer_size doesn't work with sockets. My code to show this is:

var buffer;
buffer = buffer_create();

write_ubyte(buffer,8);
write_string_delim(buffer,argument1);
chat_add("buffer size: "+string(buffer_size(argument0)));
buffer_clear(argument0);
write_ushort(argument0,buffer_size(buffer));
write_buffer(argument0,buffer);
chat_add("buffer size: "+string(buffer_size(argument0)));
socket_send(argument0);

buffer_destroy(buffer);

Both times it returns a buffer size of 0. See if you can replicate this yourself.

The socket is different from the common buffer.
The socket send/receive buffers is divided.
In my opinion. You should use the following functions.

socket_sendbuffer_size(socket) : size
socket_receivebuffer_size(socket) : size

Edited by wnsrn3436, 19 February 2012 - 10:09 AM.

[Medo42]

I can see that I made some mistakes there. First, I don't think there's an actual reason why write_buffer_to_file() shouldn't work on sockets (though if you look at the documentation, it explicitly says "buffer", so it's at least consistent).

Second, it was possibly not a good idea at all to have the write/read functions accept sockets directly, because this makes sockets work in some places where buffers work, but not in others where you might expect it, and where it does work the semantics are slightly different, all of which makes this feature somewhat confusing. It is easier in some ways, but more complex. And while it does save some code in small examples, if I consider Gang Garrison most of the networking uses explicit buffers anyway or could be changed to use them with very little extra code.

Other things that I'm considering to change include
- the lenient behavior regarding e.g. invalid handles (replaced by some way of logging / error handling)
- the names of the read/write functions to reduce confusion of type lenght (read_ushort->read_u16, read_double->read_f64)
- remove the buffer endian awareness (replaced by explicit-endian read/write functions, e.g. read_u16l, read_u16b). Not sure what to make the default there; big-endian is the "standard" network byte order, but little-endian is what 39dll does. Maybe there could be no default at all and you always have to specify l/b... at least it would force people to consider what they are doing, and if you stick with one standard it shouldn't take any mental effort to always add the same suffix.

Don't worry, those are just ideas for version 2, aka the great big redesign. If/when I get around to working on that, version 1 will still be supported.

[Ronchon le Nain]

Greatings.
I'm now testing Faucet. Congratulations for the help file, it's really helpfull and well explained.
One small question : is there an equivalent to buffer_bytes_left , but for the receive buffer of a socket, to know how many bytes are still remaining to be read ?

[jon sploder]

Thanks wrsrn and Medo, yeah I saw that it said explicitly buffer but assumed because some other buffer functions can be substituted successfully that it might be do-able. Those ideas seem interesting.

[Medo42]

Greatings.
I'm now testing Faucet. Congratulations for the help file, it's really helpfull and well explained.
One small question : is there an equivalent to buffer_bytes_left , but for the receive buffer of a socket, to know how many bytes are still remaining to be read ?

Thanks. I just checked in the code, there isn't. It's probably just something that never came up before.
I just changed the code around a bit so that buffer_bytes_left, buffer_set_readpos, write_buffer_to_file and append_file_to_buffer now all accept sockets too. This means udp_send is one of the few functions remaining which accept buffers but not all sockets - unfortunately, it already has a particular meaning for being called on udp sockets with very different semantics. How did I get the idea again that this kind of punning was a good idea?
The only other remaining functions which don't accept sockets are those where the meaning would be unclear: buffer_destroy, buffer_clear and buffer_size.

Edited by Medo42, 19 February 2012 - 10:00 PM.

[Medo42]

Here you go: Version 1.4
- Enabled more buffer functions to use sockets: buffer_bytes_left, buffer_set_readpos, write_buffer_to_file and append_file_to_buffer

Download

[wnsrn3436]

Right.
I think it would eliminate the confusion that feature.
The new version is very good.

Also, I should hope.
1. I think you need to add more files to function.
2. Encryption function of the buffer.
3. Whether the message is received. (function).
4. Get my ip. (Lan ip, Wan ip, ipv4, ipv6).
5. Get my hostname.

In the case of 3. it useful for sending exit messages.
The most similar code :

write_byte(socket, 0)
socket_send(socket)
while(socket_sendbuffer_size(socket)){} // Whether the message is received.
socket_destroy(socket)

In the case of 4.
The most similar code *Script* get_my_ip(0=Lan ip/1=Wan ip) :

/*Do not use it!
Obtaining WAN IP is unstable.*/
if !argument0{return mplay_ipaddress()} //Lan ip
else
{
    var skv_soket, skv_string, skv_buffer;
    skv_soket=tcp_connect("whatismyip.org", 80)
    if socket_has_error(skv_soket){socket_destroy(skv_soket); return ""}
    write_string(skv_soket, "GET / HTTP/1.0"+chr(13)+chr(10)+chr(13)+chr(10))
    socket_send(skv_soket)
    skv_buffer=buffer_create()
    while(!tcp_eof(skv_soket))
    {
        sleep(10)
        tcp_receive_available(skv_soket)
        write_buffer(skv_buffer, skv_soket)
    }
    skv_string=read_string(skv_buffer, buffer_size(skv_buffer))
    skv_string=string_delete(skv_string, 1, string_pos(chr(13)+chr(10), skv_string)+2)
    skv_string=string_delete(skv_string, 1, string_pos(chr(13)+chr(10), skv_string)+2)
    socket_destroy(skv_soket)
    buffer_destroy(skv_buffer)
    return skv_string
    //Wan ip
}

Edited by wnsrn3436, 03 March 2012 - 08:40 AM.

[Medo42]

1. I think you need to add more files to function.
2. Encryption function of the buffer.
3. Whether the message is received. (function).
4. Get my ip. (Lan ip, Wan ip, ipv4, ipv6).
5. Get my hostname.

In the case of 3. it useful for sending exit messages.
Methods now available. :

while(socket_sendbuffer_size(socket)){}

You think I need to? That... doesn't quite sound right.

Anyway, as for 1 and 2 I'd prefer not to add more features not directly related to networking and rather address these things in version 2, which will support some standard for efficiently handing binary data from one extension to another, so that this kind of functionality can be provided by other extensions.

As for 3, this is something the socket API simply does not provide as far as I am aware. However, a small extension to the semantics of socket_sendbuffer_size() would allow you to determine if everything has at least been sent in the way you show. Actually what it would indicate is even weaker, i.e. if everything has been written to the operating system for sending, but that should be enough to ensure everything is sent on exit. Currently, this approach might work, but I'd have to make sure and make the specification more clear on that point.

4. is partially covered by socket_local_ip. Note that your machine can have several IPs, so just asking for "the LAN IP" does not always have a single answer. There is a way to enumerate the IP configuration of the networking adapters, but I think there is simply not much need for providing it (please give a specific example of what you want to achieve). There is no lowlevel way to determine your WAN IP.

5. That would be trivial to add, but what is it needed for?

[wnsrn3436]

4. is partially covered by socket_local_ip. Note that your machine can have several IPs, so just asking for "the LAN IP" does not always have a single answer. There is a way to enumerate the IP configuration of the networking adapters, but I think there is simply not much need for providing it (please give a specific example of what you want to achieve). There is no lowlevel way to determine your WAN IP.

5. That would be trivial to add, but what is it needed for?

​I'm just trying to get IP address with lookup function using hostname​.

Ps. socket_local_ip->socket_remote_ip?

Edited by wnsrn3436, 20 February 2012 - 01:44 PM.

[Medo42]

​I'm just trying to get IP address with lookup function using hostname​.

Well, I guess that's a way to do this

Ps. socket_local_ip->socket_remote_ip?

I'll be happy to answer questions, but please make the effort of asking them in plain english.

[jon sploder]

Medo, I'm trying to write a script that passes a message on to all other clients, ie ClientA -> Server, Server -> Client[B,C,D...].
My script: (for now, only assume tcp, I'm not using udp but that's old code just for when I do include both protocols)

/*
pass_packet(all | individual_socket,source_socket,protocol,ip);
argument0: all being 0, otherwise an individual socket is assumed (tcp or udp)
argument1: the socket or buffer being copied
argument2: udp/tcp, udp being 0, tcp being 1
argument3: if script is not called from obj_player or obj_server, this argument specifies ip for individual_socket
*/

if (argument3 != "")
{
    var ip;
    ip = argument3;
}

//pass on to all other players
if (argument0 == 0)
{
    with (obj_player)
    {
        if (id != other.id) //I know this won't work locally, but I've taken that into consideration when testing
        {
            if (argument2) //send tcp
            {
                chat_add("passing a tcp packet from "+string(argument1)+" to "+string(tcpsock));
                //write size of packet bytes and then message id etc.
                write_buffer(tcpsock,argument1);
                socket_send(tcpsock);
            }
            else //send udp
            {
                udp_send(argument1,ip,global.udpport);
            }
        }
    }
}
//pass on to an individual socket
else
{
    if (argument2) //send tcp
    {
        write_buffer(argument0,argument1);
        socket_send(argument0);
    }
    else //send udp
    {
        udp_send(argument1,ip,global.udpport);
    }
}

I know this isn't correct, I wrote it a while ago (several pages of explanations ago), but I'm not sure how I should write it after the 'if (argument2) //send tcp' segment... could you help me out with an example script to pass a packet on from one players receiving tcp socket, to another players sending tcp socket.