Jump to content


Photo

Creating Ai That Learns


  • This topic is locked This topic is locked
1775 replies to this topic

#1 Solidus

Solidus

    GMC Member

  • GMC Member
  • 552 posts
  • Version:GM:Studio

Posted 09 April 2006 - 07:53 PM

Creating AI that learns from interactions with whomever is playing the game it is in. It sort of works such as a chat bot, taking information that the user has inputted and using it in responses.

Chat Bot Ex.:

AI: What is your name?
You: Chris
AI: Hi Chris!

However simple this may seem, I believe it could be applied to a game such as chess in order to learn opponent's strategies. When playing a game a of chess, the computer evaluates it's opponent's moved based SOLELY on what it's creator has programmed it to do. This, in short, is the downfall of chess AI. While a human opponent may learn it's mistakes, a chess board will continue making it.

But, let's say the AI is created in game maker, what if you create an external database in which all the AI's strategum's are confined. However, instead of being an isolated, unchangable map of what the computer can do, it would take in consideration each loss it encounters, by adding which move led to it's own checkmate (if course, still talking of chess, although this could be used in more advanced shooting games.)

For example, if the Queen, considered the most powerful of the chess pieces, is taken from our AI, but no other feasable moves could have been made during that turn, it must go back through it's moves (saved in the data file) and find which move led to it's queen's taking, and change the strategy in which it had to suit itself for the mistake.

I feel as though I should add I am somewhat of a chess connoisseur, and this is obviously very, very advanced, especially for most people who use game maker. However, the idea intrigued me after reading another post about creating standard chess AI (which is not as complicated as they made it seem, which I will explain later on in my post) and I decided that it could also be used not only for chess, but for top-down shooters, or if you are someone such as TGG, a master-if-nothing-else of the use of 3D in game maker, FPS (I will not mention 3rd person games, for they are quite complicated and not very developed in game maker as of now.)

Ways in which this may be implemented in games other than chess are:

-AI learning how the player fights.
-If a punch, kick, upper cut combo is used mutliple times, the AI may learn to block at a certain time and devise a counter-offensive.
-In FPS, if the player seems to aim for the head shot, learn to duck
-Any other practical explanation.

I dare not bore you with more examples, because I am sure if you have read this far you have many in mind.


Explaining simple chess strategies.


It would take a long time to devise your own hand crafted list of moves, but that is how it must be done. If a player moves a pawn to a certain point, and you have programmed in a counter-offensive (I say counter-offensive to illustrate the reasoning that "defense" in chess leads to your opponent controlling the board) that makes the AI move their piece to a certain position. Of course, though, then you would have to make a strategy for every single variation that a chess board could take on. Obviously this very layman's way of explaining an AI's chess board strategy, and it is so long it is impractical, which is why I suggest allowing the AI to build it's own database.

Allowing the AI to forumulate it's own database from scratch


As proposed before, the AI learns new strategies from the player it is playing against. However, in this situation, the computer starts out making random moves. This would take pathfinding, such as checking if a piece is in the way, and would be quite simple to create. Each time the computer inevitably loses, it stores it's game information in it's external database, and each time you replay the computer, it reloads it's cumulative strategy report, and uses it's information to devise strategies against you. Suppose you try and go for the 4-move check mate. (link for those who care, it's the top strategy: http://www.avlerches...trategies.html) The computer would presumably be defeated in 4 turns provided it doesn't accidentally counter it. But, the next time you go to do this same thing, the Ai searches it's database, recognizes the moves, and relays it back to do something different. This would, of course, take along time to develop, due to the fact that there are so many different variations in the game of chess for each strategy. It's be much simpler to do this with Shooting games, due to the fact that it would be a lot less information to deal with.

Edited by xot, 13 April 2008 - 05:58 AM.
** Old Experts Topic **

  • 0

#2 THE_oldy

THE_oldy

    GMC Member

  • New Member
  • 18 posts

Posted 22 April 2006 - 08:02 AM

You often hear people say that AI can’t learn. I can understand how they would think this. When they think of an AI acting in such a way that it is learning (ie. if it always dies from head shots it starts ducking) then they think of a code such as:
if (deaths_from_head shot > 10) {script_duck_tactics()}
Which is often how this type of "learning" is done, and i agree; this is not learning.

However one must take a think about what learning, in humans actually is. It’s more or less recognizing and remembering patterns to control *cause and effect*. In my opinion true learning is not some magical holly grail of programming, its simply allowing the AI to recognize patterns, and the any *cause and effect* associated with it.

I would approach it like this:
Have the AI track EVERY variable that is possible for a human to recognise on their screen. These would be pre-programmed. I hear that "But your starting to program what the AI learns" bell ringing, but remember: a human playing the game can only watch and learn from what he sees on his/her screen, and what he/she sees on the screen is predefined (or programmed if you will) by the game makers, and yet, players still learn.
Variables that the AI could track could include the following for example.

Kills
Enemy position
Enemy weapon
My weapon
My position
ect...

Now in order to learn it needs to have a goal. (ie. maximise its score, minimise the enemy’s score) Also you must pre-program small scale goals and small scale patterns, ones that a human would realize before even playing (ie the a human is programmed to know before it starts playing) such as to get allot of kills, you need to get lots of individual kills, or even more basic, you need to aim at enemy to kill them. Basically you make the bear bones of a standard "non learning" AI

Then at specific times, and say whenever it gets a kill, or some other definite *effect* it records the *cause* - the variables at that time. Over time it will eventually have recorded a lot of sets of variables for the one *cause*, at which time it can start to recognize patterns and similarity in variables at a specific *cause*

These would become its secondary level goals, to learn how to cause the variables to be at the values it has associated with desired *causes* and how to force the variables to avoid the values that they were at when undesired *causes* happen.

IE. The AI might recognize that when every time it dies in a fight, it always seems to be standing still - so it begins to try to not stand still in a fight.
or
It might realize that a high number of its deaths occur when it was at the top of the sniper tower - so it tries to avoid its position variables from equalling "top of sniper tower"

And for controlling the acquisition of these second level variables it would go though the same process of learning as it did for the first set and so forth, the amount of levels of deepness that it could be limited by you, and in a humans case is limited by how smart they are.

Then you just let the AI play its self, play and play and play. You could probably set the room speed of the game to really high, because it doesn’t effect AI. It would be an interesting process to watch.

Of course this would take an extremely long amount of time unless you help it out a bit. You could let it know how to acquire the variables it wants.
ie. say it learns that it wants to be holding a certain weapon (it wants weapon_type=5 or whatever) you could give let it know that this weapon spawns at this point on the map, and lets its path finding take care of the rest. It could learn this from guess and check by "pressing" the pick up weapon button everywhere until it learns were it spawns but you can speed it up.

Of course you would have to program a bit of randomness into its actions (or human error) so it tries new things, and decides if their out comes are good or not.

Once the AI is good enough it’s ready for the game, but you would of course leave all its learning routines going.

The examples I have written here are for say an advanced FPS, so it would be a much simpler process for a simpler game. I might make an example of an extremely simple soccer game with AI that learns the way I have been describing.
  • 0

#3 Atlantis_base

Atlantis_base

    GMC Member

  • New Member
  • 188 posts

Posted 22 April 2006 - 02:40 PM

The idea itself is pretty promising, took me a few minutes to come up with an actual concept (fitting for a shooter for example)...

Normal AI works it's way around a map via waypoints, players do it different: They get to know the map and find their personal favorite and weak spots in a map through time. An AI is able to do this, too. My idea is a data grid for that.

Example of a map:

#####
#####
#####
#####
#####

Now we assign every place a value, to keep it simple I'd say we don't assign every coordinate a single value, instead we use one value for let's say each 32x32 tile.

Okay, we begin with a neutral value for every spot in the grid. Let's say 100 means "perfect place to go", 0 means "stay the **** away". So on a new map, we start with 50 for each, okay?

Now if the AI is getting hurt or dies on a certain spot, we decrease its value in the grid. If it doesn't get hurt, we increase the value. Also, if we pick up a bonus item or something, we increase the value at that spot (item spawn points are generally useful, we see people flocking the quad damage and rocket launcher spots in Quake3 all the time, eh?)

So if the AI plays for a while, it will find good and bad spots on the map and change them dynamically when players use other strategies.

Example:

#####
#####
#####
#####
#####


(One might want to set solid areas like walls to 0 from the beginning on to keep the AI from running into walls. Heh.)



Greetz, Atlantis

Edited by Atlantis_base, 22 April 2006 - 02:43 PM.

  • 1

#4 Finland Games

Finland Games

    Finland Games

  • New Member
  • 1034 posts

Posted 22 April 2006 - 02:54 PM

AI for a RTS game could also be something like:

if the number of player_infantry>player_antitank
create 10 tanks

or

if player anti air nnext to factroy<5
send 3 bombers to bomb factory


(of course this isnt code, i just didnt feel like spending the time to write actual code)
  • 0

#5 Atlantis_base

Atlantis_base

    GMC Member

  • New Member
  • 188 posts

Posted 22 April 2006 - 02:58 PM

AI for a RTS game could also be something like:

if the number of player_infantry>player_antitank
create 10 tanks

or

if player anti air nnext to factroy<5
send 3 bombers to bomb factory


(of course this isnt code,  i just didnt feel like spending the time to write actual code)

<{POST_SNAPBACK}>

No, that'd again be limited. It only reacts with a certain, preprogrammed behavior and that is what we wanted to avoid.



Greetz, Atlantis
  • 0

#6 webdogjcn

webdogjcn

    GMC Member

  • New Member
  • 418 posts

Posted 22 April 2006 - 05:03 PM

yes what they are saying is to make a way so that the computer can solve its own probelms by examining where it went wrong the first time...not just programming specified actions based on the player's.
  • 0

#7 Finland Games

Finland Games

    Finland Games

  • New Member
  • 1034 posts

Posted 22 April 2006 - 08:35 PM

hmm, then maybe like for a turn based strategy game:

if units killed in last move>5
{dont repeat last move}

is that what you mean?
  • 0

#8 webdogjcn

webdogjcn

    GMC Member

  • New Member
  • 418 posts

Posted 22 April 2006 - 09:25 PM

something along those lines, yet more complicated...for example, in a chess game, suppose a knight starts with 2 options for movement (he can go 1 of two places) and either place he goes will open up say 2 more possible movements. Well, say he gets taken out of the game, well the AI would analyze what moves led to the knights capture. Now, the AI, the next time it encounters a similar situation, will respond differently (a.k.a. learning from its mistakes). It's something that not even the programmer can 100% accurately anticipate. Whereas what you (Finland Games) are suggesting the programmer will know for sure what the AI will do. And this is why it becomes complicated...

Edited by webdogjcn, 22 April 2006 - 09:26 PM.

  • 0

#9 3t3rNAL_Payn

3t3rNAL_Payn

    The Human Genocide

  • New Member
  • 1096 posts

Posted 23 April 2006 - 04:44 AM

The idea itself is pretty promising, took me a few minutes to come up with an actual concept (fitting for a shooter for example)...

Normal AI works it's way around a map via waypoints, players do it different: They get to know the map and find their personal favorite and weak spots in a map through time. An AI is able to do this, too. My idea is a data grid for that.

Example of a map:

#####
#####
#####
#####
#####

Now we assign every place a value, to keep it simple I'd say we don't assign every coordinate a single value, instead we use one value for let's say each 32x32 tile.

Okay, we begin with a neutral value for every spot in the grid. Let's say 100 means "perfect place to go", 0 means "stay the **** away". So on a new map, we start with 50 for each, okay?

Now if the AI is getting hurt or dies on a certain spot, we decrease its value in the grid. If it doesn't get hurt, we increase the value. Also, if we pick up a bonus item or something, we increase the value at that spot (item spawn points are generally useful, we see people flocking the quad damage and rocket launcher spots in Quake3 all the time, eh?)

So if the AI plays for a while, it will find good and bad spots on the map and change them dynamically when players use other strategies.

Example:

#####
#####
#####
#####
#####


Greetz, Atlantis

<{POST_SNAPBACK}>

That is actuallly a great idea. However, it should be modified quite a bit still.
*If it notices that it kills alot of people in one spot, it's value should go up.
*If it notices that other people are having good luck in a spot, it should go up.
*If something goes up, the area around it should go up slightly.
*It should keep track of where the player is usually. If the player always goes in a spot, and the enemy finds a spot with a good shot on that area, that place should go up alot.
I think that you're idea could be very expanded upon, Atlantis
, and I think someone could go far with it.
  • 0

#10 timewarp

timewarp

    GMC Member

  • New Member
  • 440 posts

Posted 23 April 2006 - 04:51 AM

If you truly wanted an AI that could learn, it would have to create and modify its own code. This could be done using external files, and execute string, but it seems quite difficult to accomplish.
  • 0

#11 THE_oldy

THE_oldy

    GMC Member

  • New Member
  • 18 posts

Posted 23 April 2006 - 05:04 AM

Yes, exactly Atlantis_base. That’s a good way of implementing it.

Can some one make a basic top down multiplayer shooter, that we can try to implement this type of AI onto?
  • 0

#12 3t3rNAL_Payn

3t3rNAL_Payn

    The Human Genocide

  • New Member
  • 1096 posts

Posted 23 April 2006 - 05:31 AM

If you truly wanted an AI that could learn, it would have to create and modify its own code. This could be done using external files, and execute string

<{POST_SNAPBACK}>

Or, you could use add_event for user defined events, and then when you run out of room in those, you could save events are variables and keep having them replace each other and stuff.

TO TRANSLATE MY RAMBLE:

Ok, so you know add_event or whatever functions, right? Well, there are user defined events, 15 of them. Then, you can just have the AI create code for these events. But, let's say you get over 15 reactions that the AI has built, right? Well, it saves number 15 as the variable userdef15. The next as userdef16, and so on. Then, when use event 15 is called, you need to set userdefevent to 15, then it makes user defined event 15 to the variable userdef15, then executes it, then changes it back. Get it? It's confusing, but, with lots and lots of work, but it's possible to impliment this with the system that atlantis thought of and make awesome AI
  • 0

#13 THE_oldy

THE_oldy

    GMC Member

  • New Member
  • 18 posts

Posted 23 April 2006 - 07:24 AM

I dont understand the use of having new code written in game would help, or be a prerequisite to learning. Remember, were not creating Frankenstein’s monster, just something that can adapt. I just think it save infomation to a file containing what its learned. And this is added to and updated every time it plays.

A basic version of that file might look something like this.

Kills in grid square------Kills with weapon type
1,1 : 5 ------------------ 1 : 5
1,2 : 3 -------------------2 : 4
1,3 : 5 -------------------3 : 16
1,4 : 6 -------------------4 : 9
1,5 : 26 ------------------5 : 20
1,6 : 24
2,1 : 3
2,2 : 1
2,3 : 4
2,4 : 6
2,5 : 20
2,6 : 26 -------------------ect...

What we need is a really simple multiplayer game that some one has allready made, that we can try and write learning AI for.
  • 0

#14 hellavi

hellavi

    GMC Member

  • New Member
  • 22 posts

Posted 23 April 2006 - 09:46 AM

Creating this kind of AI for a computer is not easy, since we all agree that the computer will have to recognice paterns between Cause and event. The human brain is a true master in that and computers tend to have problems linking 2 actions together perfectly. But i've seen some good concepts here and it all looks promising.

Its reminded me of a documentary i saw about the futere of the world (lots of SF) but there was a part about some programmers who made a program with animals created out of blocks. The animals would then learn by themselves how to move and could evolve into later generations. They made a lillte game from it so the animals where put to test against eachother to first capture a block that was in the middle. After a few generations something spectacular turned oud, suddenly one of the animals atacked the other animal before they even reached the block; it had completely created this behavior out of scratch without any preprogramming. The programmers were astonished since they never anticipated how effeciently the animal had reprogrammed itself just like the human brain. If only we could achieve that here :D

When we would achieve something we could always test it eaily: Let 2 computer chars compete against eachother. If the system works good they should tie after a while :P

Edit: Made my post somewhat nicer and added something (don't mind my spelling pls :))

Regards Hellavi

Edited by hellavi, 23 April 2006 - 10:18 AM.

  • 0

#15 Sureyadid

Sureyadid

    GMC Member

  • New Member
  • 234 posts

Posted 23 April 2006 - 09:50 AM

I had a thought a while back that was like this.

If you make an external file called memory or something then create the code to write the players move and the computers reaction then read the file and compare the reactions
in order to ¨create ¨ a new reaction by the computer.

pseudocode-if(computer_shoots)
{
//write to file comuter shoot code//
}
if (player_ducks)
{
//write to file player ducking code//
}
if(computer shooting code is executed a number of times)
{
//read the memory file//
}
//adjust the height/direction the computer shoots to react to the players
previous reactions//
}

well at least it looked good in my thoughts
  • 0

#16 timewarp

timewarp

    GMC Member

  • New Member
  • 440 posts

Posted 23 April 2006 - 02:23 PM

I dont understand the use of having new code written in game would help, or be a prerequisite to learning. Remember, were not creating Frankenstein’s monster, just something that can adapt. I just think it save infomation to a file containing what its learned. And this is added to and updated every time it plays.

A basic version of that file might look something like this.

Kills in grid square------Kills with weapon type
1,1 : 5 ------------------ 1 : 5
1,2 : 3 -------------------2 : 4
1,3 : 5 -------------------3 : 16
1,4 : 6 -------------------4 : 9
1,5 : 26 ------------------5 : 20
1,6 : 24
2,1 : 3
2,2 : 1
2,3 : 4
2,4 : 6
2,5 : 20
2,6 : 26 -------------------ect...

What we need is a really simple multiplayer game that some one has allready made, that we can try and write learning AI for.

<{POST_SNAPBACK}>

If you were to give the AI the ability to write its own code in response to interactions, then it would truly be an AI that learns. It could remember what tends to work in its favor, then code a specific response to a situation. It would be much more flexible than the method you suggest, but also more difficult.
  • 0

#17 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 23 April 2006 - 02:33 PM

I agree with the "Cause and effect" method. We definatly want to avoid preprogrammed "avoid this if human does this". There is always a little glitch; and once a human finds the glitch, the computer knows no way how to avoid the effect.

MY EXAMPLE

Perhaps, before writing an AI base for an RTS for example, we would tell the computer what is good to do, and what is not. At first, the computer for an RTS game builds lots of useless buildings randomly or more than once. This is resource-eating and is something that the computer should avoid, unless very neccessary. Every move the human and computer does, is stored in an external database. Now let's say the human creates a huge strike force and crushes the computer. Of course, losing is a bad thing for the computer.

The computer now goes back into it's database and tries to find the cause for the bad effect it achieved. Perhaps if it saw that the human created a Training Hall building Terablasters to crush them, he might decide to go for that tactic instead. But to get the resources he needs, he needs to stop building useless buildings. He does so, but again the human crushes him, because the human had a bigger attack force.

The computer wonders again "What did I do wrong?" and searches back through the database the differences between how the human acted and how he did. He finds that the human used some of his resources to build Collectors in order to achieve more resources over time. Next game the computer tries this as well. He finds that this time he loses once again because he built too many Collectors instead of Terablasters. Next time, he tries a few less. So we get a balanced point somewhere.

When the human loses, and the computer thinks he has the ultimate strategy, the human changes tactic. The human monitors the computer's activity with some cheap Observers and finds he is building a force of Terablasters. The human knows that a equal force of Incemirators will easily kill the Terablasters. Suddenly, as the computer loses, he wonders what was the difference in the two player's actions. He finds that the Incemerators kill the Terablasters easily, and tries to achieve the same result as the player. Eventually the computer would come to a balanced point of excellent learning and strategy.

I think that this is the way a computer can learn how to handle a human.

FPS STRATICS

Ok, let's see here. Let's pretend that in our game there are two spots you can shoot at, either head or body. Head kills the player immidietly, while body might have to take about 20 shots before dying. Let's pretend we have three diffrent "players" in the game.

GAME MASTER A human that knows all the rules and tactics.
UNKNOWING HUMAN A human knowing nothing about the rules.
CPU The learning computer system, knowing no rules.

And the rules:

1. Headshot kills immidietly.
2. Bodyshots kill after 20 shots.
3. Ducking prevents headshots if such a shot is made.
4. Ducking while body shooting leads to a headshot :P

MATCH 1: UNKNOWING HUMAN VS. GAME MASTER
The master knows all rules and thus win the first matches easily. The unknowing human learns from his mistakes with a extremly powerful brain (compared to computer) and eventually might learn the tactics and win against the game master.

MATCH 2: CPU VS. GAME MASTER
Again, the CPU loses the first times against the master. Eventually he notices through his external database that the headshot is a very effective way of shooting. He begins to try to make all shots a headshot, but the Game Master knows that ducking will prevent that easily. As the computer loses again, because the vital shot misses, the computer will try to make headshots and duck if he is shot by a bullet himself. But if the game master notices that the CPU ducks all the time, he can simply aim at the body and get a headshot as so. Again, and again, the computer will learn the balances of strategy and analyzing.

Edited by jonah.schreiber, 23 April 2006 - 02:48 PM.

  • 0

#18 Freelancer Studios

Freelancer Studios

    GMC Member

  • New Member
  • 462 posts

Posted 23 April 2006 - 06:27 PM

jonah.schreiber, your ideas were pretty good, but I noticed that they were mainly about cpying the player move for move. I believe copying some of the player's actions is good, but there must be more to make an actual learning AI instead of a copycat. I'll try to make an example using some of the suggested methods.

FLS
  • 0

#19 znorre

znorre

    GMC Member

  • New Member
  • 145 posts

Posted 23 April 2006 - 08:35 PM

I tried testing some of the suggestions in this thread. Sorry for the messy code and lack of comments, will improve them later.

Right click, save as

Move using up,down,left,right.
Quit using esc.

When you move, the AI moves at the same time, in a random direction determined by the color of the adjacent blocks. After moving, both you and the AI automaticaly shoot straight forward. After 10 hits you die, score beeing displayed in the top left corner. if the ai hits you, the square it is in gets one notch better, if you hit the ai, the square it is in gets one notch worse.
the game autosaves when you quit.

You can also manipulate the ai by clikcking with your right or left mouse button on any square.

edit: To add something to the debate, i think that if you add multiple "layers" of square preferance, for example relative to the player (including the players direction), or relative to itself, you could get more advanced and faster lerarning. i'll try to incorporate it into my model later.

edit2: i'll also look into making it two AIs instead of AI and player (two AIs with sepparate grids, of course).

Edited by znorre, 23 April 2006 - 08:43 PM.

  • 0

#20 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 24 April 2006 - 08:31 AM

Znorre, I'll try your example as soon as I get home. Writing at school.

Freelancer: I agree, although the computer is merely copying to understand how to react to a certain situation. The part which I describe in the RTS is simply the person making the game teaching the computer the basics of playing. Otherwise as you say, copying is good at some times but should be avoided at others to avoid symmetry.
  • 0

#21 Erska

Erska

    GMC Member

  • New Member
  • 578 posts
  • Version:GM6

Posted 24 April 2006 - 12:09 PM

I would like to post my first expert users post :)
Well anyway I look at human learning as said here...observe things and remember things. Now some here seem to say that a program gotta reprogramme itself witch I don't agree with, as everyone has the basics imprinted from the time they are born, these are called reflexes. Reflexes could be simulated by making simple code that handles the keys, do this to move one up and such, then the AI would check what happens when it does a thing. A child copies its parents, why would it otherwise want to stand up?, so why wouldnt AI? :GM6:

That little ramble was a effort to explain that I think a "copycat"-AI is very much a learning-AI.

Now for the thing I would want to add :)
maybe planing made by the AI, meaning that it wont only react on the current situation but also want to try new things. Maybe not even have a ultimate goal that it strives for without any thought about new things, maybe consisting of new troop-combinations in RTS's. And then some factors of "panic" maybe when the AI is in a position of bad bad and bad move it would maybe try a more primitive approach.


Maybe have some kind of script for either experimenting or running. meaning that if a AI-animal would encounter snow for the first time it could first be afraid, try to dodge and stay away, fail and notice snow isn't of immediate danger, this would lead to the AI-Animal trying different things on the snow, maybe first checking for reactions of the snow when moved toward, touched attacked, shouted at, smell, warmth, each experiment the AI-animal would do would then filter through the old info (hmm this smells a litle like that other thing, that other thing tasted bad and I had a food poisoning out of it. This is bad-->stay away/be careful)
the experiment would also have safety values indicating what to try first, reflexes(pre-programing) could include dodging and attacking and then have more complicated variations added on basis of older info.
After the AI-Animal have noticed snow isn't dangerous for it it would more or less ignore it, but as soon as the snow would do something unexpected the experiment actions be triggered, but if the snow stops doing something unexpected the AI-Animal would get confused, witch would lead to some kind of awaiting for something to happen, if the thing is dangerous(that does the unexpected thing) the AI-animal would get scared rather than just confused and would run, attack or what ever goes with the AI-Animals personality.

So one would have a AI with pre-programed basic stuff, then the reactions would be based on observations of things from older games/events.
the pre-programed stuff would need to include basic stuff for every thing the AI can do, yes but The AI could advance learn and upgrade itself and its tactics. so basically you would give the AI a basic training, in moving, attacking, eating, teaching others(in AL :D). and so on
  • 0

#22 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 24 April 2006 - 12:39 PM

Honestly, we have to make a template for us to modify and play around with. I presume all posters here have a registered version of Game Maker, so that we can play around with the objects and functions.

I think that we should make a top-down template. It is the simplest way of controlling our in-time objects. Perhaps we could do as Erska suggests, an animal or something representing our main AI learner, and some objects it could find in the world. Such as snow :)

And let's remember to keep the AI as rooted as possible. We mustn't program every action the AI wants to take at a certain point. That would be almost impossible. For example:

if (killed_by_headshot > 10)
   {
    start_duck();
   }

Which is not rooted at all. We need to group all bad things and good things together, and either we preprogram what they are (eating dirty water is bad) or let the computer learn (I ate dirty water. It made me feel bad). This list needs to be put in an external database, from where it can be pulled up again. So lets say our AI animal finds some clear water. We might have something like this:

if (FoundObjectThroughDatabase("ext_database.lib",obj_water) == false)
   {
    // Experiment with the object
    WriteObjectToDatabase("ext_database.lib",obj_water));
    iResultFrighten = AttemptFrightenObject(obj_water);
    WriteStringToDatabase("ext_database.lib","obj_water frighten " + String(iResultFrighten));
    iResultSmell = AttemptSmellObject(obj_water);
    WriteStringToDatabase("ext_database.lib","obj_water smell " + String(iResultSmell));
    // And so on and so forth...
   }
It's a bad example, and not GML code (or any other code for that reason) but we might have to take this into account and see what results we can get.
  • 0

#23 znorre

znorre

    GMC Member

  • New Member
  • 145 posts

Posted 24 April 2006 - 01:41 PM

Created an example using two AIs.
Save as.

change speed with enter.
switch ai view between players with space.

I left this example running for seven hours at max speed, resulting in this:

top player - lowest score
Posted Image

bottom player - highest score
Posted Image
  • 0

#24 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 24 April 2006 - 01:55 PM

What does it do? I see bouncing arrows go around and occasially leaving behind a brown spot... what is the AI in this?
  • 0

#25 Juju

Juju

    GMC Member

  • GMC Member
  • 862 posts

Posted 24 April 2006 - 02:24 PM

What does it do? I see bouncing arrows go around and occasially leaving behind a brown spot... what is the AI in this?

And I quote: "I left this example running for seven hours". Why don't you release this as open source? This is an experts forum and in order for people to make better games, we need to see source code.
  • 0

#26 znorre

znorre

    GMC Member

  • New Member
  • 145 posts

Posted 24 April 2006 - 04:26 PM

There is a player vs AI version on the first page, in case you are posting in a thread you havent read.

I will eventually release the source, but it's currently noncomented and full of sloppy/redundant code, and i dont want that pinned to my name.
  • 0

#27 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 24 April 2006 - 06:09 PM

I see now :)

Not unexpected, for both players in all corners the color is completely red :)
  • 0

#28 osbornecreations

osbornecreations

    GMC Member

  • New Member
  • 460 posts

Posted 24 April 2006 - 06:45 PM

have you ever played quake? well search google for the "reaper bot". now that is some killer AI!!
  • 0

#29 Schreib

Schreib

    Valen Shadowbreath

  • GMC Member
  • 1455 posts
  • Version:Unknown

Posted 24 April 2006 - 06:56 PM

Here is a link to read about it: Sounds pretty nice :) http://www.bluesnews.../guide/bots.htm
  • 0

#30 znorre

znorre

    GMC Member

  • New Member
  • 145 posts

Posted 24 April 2006 - 07:34 PM

Yet another one! this time each square has four arrows pointing to its middle, representing how attractive it is to move into that square from that direction.




save as.


Controls:
Enter: Change room speed. 1-9999
M: DisplayMode. Choose which players AI to display.
R: "Reward". How much the value of a move increases when a player hits its enemy.
P: "Punishment". How much the value of a move decreases when a player gets hit by its enemy.

I also noticed that a positive effect of this model is that when most of a players AI grid is red (because it's losing), every change matters more, thus ensuring faster evolution for pressed players.

Edit: if someone could leave one on with a low reward/punishment rate (like 1 or 2), and the report the results, that would be great.

Edited by znorre, 24 April 2006 - 07:38 PM.

  • 0




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users