Exercises in Formula and Lua AI and AI-demos add-on feedback

Discussion of all aspects of the game engine, including development of new and existing features.

Moderator: Forum Moderators

Post Reply
nelson
Posts: 91
Joined: March 19th, 2008, 11:15 pm
Contact:

Re: Exercises in Formula and Lua AI

Post by nelson »

So here is the first of a series of test cases, meant to correspond to the list of Fred's weaknesses.

Today is a test case for:
(A) Defending and retaking/reclaiming the NW villages. Fred frequently allows the enemy to steal/take his NW villages, and then has lots of trouble making a concerted effort to clear them. Even once he has killed the units holding his NW villages, he often fails to reclaim them, allowing the enemy to keep their income.
It is meant to be a re-creation of turn 10 from this replay of Ron murdering Fred in an orc mirror.

Since that replay used an old version of Fred that does not have the CA debugging code included, you can't simply load that game to do CA debugging, you must instead painstakingly re-create the scene using a modern version of Fred. That is what I have done in about an hour of monkeying with debug mode. Hopefully future test cases will go easier!

Are we happy with Modern Fred's response to this test case? Old Fred failed to reclaim either of the two villages that blue currently holds, despite having 3 units in range to do that (including 2 grunts). Instead Old Fred moved all of the units on the western front as far east as possible, presumably to prepare to deal with threats to his leader, but I'm not really sure why. Fred had better take those villages back this turn! UPDATE: I used the CA debugger to step Fred through this turn, and he reclaims the two villages, but does not keep a unit in the forward village on 11,9 that the enemy archer is threatening to grab. So this behavior may be acceptable, but I think Fred might be able to do better? The problem is that Fred moves the wolf and grunt that are in front first, and the rear grunt cannot reach the forward village, so he just stays where he is instead of taking the forward village and meleeing the archer.

UPDATE: I posted the wrong file initially, the correct version is there now.
UPDATE 2: We made some changes to the CA debugger, so the old savegame will no longer work in the current Github version of Fred, so I have uploaded a new version that mattsc has fixed. Of course, this new savegame will not work in the currently released version of Fred, so I won't delete the old one yet.
Attachments
2p_—_The_Freelands_Turn_10_Test_A_take_4_fixed.gz
This fixed savegame will work for CA Debugging in the current Github version of Fred, but right now will not work in the officially released version.
(25.17 KiB) Downloaded 398 times
2p_—_The_Freelands_Turn_10_Test_A_take_4.gz
My 4th attempt at uploading, this time the AI even has the correct gold amount :P I didn't take into account that when CA debugging you don't get income from villages at the beginning of your turn.
(24.99 KiB) Downloaded 402 times
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

We just released AI-Demos v0.11.2 for Wesnoth 1.10 and v0.11.3 for Wesnoth 1.11 on the respective add-ons servers.

Note: As of v0.11.2, development of AI-Demos has been discontinued for Wesnoth 1.10. The reason is that it is becoming increasingly obvious that the next steps in improving the grunt rushes (and some of the other AIs) will be much easier with the new capabilities of 1.11; and in some cases, in fact, impossible in 1.10. To get the most up-to-date version, please use Wesnoth 1.11 from now on. We understand that this might be somewhat inconvenient for some of you at this time, sorry for that, but in the long run it will lead to better AIs. We will make sure that we will use capabilities of released versions of 1.11 only (currently that means 1.11.0), so it should never be necessary to compile the latest trunk version.

As for the current version, the refactoring of the Freelands Grunt Rush AI is completed. Fred now acts on a zone-by-zone rather than on a map-wide basis, which is much more adaptable both for the Freelands map and for eventual generalization. There might not (yet) be huge improvements to Fred's behavior, but on average he should at least not play worse than before.

Ron has seen some more improvements to the recruitment algorithm and is currently being tested against (and is standing up very well to) SeattleDad's ML recruiter. We are also talking about combining the two to see how that works out. All this is still work in progress.
Changelog v0.11.2 and v0.11.3

Code: Select all

0.11.3, 12 Oct 2012
- Original release of v0.11.3 for Wesnoth 1.11
  - Identical to v0.11.2, except for the "this is the last 1.10 release" messages

0.11.2, 12 Oct 2012
- ***** Important Note: This will be the last release of AI-Demos for Wesnoth 1.10 *****
  - Some of the next steps required at this point will only be possible with the new capabilities of BfW 1.11
  - Except for potential bug fixes, this will therefore be the last release of AI-Demos for Wesnoth 1.10
  - Further development will only be done for Wesnoth 1.11 and later; this will _not_ be backward compatible
  - We will make sure to use only capabilities of released versions of 1.11 (currently 1.11.0), so no compiling of trunk will be necessary
- Generic Rush:
  - New recruit algorithm, incl.:
    - Know about drain not affecting undead units
- Freelands Side 1 Grunt Rush:
  - New zone_control CA (completely reworked how several of the CAs function)
    - Works on a zone-by-zone basis, rather than map-wide
    - Several of the old CAs were combined into zone_control, acting in a more coordinated way (within a zone) in this way
    - This includes some changes to behavior, but nothing earth shattering
    - A bunch of bug fixes going along with that
    - Units are now sent into zones to capture villages, if there aren't enough units already
      - This is particularly important for the NW villages
    - Capturing of enemy-owned or unowned villages has higher priority on a zone-by-zone basis
  - Updated CA debugger
    - Now needs only a 1-turn initialization and turn number is reset afterward
    - New right-click option to let AI play through an entire turn
    - New right-click option to find the highest scoring CA
    - Simulation of self.data table as used by Fred during a turn
    - New right-click option to clear or show content of self.data table
    - Only show real evaluation functions in menu
    - Debugger now gives error message if used with Side 2 (instead of crashing Wesnoth)
    - Selected CA name is now shown in right-click menu
Chaos_Farseer
Posts: 2
Joined: May 7th, 2012, 2:13 am

Re: Exercises in Formula and Lua AI

Post by Chaos_Farseer »

Found a couple of quirks. (Wesnoth 1.11.0)

Bottleneck Defense: (AI vs. AI) The human AI sometimes tells its Bowmen to attack the Orcish Grunts in melee. Also, the Orc AI stopped moving at one point, despite having an extra four archers to attack with. As another note, both AIs eventually stop moving. That's bad, because their turns become less than a second long.

Prune Cart: (AI vs. AI, Stationary) The Orc AI frequently attacks skeletons with normal arrows, despite obviously doing more damage with fire arrows. This isn't just on low-health skeletons. The Orc AI still uses fire arrows with counterattacks.

HttT: The Elves Besieged: (Default settings) Konrad's AI agressively attacks the upper-right Orc army with everyone but Konrad and Delfador. Then, when the upper-left Orc army got nearby, Konrad and Delfador slowed down until the elves killed them all. Konrad's army should attack the Orcs in his way, instead.

Other than that, it's good to see that an AI can do something aside from charge and die.
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

Chaos_Farseer: Thanks much for the feedback! In all the scenarios where you witnessed incorrect weapon choices, you unfortunately found a bug that we had also found (and fixed) recently. We just released v0.11.5 which includes that bug fix, so if you want to go back to those scenarios, you will hopefully see more sensible behavior.

The AIs not moving any more at the end of Bottleneck Defense is not really a bug. It is caused by the orc side, played by the (unmodified) RCA AI, not attacking because it is deemed too dangerous. The pass defenders do exactly what they should in that case: hold the pass and wait. But it's still very frustrating to watch, so we increased the aggressiveness of the RCA AI, which will hopefully prevent a situation like this from happening in the future.

As for the HttT you observed, I've never quite seen it like that. You wouldn't happen to have a replay left from that that you could post here?

Again, thanks much for the comments!

--------------------------

We just released v0.11.5 in order to fix the bug reported by Chaos_Farseer. It contains many changes to both Ron and Fred (see complete changelog below). Unfortunately, it turns out that the changes needed to make Fred play better are more extensive than anticipated. So Fred's behavior is probably about as before, maybe a little better in some detail and a little worse in another. Hopefully we will have this ironed out by the next release...
v0.11.5 Changelog

Code: Select all

0.11.5, 4 Nov 2012
- Bug fix: Units now use correct weapons to attack again in all scenarios
- Generic Rush:
  - Many improvements to recruit algorithm
  - Significantly speed up grab_villages CA
    - *** One of the grab_villages changes makes return to BfW 1.10 impossible ***
  - Fix bug with leader movement when keeps are present on map borders
  - Better handling of split keeps
  - Improved movement to keeps when unclaimed villages nearby
- Freelands Side 1 Grunt Rush:
  - Attacks now take XP into account, discourage use of poisoners under certain circumstances
  - zone_control:
    - Incorporate attack_leader_threads
    - Zone shapes do not have to be rectangular any more
    - Leader now participates in zone attacks and other actions, if safe
      - Recruit first if he does so
  - Faster and more accurate counter attack calculation
  - Make use of new ai_helper functions and functionality
  - Lots of clean-up, optimization and refactoring: improves speed, functionality and configurability
- Bottleneck Defense: 
  - Make RCA AI side more aggressive, in order to avoid potential stalemate at the end
- Remove era "Experimental AI (adjustable parameters) as there currently are no adjustable parameters
- ai_helper:
  - Many changes and improvements to existing functions, and addition of new functions, incl.:
    - attack_combo_stats, attack_rating, get_attack_combos*, get_attacks*, get_attack_map*, simulate_combat*, done_eval_messages
- New function library battle_calcs.lua for battle outcome calculations: not used in an AI yet
User avatar
Alarantalara
Art Contributor
Posts: 784
Joined: April 23rd, 2010, 8:17 pm
Location: Canada

Re: Exercises in Formula and Lua AI

Post by Alarantalara »

Fred's first victory as Undead.
Attachments
2p_—_The_Freelands_replay.gz
(24.84 KiB) Downloaded 430 times
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

AI-Demos v0.12.0 is out with lots of exciting news and additions (well, if you find things like this exciting, that is):
  • SeattleDad's Machine Learning Recruiter and test tools are now part of AI-Demos. This is the main reason why we went to v0.12.
  • As Alarantalara hinted on in his post, Fred can now play any faction; although he's still optimized for chaotic factions (meaning that the attack/retreat cycle is horribly wrong for lawful factions; that's next on the list to change)
  • Ron has seen significant improvements, in particular to recruiting, which now results in an 86% win percentage against the RCA AI across all factions and all maps. As a result, Ron is now in mainline. Right now, you'd still have to compile trunk to make use of that, but he'll be a default part of Wesnoth starting from Wesnoth 1.11.1.
  • Fred has seen lots of improvements. He still makes too many stupid mistakes, but, as Northerners, has an average win percentage around 80% against Ron on Freelands. [We've changed both Ron and Fred since the last patch tests, so the exact current win percentage might be slightly different, but Fred clearly wins much more often than he loses to Ron these days.]
  • Not directly related to AI-Demos, we've also put in a patch that removes the 'testing' label from the RCA AI and declares it the default AI.
v0.12.0 changelog

Code: Select all

0.12.0
- New recruitment AIs
  - The AIs for the ML recruiter described at http://forums.wesnoth.org/viewtopic.php?f=10&t=36642 are now in AI-Demos.
  - This adds 3 new multiplayer AIs: The two from ML Recruiter 0.3 (Recommended and Less variety) and one that replaces Ron's current recruitment algorithm with the Recommended ML recruitment.
  - All these AIs require the ML Recruiter 0.4 patch from https://gna.org/patch/?3479 to be applied to Wesnoth before they will run
- New test harness and analysis tools
  - ml_utils contains a set of scripts to run and analyze large numbers of AI vs. AI games
- Generic Rush
  - Much improved placement of recruited units
  - The leader now captures villages rather than moving to a new keep if it can't recruit any more in the current turn
  - Replays no longer go OOS if the Generic Rush is playing a side
- Freelands Side 1 Grunt Rush:
  - Use Ron's recruiter instead of custom one
  - Fred can now play any faction
  - Switch to use of battle_calcs.lua instead of wesnoth.simulate_combat
  - zone_control:
    - Lots of adjustments to zone_control behavior to improve performance (this is still work in progress)
    - Move trapping and poison attacks into zone_control
    - Units now move off villages to allow injured units to move there
    - Add do_action and skip_action parameters
    - More refactoring
- battle_calcs is now a complete library independent of wesnoth.simulate_combat, including:
  - Fast (and cached) battle-outcome calculations (still some work to be done for specials like drain and berserk)
  - Calculation of single-strike damage and best weapon (includes terrain TOD and illumination)
  - New gold-based attack rating system
  - Correct HP statistics and attack rating for battle combinations
- New lua test scenario:
  - Can be accessed by moving Grnk to snowy mountains at 9,1 in Switchboard scenario
- Patrols:
  - Turn Jabb AI into a BCA, in preparation for making it a Micro AI
  - Create a separate side for Jabb and move engine into separate file (same reason)
- Protect Unit and The Elves Besieged:
  - Disable parameter selection dialog
- Micro AIs:
  - Some cleaning up and creating of templates in preparation for Google Code-in projects
- ai_helper.next_hop now has option to ignore own units that can move out of the way
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

Fred's first victory as Loyalists vs. Ron playing Northerners.

Fred's still making plenty of silly mistakes, but he can now play any faction and even win against Ron's strongest faction (not sure of the win % yet). Still restricted to Side 1 on Freelands though. (Note: this is the current development version on github, not v0.12.0)
Attachments
2p_—_The_Freelands_replay_Fred-Loyalists_Ron-Northerners.gz
Fred now knows how to play lawful factions
(38.35 KiB) Downloaded 499 times
User avatar
Crow_T
Posts: 851
Joined: February 24th, 2011, 4:20 am

Re: Exercises in Formula and Lua AI

Post by Crow_T »

Thanks Mattsc for expanding on the AI, pretty cool stuff happening here. This is probably in the wrong place, but I thought of a method that may help to control AI a bit- only in theory of course ;) It would involve weightpainting maps and using those weighted maps to do things like make AI avoid certain areas, protect certain areas, or follow specific routes. You could have multiple weight maps in one scenario map that control different groups/types of units, or random placement of units, or even things like aggression. I wonder how well that would work, or how useful it would be.
User avatar
Rigor
Posts: 941
Joined: September 27th, 2007, 1:40 am

Re: Exercises in Formula and Lua AI

Post by Rigor »

this is not at all wrong and only in theory: how else do you think Fred managed to be able to hold the west whileattackign the east? ^_^ its the previous knowledge that you should not lose village X and you should be careful about tod while in area Z.

btw, this replay looks like a mediocre player was beating the AI thoroughly. could you please upload this good stuff on the addon server so its easier to have the correct version? im looking forward to playing under harder circumstances again
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

Crow_T: Thanks. As for maps, the different AIs use all kinds of maps. There are sub-maps that simply split up the scenario maps into sections, such as what's happening on Freelands (as Rigor describes). Then there are maps that do ratings based on terrain, own and enemy strength, distance from something, etc. Some of them are pre-defined, some of them the AIs figure out themselves. Ideally, we'll eventually have an AI that can figure it out all by itself to some extent, but where the scenario designer has the option of adding pre-defined maps in order to improve play on a certain map. So I think that (some of) this is along the lines of what you are thinking. If you have more concrete ideas, by all means, please let us know.

Btw, AI-Demos is by far not just my work any more. There are lots of people contributing by now.

Rigor: well, the reason why I hadn't uploaded the new version yet is because that replay was kind of a lucky game. It was the second I tried (Fred lost the first). After that, going through some more games, it's pretty obvious that Fred's still making too many embarrassing mistakes. I started to work on that, but then the Google Code-in (GCI) project started on Monday and I've spent all my available time on that (converting AIs to Micro AIs that, hopefully, will make an appearance in mainline pretty soon). So Fred's still not quite where I'd like him to be ...

However, I know that you've been waiting for us to make these AIs play other factions for a long time -- we do listen to you (and others), it's just been very slow progress for some time. :) So I just uploaded v0.12.1 with the current "state of the art". Just remember that parts of Fred are still in a state of disarray (he puts units into too vulnerable positions, moves one hex off villages, that kind of stuff). Once GCI calms down, I'll get back to that.

Thanks for sticking with us in spite of all that. :D
v0.12.1 changelog

Code: Select all

0.12.1, 2 Dec 2012
- Freelands Side 1 Grunt Rush (Fred):
  - Adjusted time-of-day dependent cycles so that they work for any unit/faction
    - They are now based on relative damage that AI and enemy units can do, rather than on the alignment per se
    - This makes Fred better at playing lawful factions, but he's still not very good at it, at least not against Northerners
  - A number of small adjustments to the attack/move evaluation functions
    - Generally improves Fred's play, but still work in progress
- Google Code-in 2012 student tasks:
  - Conversion of lots of AIs to Micro AIs (including, in some cases, additional functionality):
    - Messenger Escort
    - Hunter (formerly Dragon)
    - Swarm
    - Wolves and Multi-pack Wolves
    - Big Animals
    - Forest Animals
    - Sheep
    - Patrol
    - Guardians (return, stationed and coward)
    - Protect unit
    - Lurkers
- Generic Rush (Ron)
  - Copy some improvements from Fred's poisoning to Ron
- Healer Support: exclude regenerating units from potential healees
- ai_helper:
  - get_closest_location: speed up function and add optional check for passability of terrain
  - get_attack_map: now takes units with MP>0 off the map for enemy path finding
  - New function serialize()
- battle_calcs: new function relative_damage_map()
Quitch
Posts: 69
Joined: January 10th, 2006, 2:32 pm

Re: Exercises in Formula and Lua AI

Post by Quitch »

mattsc, just worked my way through the whole thread and want to thank you for starting a real push to improve the Wesnoth AI even more. I don't play with humans only AI and I haven't lost a MP game to an AI yet, I always welcome developments in this space. What you're doing here is very exciting :)

Which version of Ron is being used in 1.11.1? 0.12.0?

Anyway, a few things:

1. On The Clash the version of Ron in 1.11.1 will not capture villages with his leader on the first turn, rather he moves towards the castle above/below him.

2. I'm sometimes getting the following error using AI-Demo 0.12.1:

stack traceback:
...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:540: in function 'find_best_recruit'
...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:408: in function 'recruit_rushers_exec'
[string "(...):recruit_rushers_exec()"]:1: in main chunk
20121228 03:37:35 error scripting/lua: ...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:540: attempt to index local 'efficiency' (a nil value)

I'm trying to track down the conditions which cause this. Once the error happens it occurs each turn thereafter.
User avatar
Alarantalara
Art Contributor
Posts: 784
Joined: April 23rd, 2010, 8:17 pm
Location: Canada

Re: Exercises in Formula and Lua AI

Post by Alarantalara »

Quitch wrote:mattsc, just worked my way through the whole thread and want to thank you for starting a real push to improve the Wesnoth AI even more. I don't play with humans only AI and I haven't lost a MP game to an AI yet, I always welcome developments in this space. What you're doing here is very exciting :)

Which version of Ron is being used in 1.11.1? 0.12.0?
Approximately. There may be minor differences though, since the releases are not at exactly the same time.
Anyway, a few things:

1. On The Clash the version of Ron in 1.11.1 will not capture villages with his leader on the first turn, rather he moves towards the castle above/below him.
Indeed. Most of the effort with leader movement has been spent dealing with multiple keeps on 2-player maps. Most testing on 4 player maps has been done on maps where the keeps of the players are well separated (Castle Hopping Isle, Siege Castles). I'll work on fixing this. Most of it is that the leader looks for new keeps even when the present one is a good option. This is good for maps like castle hopping isle, where you might otherwise run out of room to recruit, but it doesn't work nearly as well when the other castle is the ally's and there is plenty of space in the current castle.
2. I'm sometimes getting the following error using AI-Demo 0.12.1:

stack traceback:
...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:540: in function 'find_best_recruit'
...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:408: in function 'recruit_rushers_exec'
[string "(...):recruit_rushers_exec()"]:1: in main chunk
20121228 03:37:35 error scripting/lua: ...ata/data/add-ons/AI-demos/lua/generic-recruit_engine.lua:540: attempt to index local 'efficiency' (a nil value)

I'm trying to track down the conditions which cause this. Once the error happens it occurs each turn thereafter.
This shouldn't be a problem any more — the code associated with it was replaced about 5 days after the last release to fix a problem that looked much like what you describe.
Quitch
Posts: 69
Joined: January 10th, 2006, 2:32 pm

Re: Exercises in Formula and Lua AI

Post by Quitch »

I assume you mean not a problem any more in the current github version, rather than the addons copy, right?

Also thanks to you, and all the others, working on this as well. People like me greatly appreciate your efforts :)
mattsc
Developer
Posts: 1184
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Exercises in Formula and Lua AI

Post by mattsc »

Quitch: Thank you very much for the comments, that's great to hear.
Quitch wrote:I assume you mean not a problem any more in the current github version, rather than the addons copy, right?
That's what it did mean, but not any more. Alarantalara fixed the issue with The Clash as well, so: We just released AI-Demos v0.12.2

The biggest change this time is that, thanks to the work of the Google Code-in students, there are now 18 AIs available as Micro AIs, most of them with lots of configuration parameters so that they can be applied in a wide range of situations. These MAIs are almost ready for mainlining and should appear in Wesnoth trunk shortly.

There have also been lots of improvements made to the generic rush AI (Ron) and in particular to the rush recruiter (used by Ron, Fred, and the mainline Experimental AI). Unfortunately, Fred's seen no other progress due to the Micro AI work.
Complete changelog v0.12.2

Code: Select all

0.12.2, 28 Dec 2012
- Micro AIs
  - New Micro AIs:
    - Rush recruiting
    - Random recruiting
  - Additions to Micro AI behavior:
    - Improve patrol behavior when the waypoints are occupied
    - Add waypoints to Messenger AI
    - Wolves AIs aren't restricted to a pack size of three any more
    - All Micro AIs are now parametrized
  - Lots of cleaning up of the code, in order to get ready for mainlining the MAIs
- General
  - Updated scenario texts to represent changed behavior and that they are now Micro AIs
  - Switch to mainline maps and units as much as possible, some significant changes of scenario setups
  - Eliminate most macros
- Generic Rush (Ron)
  - Honour passive_leader AI aspect if set
  - Recruit minimum amount to capture all nearby villages turn 1 if there is a castle closer to the enemy that we will recruit from soon
  - Do not move to a new keep if unclaimed villages exist and the new keep isn't much better
- Rush Recruiting (algorithm used by Fred/Ron)
  - Time of day awareness
  - Account for regeneration
  - Increase isolation from parent AIs to improve reusability
  - Increase chance that all nearby villages are reachable by recruit
- ai_helper: new function get_passable_locations()
Quitch
Posts: 69
Joined: January 10th, 2006, 2:32 pm

Re: Exercises in Formula and Lua AI

Post by Quitch »

That's great news, I'll be sure to give it a whirl. I must admit, while I understand the purpose of Fred, improvements in Ron are much more exciting since I perceive him as the end-goal.

Incidentally, I don't know if this is on the goal list (didn't see it in the topic earlier), but a long-term goal would be smarter teamwork. Does the AI currently account for allied units when forming battlelines, etc.?
Post Reply