Help with testing Fred - Freelands MP Custom AI v0.14.15

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderator: Forum Moderators

Post Reply
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.9

Post by mattsc »

singalen wrote: March 5th, 2018, 6:32 am Sorry, it broke again on 1.13.12 - dev version 8(
attempt to call a nil value (filed 'maxn')
ca_zone_control.lua:2636: in field 'execution'
[string "local self, params, data = ..."]:2:in main chunk
Huh, that's strange. The message complains about Lua function table.maxn() not existing. That function was indeed deprecated in Lua 5.2 and removed in 5.3 (I think), but I don't understand why this starts producing errors now. I don't see anything in the Wesnoth changelog since the update to 5.3.4 in Wesnoth 1.13.10. Also, I didn't get that error in my own tests. I'm traveling and don't have access to a compiled version of current master right now though (will have one by the end of the week), so I've been testing with 1.13.11, so maybe something's changed since then that I am not aware of.

Well, whatever the cause, the fix is simple and has been committed. Thanks for reporting.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by mattsc »

So ....

I took a while off from working on Fred because RL was crazy, because I wanted to concentrate on getting my SP campaigns ported to 1.14 and, well, mostly because I wanted some time off. I've been re-familiarizing myself with the status of the code over the last few days and am back at it now.

First thing I did was committing all the previous changes and releasing this as a new version to both the 1.12 and 1.14 add-ons servers. This means that:

Fred / AI-demos v0.14.10 is now available on the Wesnoth 1.12 and 1.14 add-ons servers

A couple important comments: If you've played against Fred before, there's no need to bother with this version. It's a lot of partial fixes to existing problems, some of which are probably making some aspects temporarily worse. The next thing that needs to happen is some serious work on the operations layer which, I hope, will in the end actually make a real difference.

First, however, I am going to remove all the conditionals and duplicate code that makes AI-demos work in both 1.12 and 1.14 and properly port everything to work correctly and without deprecation warnings in 1.14, and 1.14 only. Thus, starting from AI-demos v0.14.11, support for Wesnoth 1.12 will be discontinued. That part isn't all that much work, I think (hope), so I should have that version out within a week or two, after which I'll be back to working on the AI behavior itself.

Anyway, tl;dr. There remains just this:
v0.14.10 complete changelog

Code: Select all

----- 0.14.10, 8 May 2018 -----

- Fred (Freelands AI for Northerners, Side 1):
  - Operations layer:
    - Calculate fronts from in-zone influence only
    - New full-move versions of influence maps
    - Add individual unit influence maps
    - Change to leader_distance metric
    - Fix bug in base_power calculation
    - New menu options to display behavior instructions
  - Attacking:
    - Fix bug which caused Fred to prefer attacking full-health units over easy
      kills
    - Allow using the leader in attacks when there is a small chance of the
      leader to die in the counter attack.
      - This is configurable and can be (re)set to zero CTD if desired
  - Holding:
    - Do not push too far ahead of main front with few units
    - Do not do normal hold if it is too wasteful
    - Keep already protected hexes in protect_locs
  - Advancing:
    - Discourage advancing behind enemy lines without sufficient support
    - Set advance direction to forward or backward depending on enemy threat
      and own-unit support
    - Full reworking of advancement rating (roughly gold-based now)
  - Functionality and misc. other changes:
    - Also turn shroud off at scenario start (shroud can get the AI to crash)
    - fred_utils: new functions linear_regression(), behind_enemy_map(),
      support_maps(), approx_value_loss()
    - Many changes/additions to rating functions, game state maps, supporting
      code and debugging information to support the changes above
    - Add warning that development will be discontinued on Wesnoth 1.12
name
Posts: 564
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name »

mattsc wrote: May 9th, 2018, 2:38 amIf you've played against Fred before, there's no need to bother with this version.
The changelog looks pretty enticing so I might have to bother just the same.
mattsc wrote: May 9th, 2018, 2:38 amThe next thing that needs to happen is some serious work on the operations layer
Huzzah!
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by mattsc »

Cold Steel wrote: May 10th, 2018, 4:52 amThe changelog looks pretty enticing so I might have to bother just the same.
Well, I won't tell you not to, of course, any input is great, but ... The problem is that some of these new changes aren't balanced right yet. Things like "discourage advancing behind enemy lines without sufficient support" do what they say, but they also cause Fred to be (yet again) too hesitant in some situations. Essentially, it comes down to what you comment on next:
Cold Steel wrote: May 10th, 2018, 4:52 am
mattsc wrote: May 9th, 2018, 2:38 amThe next thing that needs to happen is some serious work on the operations layer
Huzzah!
I know I have been saying that I'd do this for several releases now, and then I've always gotten side-tracked by problems I found in the tactical layer. This last round of changes really made it obvious though that I am at a dead end here unless I do actually put in the work on the operations layer. So that's next — until I find something else to distract me. :P

[There's also a not super long, but noticeable delay before every single move now, but that's a comparatively minor issue that I will take care of once the functionality is there.]
name
Posts: 564
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name »

Yeah, the operations layer is the holy grail for wesnoth AI.

Very excited. ^_^
name
Posts: 564
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name »

Just a bit of fun with v14.10:

knalgans_vs_14.10_income+6.gz
(33.61 KiB) Downloaded 533 times
undead_vs_14.10_income+6.gz
(41.52 KiB) Downloaded 594 times

It could be that I am rusty, but I did not notice any new or severe issues from the tactical layer. The utility weightings seemed about right.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc »

Thanks, Cold Steel, for continuously doing all that play testing (those are some long games!) and posting the replays! Looks like maybe I did not make things significantly worse in 1.14.10 after all. ;) And I'll get a couple nice new test cases out of this.

In the meantime, I just released v0.14.11 with the announced move to Wesnoth 1.14 only. This release consists almost entirely of internal changes and clean-up (well, and the removal of some outdated or broken code or things that are now in Wesnoth mainline). None of Fred's behavior has been changed.

I will now move on to working on the operations layer. I have no idea how long it is going to take until I have progress that will actually make a difference. I am sure that it will take some time, but I am hoping that it is a question of when, not if.
v0.14.10 complete changelog

Code: Select all

----- 0.14.11, 16 May 2018 (drop support for BfW 1.12 and code clean-up) -----

- Remove all the version checks and only keep Wesnoth 1.14 compatible code
  - This means that AI-demos will not work in Wesnoth 1.12 any more starting
    from this version
  - Replace deprecated functions and macros
- Fred (Freelands AI for Northerners, Side 1):
  - Set up the scenario events, menu items, CA debugging tools etc. so that they
    are only present in scenarios actually using Fred.
  - Fred quitting because he does not know how to play the map, faction or side
    should count as player victory, not defeat
- Other misc. changes and clean-up
  - Remove local ai_helper.lua and battle_calcs.lua and use mainline versions
  - Remove prune cart scenario
  - Remove machine learning recruiter code
  - Remove 'Default + Extra Targets' test AI
  - Lots of internal changes and clean-up of Lua code and scenarios
name
Posts: 564
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by name »

mattsc wrote: May 17th, 2018, 4:32 amdoing all that play testing (those are some long games!)
The funny thing is, they are so long because I tried to make them shorter with more aggressive play early on. Fred did a good job of punishing that recklessness as I should by now have expected.
mattsc wrote: May 17th, 2018, 4:32 am I will now move on to working on the operations layer. I have no idea how long it is going to take until I have progress that will actually make a difference. I am sure that it will take some time, but I am hoping that it is a question of when, not if.
Looking at your formidable task list for the operations feature, I wonder if it could be broken up over a number of versions? I imagine many of those sub features will heavily influence each other to the point they may be inseparable. But breaking it up allows you to better evaluate the effects and appreciate the progress you are making so that you do not lose your mind in the middle of it. :)
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc »

Cold Steel wrote: May 18th, 2018, 4:36 pm The funny thing is, they are so long because I tried to make them shorter with more aggressive play early on. Fred did a good job of punishing that recklessness as I should by now have expected.
Well, that's good to know. :)
Cold Steel wrote: May 18th, 2018, 4:36 pm Looking at your formidable task list for the operations feature, I wonder if it could be broken up over a number of versions? I imagine many of those sub features will heavily influence each other to the point they may be inseparable. But breaking it up allows you to better evaluate the effects and appreciate the progress you are making so that you do not lose your mind in the middle of it. :)
Yeah, I agree, and that's certainly the plan. I am not entirely sure yet what is needed for that and what steps might make sense for intermediate release, but they always seem to present themselves as I go along. I'll keep you posted. It's too late for my mind though! ;)
denispir
Posts: 184
Joined: March 14th, 2013, 12:26 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by denispir »

Hello,

First, thank you very much @mattsc for all your great and hard work on Lua AIs and micro AIs for Wesnoth. (Without all that I would not even start and explore the domain, or only in mind, which cannot go very far...)

I wish here to expose a very general idea. Sorry to post here [*]. And more sorry for the big post. (Also, english is a foreign lang for me so it may be a little hard to understand, but it is not my fault ;) .)

Imagine a very simple scenario: 2-side battle, fair map, possibly designed equivalent or even symetric, few and ordinary unit types, maybe just 1 per side, no special "gifts" (abilities, weapon specials), no levelling thus no issue of XP... The only difference between the sides is that side 1 recruits more but weaker units while side 2 recruits less but stronger units. Those unit types may be level 1 and 2, but tweaked to make them weaker/stronger in a simple way, to make things obvious, say the stronger type just inflicts double damage.
  • How would you play this to win, in either case? (side 1 or 2)
  • How should an AI be designed to (have an optimal probability to) win with side 1? side 2?
  • How should an AI be designed to win whatever the side it controls?
I had thoughts about that (this example is just a way to expose them more easily), but decided to post about them after reading by you (somewhere in this thread?) that Fred's attack decision is basically based on:
Attack when the balance of damage is positive.
(I hope I understood it rightly, because you wrote it out of context.)
If we did so strictly and clearly and simply, as human player or AI designer, we would never win with side 1 (numerous weak units). All else being equal or random, despite our bigger number of units (which lead to local balanced ot advantageous situations) most combat opportunities would be in favor of side 2.

And yet, the scenario is fair and balanced, and any non-novice human player will easily win (with either side for the matter) against the RCA AI. Fred is not directly concerned because of its specialisation (in map, faction, side), but indirectly probably. I started to think at this question after playing several times in a short while with or against only or mainly level 0 units. How would/could/should an AI be designed, in terms of fundamental principles, so that it "naturally" plays well such a game? especially with the side having numerous weak units.

This question radically crushed down my initial idea, more or less the same as yours (IIUC): try to evaluate the balance of damage (albeit I want the eval to remain approximate, but this is another question, related but distinct) (but firstly evaluating probas of death, yet another question, related but distinct). Obviously, since they basically deal half damage, weak units have few(er) opportunites to fight and deal more damage, even widening the evaluation to a "group attack" (my term), meaning including potential attacks by others units (time). [Further widening the evaluation obviously leads to include counter-attacks, which have even greater chances to be disadvantageous in stricts terms or ratio of damage, for the same reasons plus the question of initiative, firstly choice of weapon range but also choice of terrain etc...]

Note: AIs may not be that bad in such a situation for reasons which are clearly distinct from the principles hidden behind this question, such as blindly or exaggerately following ToD for instance (RCA AI not corrected is not that bad with level 0s because it is far too agressive and too few cautionous).

As often happens the answer popped up intuitively in my mind. But I know, also intuitively, that it is only half right, thus half wrong, so that instead of elaborating I will say it as concise as possible, expecting collective intelligence: expose your own reflexions and/or answers, and let us all brass all that into a better understanding. So my present answer is:
Attack when the situation is better than average.
(or than random, or than statistical expectancy...)
(Consider the contrast with: Attack when the balance of damage is favorable.)
(And apply this to the example exposed above.)

I also wish to open another reflexion caused by this one, which I call "meta-criterion". what is the criterion to judge an AI criterion? (a factor of choice, since finally AI as well as human play is all about choosing) My present idea is that an AI criterion is good if ever it makes a favorable difference in average (or it is better than random, or than ignoring it, or than processing it wrongly...)

Thank you,
Diniz

[*] Please someone create an AI forum; for both AI usage & research/developpement, both standard and custom AIs, both general and micro AIS. It would help very much to share and mix and improve reflexion and also knowledge (of existing AIs, their usage, their principles). And basically: where to post AI questions or reflexions?
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc »

Hi denispir,

No problem posting here, as far as I am concerned. This thread or opening your own thread would also be possible, but this is very similar to some of the ideas implemented in Fred, so this is perfectly appropriate.

As for your comment concerning a dedicated AI forum, you could write the forum moderators group, but TBH there aren't all that many AI topics, so they might not consider that worth it. The Coder's Corner forum used to say in its description that it was for AI topics, but I just saw that it does not state so specifically any more. Anyways, that's the place where AI topics usually go.

As for your comments on AI evaluations, I entirely agree. I don't know in what context that statement of mine that you quote was made, but it was either an early version or just a very high-level summary. I mean, that is part of the individual attack evaluation in the tactics layer, but there are also additional considerations in the operations layer that lay out the overall strategy and make the AI more aggressive when the overall situation is good and less so when it is bad. So the damage calculation gets influenced by overall modifiers.

Let me give you another simple example when straight out damage statistics don't work well. Let's say you have a valuable unit that has one strike and 1 HP. It is up against an enemy unit with the same stats and both are on 50% defense terrain. You attack and strike first, so you have a 50% chance of killing the enemy. If you miss, the enemy strikes back and has a 50% chance of killing your unit, making it an overall 25% probability of you losing your unit. So, statistically, things are very strongly in your favor. As a human player, would you do that attack? And the answer is, it depends. If there are a large number of pairings like that on the map: sure I would, because overall (on average) it would come out in my favor. If this were the only pairing like that, and if that unit were important: hell no, 25% chance of losing it is way too high a risk.

So what I am saying is that I agree with you, and what I am trying to do with Fred is to implement more of the kind of thought processes that a human player goes through. What I am learning though is that the devil is in the details, and that finding something that works in general is quite hard to do. General thoughts like these are good and important, but making things work in practice, and in particular in a general setting, is very difficult, at least for me.

I'm afraid I don't really have much more to add at this time. Maybe I missed something you wanted me to comment on specifically. Please remind me if that's the case.


BTW, I have not replied to your other post, because I do not really have anything to add there. What you write there sounds good in general too, but in my experience until you actual try something like that in practice, it's hard to say if it will work as you imagine. Same comment as above, really. :)
denispir
Posts: 184
Joined: March 14th, 2013, 12:26 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by denispir »



Hello @mattsc,

For some reason I had not seen your reply here, just checked now (before addressing the question in our other exchange thread). I have some more reflexions on the topic and want to insist or make clearer the core principle "better than average": it is really different from "positive balance of damage", even if you widen the evaluation.
Below, as above in my previous post, I assert ideas as if they were grand truths. Writing thus is simpler (and the ordinary way in our culture). In fact, they are just the present state of my reflexion, which may change (it has already as you'll see), and may well be partially or totally wrong (a still hidden to me, really right criterion may indeed exist).

[Remind that in the example case, stronger units just deal twice more damage, but (this remained implicite) have the same max_HPs. Thus, in general or statistically, each side just needs to deal a little more damage than proportionnally (half/double) to win.]

Why side 1 loses, and what it should do:
The "standard" attack criterion of balance of damage obviously leads to very few, if any, good opportunities of attack for side 1, just because they deal half damage in average, even with the help of good ToD and favorable terrain. Since the attacker chooses when (ToD), where (terrain defence), when/where (situation), and attack distance, indeed having more attack initiatives is good overall; and just leads to victory in an ordinary battle scenario (one that does not require a specific tactic imagined and setup by the scenario author --as you often do ;-)).
The right criterion for side 1 is just to attack when it deals more damage than it would do by random attacks, meaning more than average. This is enough to win in general, since it leads to more than half as many kills (in this example case). (In this case, this also means more than half the damage dealt by side 2, but *this is notthe criterion, see below.)

Why widening the evaluation is not the same:
Widening the evaluation, to more units and more turns and more space, indeed increases the number of attack initiatives for side 1, since there will often be 2 or 3 weak units facing 1 strong unit, or similar at higher scales. But all in all there will still be more favorable attacks for side 2. And anyway this is not the same as changing the basic criterion of attack, the idea I am testing here: we are still deciding on a positive balance of damage, just at a larger scale. It is not only not good enough as basic criterion, it is really different from "better than average".
Same for side 2, see below.

Why side 2 (less but stronger units) also loses:
I just realised that side 2 also loses with the standard criterion of attack. It will often attack, maybe very often (if there are many cases of 1 vs 1), each time the balance of damage is positive, which means also in many bad situations of terrain or ToD. But positive balance is not enough: side 2 needs to attack when the situation is favorable, and only then, meaning when it deals more than average damage, again. This is required to win in general, to deal more than twice more damage overall, thus kill twice more units.
Again, widening the evaluated situation would lead to side 2 avoid some of the wrong cases of attack; again this is not the right criterion.

"better than average"
With terrain defence and ToD, Wesnoth rules create situations where the (expected) damage is better or worse than average. Without such rules, favorable situations would only appear with "zonal" unbalance of forces (1). In any case, it is precisely such situations that need to be used when they happen; and that need to be created or avoided at a tactical level. Such situations lead to "better than average" outcome; and that's what we need to win, and all what we need. If the scenario is more or less fair, indeed, see below the case of human players and an attempt at generalisation.
Thus, the criterion I am trying to introduce may be called "better than average" but also "favorable situation" (or advantageous) and it is also equivalent to "better than random".

general tactic:
We can go as far as saying that in a battle scenario, the basic tactic and also the goal of every tactical action, is to create more attack initiatives for our side; more good attack opportunities, meaning more advantageous situations. But when are they good in fact? what is the actual criterion? I pretend here that it is not more damage inflicted than the other side, but instead more damage than average (or than random) (or than probabilistic expected value).
For instance, this is just what we do (as human players or AI designers) when we agressively push our attacks in good ToD and retreat or reform our ranks when ToD changes. We do not, and do not need to, and should not, evaluate the balance of damage, neither individually for each 1-1 attack, nore globally. We know, and we are right on that, that such an approach is good overall; also whatever the composition of forces, including cases of disbalance of units levels. If we did not do so but instead evaluated the balance of damage, then we would often avoid attacking in right cases, and conversely attack in wrong cases.

AI design:
In AI design, this may as first approach lead to ignore damage computation altogether! Both for 1-1 potential fights and collective combat zones. But instead compute a mix of *factorsinfluencing damage (ToD, terrain, zonal ratio of forces, what else?). At different scales of situations for different "depths" of evaluation.

Against a human player all is different...:
The reasoning so far implicitely assumes a fair scenario overall. If I am right, then when forces are equivalent (2) deciding if/when to attack according to "situation is better than average" leads to victory (in general, or "all else being equal"). If other factors, eg a map with unbalanced terrains, are unfair, then the advantaged side will just win in general because it has more opportunities of good attack.
We know from experience that humans play better than the std AI, they can and do win in unfair situations. Similarly, we could have AIs of different skill levels, other than the criterion discussed here, in tactics in particular which as said above allows creating good situations and avoiding bad ones.
When the scenario is unfair we need to adapt the criterion: the davantaged side (eg an AI having 300 start gold against a human with 200) does not need a situation to be "better than average": with stronger forces, it can afford fighting in slightly bad cases. And it should do so to have a chance to win against a better opponent.
Conversely, we humans certainly (but how to check that?) only fight whenever the situation is much better than average, not just slightly. Our talent is our ability to provoke such situations, and avoid others. Read: our talent is firstly tactical. (However, we have much more also, at technical and strategic levels, but this is a distinct topic.)

...as well as at different difficulty levels:
Similarly, the criterion needs to be adapted for different difficulty levels, since unfairness increases with difficulty. The weaker an opponent (3), the more an AI should attack in moderately bad situations.

overall ratio of forces:
This leads to a more general criterion, also aplying to AI vs AI in principle, where "better then average" is weightened by a factor grossly reflecting the ratio of forces.


(1) Which can happen in a random or disordered manner, thus again balanced in average, and because of positive feedback loops (disbalance of forces mechanically increases), the winner will certainly be the side having the very first positive situations, even taking random into account. Or else such situations are actively created (and avoided) by players, AI or human, meaning a tactical level. Very similar in principle to chess, go, etc...

(2) Which can in first approach be evaluated by gold/income iff the composition of forces, meaning the diverse recruitable unit types, is also fair. Eg a single healer has such an impact that it disbalances all.

(3) To avoid yet another factor of positive feedback loop, we should not balance hard difficulty by strengthening the enemy but by handicaping the human player. Else, mechanically, through XP, better players have always stronger forces, meaning we are forced to oppose them even stronger AIs in later scnearios, etc... I mean, in campaigns.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc »

At long last:

Fred / AI-demos v0.14.12 is now available on the Wesnoth 1.14 add-ons server

Why it's taken so long since the last release
We're closing in on a year since the last release. In part this is because I've been busy with other things and because I took two extended breaks from Fred (one for working on mainline AI tasks, and one just to take a break from Wesnoth altogether), but that does not mean that work on Fred has stopped. Not all all. In fact, the other reason why it took so long is that I have finally done the big restructuring of the operations layer that I have been talking about for so long.
A few of the main points:
  • The main restructuring of the operations layer is done with this release.
  • Much work remains to be done on the detail level, of course, but that should be possible without another massive refactoring effort.
  • The next couple releases will concentrate on fixing some of the known issues and improving the fine tuning and balancing. As an example, one of the main problems on the specific-action level at the moment is that Fred often unnecessarily puts injured units on the front line.
  • After that, I would like to move on to expanding the AI to other factions, playing side 2, and then to playing other maps, doing further refinements as part of that process, rather than endlessly trying to improve Fred on Freelands. I bet I am not the only one who is getting tired of that.
I could use your help
I will continue to work on the type of details I mention above, but in the meantime it would be good to know how Fred is doing in "broad strokes". I've made very significant changes to many evaluations, so I am sure that some of the balancing is off at this time. With that I don't mean balancing between factions and the like, but things like the balance between individual AI actions/moves, when to be offensive or defensive, and the like.

I have done a lot of testing myself, of course, and I do see significant improvements compared to previous versions in some aspects of Fred's play, but there are several problems with that. First, having written the code, I know exactly what Fred is trying to do, so I can identify an improvement in some aspect even if it is followed immediately by a horrendous mistake. That's essentially impossible for anybody not knowing the code.

Also, as those of you who following this thread know, I do most of my testing using individual test cases or against the default AI simply for time reasons, with the well-known rat's tail of problems that brings with it. So there's a good chance that, against human players, Fred is currently too defensive again. Or too aggressive. Or both. However, I am reasonably confident that the mechanisms needed to adjust that sort of problem are in place now and require mostly tuning rather than yet another massive restructuring effort. In fact, if it turns out to be the latter I might be done with Fred ...
In summary, a few replays against (ideally different) human players would be extremely useful at this time. I don't need many, a couple would help a lot already and mostly I am interested in the big picture at the moment, although I will of course also look for glaring mistakes on the detail level.

In the meantime, I will keep working on those details and release new versions as I make progress, hopefully at a much quicker cadence than for 0.14.12.

v0.14.12 complete changelog

Code: Select all

----- 0.14.12, 17 April 2019 (massive operations layer reworking) -----

This release focuses on a major reworking of the operations layer. This includes
augmenting and refining the decisions that are made up front in this layer as
opposed to by the individual action evaluation functions, considering actions in
combination rather than individually, setting up a reserved-actions system and
switching all evaluations to a gold-based rating system.  It also includes
significant changes to essentially all action evaluations.

- Fred (Freelands AI for Northerners, Side 1):
  - Operations layer:
    - Switch all evaluations consistently from a utility-based (0..1) system
      to a benefits/penalties based system scored in units of gold
    - Set up a reserved-actions mechanism which evaluates certain actions
      (leader moves, recruiting, village grabs, ...) up front but does not execute
      them until later. These are combined with an interaction matrix that
      determines whether other actions may use the assigned units instead (taking
      the appropriate penalties and alternatives to the reserved actions into account)
    - Rework how units are assigned to the zones and to individual actions
    - Expand the objectives analysis determining the goals for the current turn,
      specifically (but not only) for the leader. This includes:
      - Full up-front evaluation of where the leader is going to end after the turn
      - Use effective reach maps for the leader to take going to the keep to
        recruit before heading out for other actions into account, without having
        to start each turn by moving the leader back to the keep
      - Don't force the leader to move to the keep if there is nothing to recruit
      - Recruit preferentially in between leader and threats
      - Decide what other units and hexes should be protected
    - Determine up front how hard the AI should push forward in each zone based
      on power ratios and time of day
    - The operations assessment is now refined almost entirely after every AI
      move to take the changed situation on the map into account
    - Action scoring:
      - Split up holding into separate actions (with different scores) for
        protecting and "normal" holding
      - Add an eval-only protect-leader action, used as baseline for other actions
      - Grab villages between protecting and holding
      - Retreat before advancing
      - Leader moves, except recruiting, are now done by the retreat action
      - All-map advancing now has a separate score and is the very last thing done
    - Remove move_leader_to_keep and recruitment candidate actions. These are now
      also handled by the zone_control CA in order to allow more flexibility
    - Combine zones at the very beginning of the ops analysis, not at the end
    - Remember in which zone a unit was used
    - Improved behavior debug output
    - Behavior debug output menu options are now shown only in debug mode
  - Attacking:
    - Take penalties for reserved actions into account
    - Attack scores now include contributions for how well the leader, castle
      hexes, villages and other units are protected
    - Disqualify attacks which do not sufficiently protect the leader
    - Add "XP milking" attacks before enemy leader kill
  - Holding:
    - Take penalties for reserved actions into account
    - Protect holds now include a protected value evaluation that is scored
      against the risks/exposures of the holds
    - Try to protect castle hexes also; this is evaluated in combination with
      leader, villages and other unit protecting using benefits based scoring
    - Acceptable damage limits are now used for both holding and protecting. What
      counts as acceptable is different for the two types of holds
    - Use projected leader position at the end of the turn for hold evaluation,
      rather than the leader's current position or the keep
  - Villages:
    - Do not grab villages that are not worth it using, again, the benefits
      based scoring system
    - Consider village grab benefits/penalties for combination of all villages,
      not by individual village
  - Move execution:
    - Some fixes and improvements to direction in which units move out of the way
    - Change order of attacks if units moving out of way end up with no moves
      left, so that they do not become stranded
  - General:
    - Significant refinements to the ratings of many of the actions
    - Lots of internal clean-up, reorganizing, bug fixes and efficiency improvements
    - New fred_virtual_state library
- Misc.
  - Add the mp_rank parameter (new in Wesnoth 1.15) to the AI configs
name
Posts: 564
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by name »

'
Excellent work as always, mattsc! Some replays:

loyalists_vs_14.12_income+6.gz
(26.94 KiB) Downloaded 424 times
undead_vs_14.12_income+6.gz
(29.21 KiB) Downloaded 451 times

Got a couple of nonfatal errors on the above loyalist match:

Code: Select all

error scripting/lua: ~/add-ons/AI-demos/lua/fred_benefits_utilities.lua:535: attempt to perform arithmetic on a nil value (field '?')
stack traceback:
	~/add-ons/AI-demos/lua/fred_benefits_utilities.lua:535: in field 'action_penalty'
	~/add-ons/AI-demos/lua/fred_hold_utils.lua:483: in field 'find_best_combo'
	~add-ons/AI-demos/lua/ca_zone_control.lua:2328: in upvalue 'get_hold_action'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3053: in function '.get_zone_action'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3257: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3177>
	(...tail calls...)

Code: Select all

error scripting/lua: ~add-ons/AI-demos/lua/ca_zone_control.lua:3591: Fred to move out of way with dx = dy = 0
stack traceback:
	[C]: in function '.error'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3591: in field 'execution'
	[string "local self, params, data = ......"]:2: in main chunk
Not sure if you wanted to worry about optimizing yet, but there was a notable slow down while Fred was deciding what to do for turn 26 of the undead match.
mattsc wrote: April 17th, 2019, 4:35 pm The next couple releases will concentrate on fixing some of the known issues and improving the fine tuning and balancing. As an example, one of the main problems on the specific-action level at the moment is that Fred often unnecessarily puts injured units on the front line.
In the above matches, most of the time he seemed to protect the injured quite well, I thought.
mattsc wrote: April 17th, 2019, 4:35 pm After that, I would like to move on to expanding the AI to other factions, playing side 2, and then to playing other maps,
These are exciting times we live in!
mattsc wrote: April 17th, 2019, 4:35 pm However, I am reasonably confident that the mechanisms needed to adjust that sort of problem are in place now and require mostly tuning rather than yet another massive restructuring effort. In fact, if it turns out to be the latter I might be done with Fred ...
Well try not to be too hard on him in any case. :) After all, he was devastatingly more lethal than default RCA years ago even. The moment he can play the other sides/factions on the other mainline map, is the moment RCA becomes obsolete.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc »

Thanks, name, for the kind words and especially for the testing and sending the replays. I can see some things that Fred is doing pretty well (better than before, I think), but also some serious issues. I'm pleased with the former, not surprised by the latter, so overall this is good.

For example, they show a mix of Fred being too defensive (again) and not attacking, but also exposing individual units or small groups too much and fragmenting. Getting units stranded in the corners is one of the symptoms of that. Oh, and I like the "let's leave our leader to be killed but use many units to hold off against that one walking corpse coming from the south" tactic at the end of the undead game. :D

BTW, I also received a few replays via direct messages in Discord which show quite similar overall behavior. So I know what to work on next. And as a note, while that might not seem very different from what he did before, the refactoring of the code means that (I think) I am in a better position to deal with this now. Still doesn't mean that I kow how to do that, but as I told somebody on Discord, figuring that out is part of the fun.
name wrote: April 20th, 2019, 7:42 am Got a couple of nonfatal errors on the above loyalist match:
The second also happened in one of the other replays and is already fixed. Do you remember at which turn (or approximately which turn) the first error happened?
name wrote: April 20th, 2019, 7:42 am Not sure if you wanted to worry about optimizing yet, but there was a notable slow down while Fred was deciding what to do for turn 26 of the undead match.
Thanks. I'll have a look. Whether I'll do something about it depends on whether that's code that needs to be reworked anyway or whether it's relatively stable.

Some musings about expanding Fred's capabilities
name wrote: April 20th, 2019, 7:42 am Well try not to be too hard on him in any case. :) After all, he was devastatingly more lethal than default RCA years ago even. The moment he can play the other sides/factions on the other mainline map, is the moment RCA becomes obsolete.
Well, thanks for the confidence, but it might not come to that. One of the great advantages of the default AI is that you can just plug it into any situation and it will operate pretty well (within its constraints). I'm not sure if that will ever be possible with Fred. And with that I actually don't mean that I might not be able to do it, but more fundamentally based on how the AI is set up. But anyways, we 'll worry about that when we get to it.

For reference though, letting Fred play other factions is trivial. In fact, it's already possible, it's just disabled because I want to be able to see whether an apparent difference in play style comes from a change I am making to the code, or because Fred is not as good at playing, say, Knalgans. (If you wanted to let him play other factions, all you need to do is disable one conditional in the code; I know people have done that.) Of course, there are certain things that Northerners do not have (such as leadership) that Fred does not know how to use.

Having him play Side 2 on Freelands is pretty easy too, we just need to mirror a few of the map-specific configuration parameters. But there's a subtlety because Fred does look ahead to the enemy's turn, which happens at a different time of day than the AI turn then. That's going to take a bit of work, but is theoretically not too complicated.

Now, different maps is a different story. I simply don't know yet how well the Fred algorithms would work in different settings. Hopefully they can be adapted to other 2-player maps at least. There are a lot of implicit assumptions about there only being one enemy side (and no allied sides).

In other words, there's a long road ahead before Fred will be more generally usable, but hopefully we'll get there in the end.
Post Reply