[Proposal] Standard Release Schedule

Discussion among members of the development team.

Moderator: Forum Moderators

User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

[Proposal] Standard Release Schedule

Post by Pentarctagon »

Previous releases have been made based on fairly subjective arguments of whether or not enough has changed or if there have any notable fixes and-or improvements implemented. This has resulted in nobody, users or developers, really knowing when the next release will be and therefore when the newest features and fixes will be available. For users in particular it's less than helpful, since often the response to a bug report is that it's already been fixed and the fix will be in the next stable release, but nobody knows whether the user will have the fix available to them in a week, a month, or even longer.

To address this, I believe the release schedule should become time-based, so that users, developers, translators, etc can know ahead of time when the next release will be. The tentative schedule I'd like to implement is:
  • Regular stable point releases
    • There will be a bi-weekly pot update so that translations' status and statistics are kept up to date. Relatedly, this would also mean no more pushing major campaign overhauls or rewrites into the stable branch.
    • There will still be a string freeze two weeks prior to any new stable branch point release.
    • All commits with string changes must be submitted as a PR and approved by the UI, SP content, or MP content maintainer as appropriate. The purpose of this is to double check for correctness in order to reduce the need for minor typo fixes and corrections later, which then also increases rework for translators.
    • Releases would be done as needed as determined by the PM, rather than on a strict schedule, other than at minimum requiring a point release every three months. This will ensure that in the case of development of the stable branch decreasing that minor bugfixes will still eventually be released to users.
  • Regular development point releases
    • Development point releases will be done at 00:00 UTC on the third Saturday of every month.
    • Development point releases have no string freeze or bi-weekly pot update.
  • Development beta releases
    • Development beta releases will be done at 00:00 UTC on the third Saturday of every month.
    • The start of beta will be determined by the project roadmap.
    • The development branch string freeze begins.
    • The development branch feature freeze begins - no major overhauls/refactors/etc of any area of the game should take place at this point either.
  • Development release candidate releases
    • The third beta becomes RC1.
    • Further RC releases will be done as needed.
  • New stable release
    • Released one month after RC1, assuming there are no remaining blocker issues.
  • Emergency releases
    • Hopefully rare, and done as needed to address security and-or game breaking bugs.
    • Emergency releases do not replace the standard point release dates, especially for the stable branch which needs a string freeze and translations update for regular point releases.
Previous proposal:
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
doofus-01
Art Director
Posts: 4122
Joined: January 6th, 2008, 9:27 pm
Location: USA

Re: [Proposal] Standard Release Schedule

Post by doofus-01 »

I support this time-based release schedule. As much as I hate them, I know I need deadlines to stay focused. Learning about the deadline two weeks out or less is unhelpful.
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
User avatar
loonycyborg
Windows Packager
Posts: 295
Joined: April 1st, 2008, 4:45 pm
Location: Russia/Moscow

Re: [Proposal] Standard Release Schedule

Post by loonycyborg »

This makes sense. My main concern with this that potentially you might end up releases in rather broken state(dev releases at certain times) or with not enough changes. But I guess you can just skip release in such a situation and fallback to next time slot.
"meh." - zookeeper
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

The goal, at least, would also be to have both the stable and master branches in a working state as much as possible, since if they're broken and we discover an emergency release is needed then it'd become a major problem.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: [Proposal] Standard Release Schedule

Post by gfgtdf »

I'm not sure what to think about this, from my own experience especially when i was actively writing add-ons, my main problem was that dev versions take veery long to release. So even when a new feature is added the originally author that wanted that feature probably doesn't develops wesnoth add-ons anymore.


Afaik we had a similar discussion already but i think the only way to fix this is to either go either to a rolling release procedure. (afaik the main reason why we didn't do this then was that we didn't want to change how stable 1.14 works after we released it).

Alternatively we could also just allow new features in stable releases and leave the new major versions for wesnoth version that have incompatible changes.

In fact i don't even fully remember why we block features in stable releases ion the first place, someone once said we want addons also to work with older (during the same stable series) wesnoth, but i really don't see the point. First: the idea that addons work with old stable versions is just wrong: Bugs get fixed and new (or old) campaign relie on this behavior. For example the wesnoth version (1.14.5) that is AFAIK shipped in certain Ubuntu distros probably won't even play popular add-ons like BadMoonRising because they have a bug that crashes when using custom themes. Second: i'm quite sure that people can download add-ons can also download newer wesnoth version. Especially now that steam and faltpack exist. Third: people with older wesnoth version could still download the older version of these addons.


Lastly i wanted to say that such a change probably doesn't contradict your proposal since it would only start to take a effect after 1.16 is released.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

I could see going more towards a rolling release model for the development branch, but for stable branches I think that UMC authors and players would want a longer term guarantee of API stability.

For your first point: I think there's a difference between an add-on not working with older stable branch releases because Wesnoth had a crash bug vs not working because it's now using a new API feature that was added a couple months ago.

For your second point: it's as much a question of whether they want to as it is whether they can. Though in practice I'd expect not much would really change, since probably all non-rolling-release distros are behind on Wesnoth versions already anyway.

For your third point: I'm not sure how they could do this? At least currently, only the most recent versions of add-ons are available from the ingame add-ons manager.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
LordBob
Portrait Director
Posts: 1309
Joined: December 8th, 2008, 8:18 pm
Location: Lille, France
Contact:

Re: [Proposal] Standard Release Schedule

Post by LordBob »

Not much to add on my end as I very rarely have anything to do with releases. It does seem a good idea to have a default release schedule though, if only because deadlines will structure team work as opposed to "let's release whatever we just happened to have developped whenever we feel like it".

What I would like to see in conjunction with the schedule, though (and is probably the next step you're planning to take anyway), is goals to reach and at least a general direction we should work on. We don't need a detailed content outline for each release, but at least to know on what to focus our effort. On the art front, I know I've been a lot more productive in the past with commissions, which, aside from the remuneration aspect, offer the opportunity to do focused work because they have a clear objective. So, deadlines and goals. :)
Want to see more of my art ? Visit my portfolio !
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

Coming up with a roadmap for 1.16 is the next step :)
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

gfgtdf wrote: July 12th, 2020, 8:19 pm Afaik we had a similar discussion already but i think the only way to fix this is to either go either to a rolling release procedure. (afaik the main reason why we didn't do this then was that we didn't want to change how stable 1.14 works after we released it).
Also, I think(?) that previous discussion was here (do a thread search for "rolling"). Ultimately though I think a decision will in part come down to trying to ask Wesnoth's players and UMC authors their preference, especially since while the majority of the MP players are using Steam now, about 1/3 aren't (the SourceForge and Default download sources) which isn't insignificant either.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Iris
Site Administrator
Posts: 6797
Joined: November 14th, 2006, 5:54 pm
Location: Chile
Contact:

Re: [Proposal] Standard Release Schedule

Post by Iris »

Pentarctagon wrote: July 10th, 2020, 4:54 pmFor users in particular it's less than helpful, since often the response to a bug report is that it's already been fixed and the fix will be in the next stable release, but nobody knows whether the user will have the fix available to them in a week, a month, or even longer. [...]

The tentative schedule I'd like to implement is:
  • Regular stable and development point releases
    • Do both on the same day, for consistency and so there are fewer release dates overall.
    • The date and time would be 00:00 UTC on the third Saturday of the 1st/4th/7th/10th month of the year.
    • The stable string freeze would start two weeks ahead of that, at 00:00 UTC on the first Saturday of the 1st/4th/7th/10th month of the year.
    • Regular development point releases have no string freeze.
The premise is that we have timing issues when it comes to delivering bug fixes in particular, and yet your proposal would have us ship releases only 4 times a year unless something is considered “game breaking” (which is a very subjective qualifier as it is, developers don’t usually see things from the same perspective as players and add-on authors). This seems like a lateral move to me.

In my opinion stable releases should be delivered as needed but always in a quick fashion. Back in January I suggested that in order to mitigate the existing issues with translators missing the string freeze windows, we could require all patches containing translatable strings to be submitted as PRs to be reviewed by the UI or Content maintainer as appropriate, and establish periodic pot updates every 2 weeks. In a stable branch strings should be changed very rarely, and the point of forcing people to go through the PR system is to avoid situations where we are like “oops I meant to say ‘its’ not ‘it’s’” as much as possible. Then translators wouldn’t need to worry about string freezes and we wouldn’t need to worry about the logistics of shipping emergency releases either.

(Note that this does not mean we wouldn’t still have a string freeze before a regular stable release. The intention is simply to make the process as short and uncomplicated as possible for everyone involved. It was very frustrating back in January to announce as loud as possible that we were having a 13 days-long string freeze, whilst giving people a chance to tell me if they needed more time, and then have a translator maintainer decide to have an existential crisis in my face a minute after creating the tag.)

It probably makes more sense to ship development releases on a more rigid schedule, but 4 times a year still feels really bad unless we're officially admitting that Wesnoth development has come to a standstill. Not to mention that development releases tend to be even buggier (how many highly-visible bugs keep being reported for 1.15.3 that have long since been fixed or rendered irrelevant?).

Also, it's worth noting that for 1.11.x and earlier, the first beta immediately came with a feature and string freeze, even though in 1.11.x's case the feature freeze was broken multiple times because of the slippery slope issue (plus the big thing in RC 3).



Okay, now I'm going to rant about stable releases and OS distributions for a bit.
gfgtdf wrote: July 12th, 2020, 8:19 pm In fact i don't even fully remember why we block features in stable releases ion the first place, someone once said we want addons also to work with older (during the same stable series) wesnoth, but i really don't see the point. First: the idea that addons work with old stable versions is just wrong: Bugs get fixed and new (or old) campaign relie on this behavior. For example the wesnoth version (1.14.5) that is AFAIK shipped in certain Ubuntu distros probably won't even play popular add-ons like BadMoonRising because they have a bug that crashes when using custom themes. Second: i'm quite sure that people can download add-ons can also download newer wesnoth version. Especially now that steam and faltpack exist. Third: people with older wesnoth version could still download the older version of these addons.
The issue of stable series releases having game-breaking bugs is not unknown to me as an add-on creator at all. In fact, both IftU and AtS ship with a library that includes facilities for delivering workarounds to bugs in the game by "patching" WML actions as needed, on top of some liberal use of #ifver all over the codebase as needed. Sometimes I go farther than that and block buggy versions* entirely — for example AtS will refuse to run on Wesnoth 1.14.2 and earlier since there was an issue with map_file= in those.

Presently I haven't done anything about the infamous black screen glitch because, by all means, it is the distributions' responsibility to address. The patches already exist in mainline and were shipped with 1.14.10 and later. The only issue is that no-one has filed bugs to the distributions to get them to backport our patches or (preferably) just update to a newer Wesnoth version since it's a leaf package and a game, and nobody cares if behaviour changes between two stable releases. Furthermore, the issue only existed in the first place because 1) Wesnoth uses SDL APIs in an unintended fashion; 2) software rendering in SDL 2.0 is almost completely an afterthought and upstream is willing to pull the rug from under our feet without warning and without bumping the major version.

Should we be allowed to change behaviour between stable releases willy-nilly? No, not really. Sometimes it's necessary to change behaviour because the existing behaviour is buggy or harmful, but things like “hm maybe we should make [kill] force a shroud map refresh unless do_not_break_old_crap=true is passed to it because things look better that way” aren't helpful to content authors with non-trivial codebases and don't belong in stable series. The general thing I'm trying to get at is that stable series X.Y should guarantee that one add-on that was written for X.Y.0 should not break in X.Y.1 unless it's necessary to fix 1,000 other add-ons also written for X.Y.0.
Author of the unofficial UtBS sequels Invasion from the Unknown and After the Storm.
User avatar
LordBob
Portrait Director
Posts: 1309
Joined: December 8th, 2008, 8:18 pm
Location: Lille, France
Contact:

Re: [Proposal] Standard Release Schedule

Post by LordBob »

Iris wrote: July 15th, 2020, 4:56 amThe premise is that we have timing issues when it comes to delivering bug fixes in particular, and yet your proposal would have us ship releases only 4 times a year unless something is considered “game breaking”
Now you mention it like this, maybe the standard release schedule could allow more flexibility, such as quarterly release being only an "at least" default pace, or the condition for emergency release being made less rigid.
Want to see more of my art ? Visit my portfolio !
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: [Proposal] Standard Release Schedule

Post by gfgtdf »

Pentarctagon wrote: July 12th, 2020, 9:33 pm For your first point: I think there's a difference between an add-on not working with older stable branch releases because Wesnoth had a crash bug vs not working because it's now using a new API feature that was added a couple months ago.
I don't realy agree here. For the players it just means it wont work, but actually this is also not my main point.

Pentarctagon wrote: July 12th, 2020, 9:33 pm I could see going more towards a rolling release model for the development branch, but for stable branches I think that UMC authors and players would want a longer term guarantee of API stability.
Iris wrote: July 15th, 2020, 4:56 am Should we be allowed to change behaviour between stable releases willy-nilly? No, not really. Sometimes it's necessary to change behaviour because the existing behaviour is buggy or harmful, but things like “hm maybe we should make [kill] force a shroud map refresh unless do_not_break_old_crap=true is passed to it because things look better that way” aren't helpful to content authors with non-trivial codebases and don't belong in stable series. The general thing I'm trying to get at is that stable series X.Y should guarantee that one add-on that was written for X.Y.0 should not break in X.Y.1 unless it's necessary to fix 1,000 other add-ons also written for X.Y.0.
I agreem, but most features, in particular when it come to the api can be added without breaking any code at all. In particular those that are of the type 'the c++ function already exists lets add a lua api for it' or forgotten properties of lua unit/side object are basicially guaranteed not to break old code.

Also i look at the 'breaking changes' is see really mostly 'intential' breaking since people disliked he old api (for a reason), Those changes would of course not be added to stable releases
Pentarctagon wrote: July 12th, 2020, 9:33 pm
For your second point: it's as much a question of whether they want to as it is whether they can. Though in practice I'd expect not much would really change, since probably all non-rolling-release distros are behind on Wesnoth versions already anyway.
Not sure whether i understood you correctly here so i'll ignore this part.
Pentarctagon wrote: July 12th, 2020, 9:33 pm For your third point: I'm not sure how they could do this? At least currently, only the most recent versions of add-ons are available from the ingame add-ons manager.
Yes it would need to be implemented first, i started writing this code once, but stopped for a rather unimportant reason (i couldnt decide whether i wants multiple [campaign] tags in the addon server)





Another reason why i want this is that i think that in particular when it comes to new api features, they are best added 'on demand' when people (umc authors) want them (okay i see that there are exceptions). I feel like (maybe i'm wrong here, idk) _a lot_ of features in dev releases are added because of developers thinking "it'd be nice to have, let's add it before the stable release comes and we cannot add them anymore" without having any actual usecase. Which in the best case will be a feature that will be used, In the bad case be a a bit work for nothing and in the verybad case will introduce a new maintenance burden for nothing.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

Iris wrote: July 15th, 2020, 4:56 am
Pentarctagon wrote: July 10th, 2020, 4:54 pmFor users in particular it's less than helpful, since often the response to a bug report is that it's already been fixed and the fix will be in the next stable release, but nobody knows whether the user will have the fix available to them in a week, a month, or even longer. [...]

The tentative schedule I'd like to implement is:
  • Regular stable and development point releases
    • Do both on the same day, for consistency and so there are fewer release dates overall.
    • The date and time would be 00:00 UTC on the third Saturday of the 1st/4th/7th/10th month of the year.
    • The stable string freeze would start two weeks ahead of that, at 00:00 UTC on the first Saturday of the 1st/4th/7th/10th month of the year.
    • Regular development point releases have no string freeze.
The premise is that we have timing issues when it comes to delivering bug fixes in particular, and yet your proposal would have us ship releases only 4 times a year unless something is considered “game breaking” (which is a very subjective qualifier as it is, developers don’t usually see things from the same perspective as players and add-on authors). This seems like a lateral move to me.

In my opinion stable releases should be delivered as needed but always in a quick fashion. Back in January I suggested that in order to mitigate the existing issues with translators missing the string freeze windows, we could require all patches containing translatable strings to be submitted as PRs to be reviewed by the UI or Content maintainer as appropriate, and establish periodic pot updates every 2 weeks. In a stable branch strings should be changed very rarely, and the point of forcing people to go through the PR system is to avoid situations where we are like “oops I meant to say ‘its’ not ‘it’s’” as much as possible. Then translators wouldn’t need to worry about string freezes and we wouldn’t need to worry about the logistics of shipping emergency releases either.

(Note that this does not mean we wouldn’t still have a string freeze before a regular stable release. The intention is simply to make the process as short and uncomplicated as possible for everyone involved. It was very frustrating back in January to announce as loud as possible that we were having a 13 days-long string freeze, whilst giving people a chance to tell me if they needed more time, and then have a translator maintainer decide to have an existential crisis in my face a minute after creating the tag.)

It probably makes more sense to ship development releases on a more rigid schedule, but 4 times a year still feels really bad unless we're officially admitting that Wesnoth development has come to a standstill. Not to mention that development releases tend to be even buggier (how many highly-visible bugs keep being reported for 1.15.3 that have long since been fixed or rendered irrelevant?).

Also, it's worth noting that for 1.11.x and earlier, the first beta immediately came with a feature and string freeze, even though in 1.11.x's case the feature freeze was broken multiple times because of the slippery slope issue (plus the big thing in RC 3).
I'm fine with releasing more frequently, if it makes sense.

For stable releases, what I'd still want set is a maximum time between releases, so that even if development on the stable branch slows down there won't be a situation where a few minor bugs have been fixed and end up just never part of a release.

For development releases, I'd be fine with monthly or even twice per month releases (though the latter is probably a bit much right now), as long as the overhead for creating and uploading the updates to Steam/SourceForge/etc isn't a problem. We'd also need to streamline how the release announcements are created, but you had a proposal that would help with that as well.

My thought regarding having the string/feature freeze start at RC1 was that RC releases are essentially what will turn into the official new stable release assuming there aren't any further issues discovered, so that'd be what translators should target for updating translations.
gfgtdf wrote: July 15th, 2020, 2:26 pm
Pentarctagon wrote: July 12th, 2020, 9:33 pm For your second point: it's as much a question of whether they want to as it is whether they can. Though in practice I'd expect not much would really change, since probably all non-rolling-release distros are behind on Wesnoth versions already anyway.
Not sure whether i understood you correctly here so i'll ignore this part.
To clarify: On vanilla Ubuntu for example, flatpak is not installed by default since Canonical is pushing for snaps instead. Steam of course is still an option, but I think the more important change would be for us to now more actively be saying that people shouldn't use the Wesnoth distro package, since it will likely never be updated to work with the APIs added to newer stable versions.
gfgtdf wrote: July 15th, 2020, 2:26 pm
Pentarctagon wrote: July 12th, 2020, 9:33 pm For your third point: I'm not sure how they could do this? At least currently, only the most recent versions of add-ons are available from the ingame add-ons manager.
Yes it would need to be implemented first, i started writing this code once, but stopped for a rather unimportant reason (i couldnt decide whether i wants multiple [campaign] tags in the addon server)
I would prefer to make a decision based on currently available functionality. Opening it up to functionality we'd like to implement as well seems like a slippery slope.
gfgtdf wrote: July 15th, 2020, 2:26 pm Another reason why i want this is that i think that in particular when it comes to new api features, they are best added 'on demand' when people (umc authors) want them (okay i see that there are exceptions). I feel like (maybe i'm wrong here, idk) _a lot_ of features in dev releases are added because of developers thinking "it'd be nice to have, let's add it before the stable release comes and we cannot add them anymore" without having any actual usecase. Which in the best case will be a feature that will be used, In the bad case be a a bit work for nothing and in the verybad case will introduce a new maintenance burden for nothing.
That sounds generally more like a bad development practice, to be honest. All features added should have a definable usecase (and IMO as possible a unit test or two proving that usecase works), and by itself opening up stable releases to adding API features wouldn't really prevent anyone from doing the same thing just on the stable branch instead.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
gfgtdf
Developer
Posts: 1432
Joined: February 10th, 2013, 2:25 pm

Re: [Proposal] Standard Release Schedule

Post by gfgtdf »

Pentarctagon wrote: July 15th, 2020, 3:08 pm That sounds generally more like a bad development practice, to be honest. All features added should have a definable usecase
Its easy to define usecase, un not easy to decide whether it's realtistic or not (whether it will actually be used)


Pentarctagon wrote: July 15th, 2020, 3:08 pm and by itself opening up stable releases to adding API features wouldn't really prevent anyone from doing the same thing just on the stable branch instead.
It would eliminate the main reason for doing this, which is exactly that these feature have to be added during dev release, before people can actually use them and before one can see whether they will be useful or not.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Pentarctagon
Project Manager
Posts: 5526
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: [Proposal] Standard Release Schedule

Post by Pentarctagon »

gfgtdf wrote: July 15th, 2020, 3:40 pm
Pentarctagon wrote: July 15th, 2020, 3:08 pm That sounds generally more like a bad development practice, to be honest. All features added should have a definable usecase
Its easy to define usecase, un not easy to decide whether it's realtistic or not (whether it will actually be used)


Pentarctagon wrote: July 15th, 2020, 3:08 pm and by itself opening up stable releases to adding API features wouldn't really prevent anyone from doing the same thing just on the stable branch instead.
It would eliminate the main reason for doing this, which is exactly that these feature have to be added during dev release, before people can actually use them and before one can see whether they will be useful or not.
At this point I'd like to keep this thread focused on establishing what we're going to be doing for 1.14/1.15 going forward. Can you open another thread for discussing whether to allow new API features into future (1.16+) stable releases? Keeping in mind as well that with the 1.16 release there will be another PM election.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
Post Reply