Welcome TurboGears 2.4

Today was finally made public the pre release of TurboGears 2.4 through the official announcement.

The announcement states that the new version provides:
* More predicable configuration flow through configuration blueprints
* Pluggable Components based configuration
* Ability to extend the framework with new components.

Given the *major* improvements in flexibility and ease of use that happened in that release, I’m super excited about it and I thought of writing a bit about it to make everyone aware of what’s going on.

Little bit of history

Historically, the most frequently discussed concern between TurboGears users has always been the complexity of the configuration system.

While configuring a TurboGears application was quite easy, getting into how an option was used, where it came from, if it was supposed to be expressed as config['group.option'] = 5 or config['group']['option'] = 5 and even extending the configuration process was not very straightforward.

Most of that was caused by the fact that TurboGears was originally a layer on top of Pylons and thus had to build on top of the Pylons configuration system and heavily customise it.

Since version 2.3 got rid of Pylons and introduced the minimal, micro-framework mode as its foundation, a process to simplify the configuration system was put in place and slowly rolled out across the years from 2.3.0 to 2.3.12.

Given that TurboGears aims at being a reliable long term framework for people that build long living production applications, this refactoring happened one small step at a time and lead the framework to a state where the configuration flow was more predictable, while retaining compatibility with previous versions.

2.4 was finally the final major leap forward and moved the configuration system from the once do all AppConfig to the more flexible ApplicationConfigurator that allows to add/remove components to the framework, explicitly describes the options that every component exposes, explicitly performs type casting of those options and makes a clear difference between options coming from the .ini configuration file and those coming from a blueprint used to build new configurations.

Keep it simple!

To understand why the redesign process was started, we can look at the configuration of an old 2.0 — 2.3 application:

base_config = AppConfig()
base_config.package = testapp
base_config.renderers = ['json', 'kajiki']
base_config['templating.kajiki.strip_text'] = False
base_config.default_renderer = 'kajiki'
base_config['session.enabled'] = True
base_config.prefer_toscawidgets2 = True
base_config['session.data_serializer'] = 'json'
base_config.use_sqlalchemy = True
base_config.sa_auth.cookie_secret = "SOME_SECRET"

See how some options were set with .option = X syntax and some were set with ['option.something'] = X .

That was needed depending on the fact that some options were within a sub dictionary, some were not and it wasn’t clear if the option was named 'a.b'or if it meant to set option bwithin a. All based on the fact that historically those options came from different pieces that had their own implementation (Beaker for session/caching options, repoze.who for authentication options, and so on…)

Also all options that required rich python types ( True, list, etc... ) couldn’t be set from .ini configuration file.

Finally it wasn’t clear what would happen with those options, because we would set them as attributes or values of the AppConfig it wasn’t clear how they ended up in tg.config and through which flow.

The new configuration system comes in to provide consistency to all of this.

base_config = FullStackApplicationConfigurator()

base_config.update_blueprint({
    'package': testapp,
    'tw2.enabled': True,
    'renderers': ['json', 'kajiki'],
    'default_renderer': 'kajiki',
    'templating.kajiki.strip_text': False,
    'session.enabled': True,
    'session.data_serializer': 'json',
    'use_sqlalchemy': True,
    'sa_auth.cookie_secret': "SOME_SECRET",
})

The configuration of the application is now always built from a blueprint.

All the options specified in the blueprint get copied into TGApp.conf, and tg.config always represents the configuration of the currently served TGApp within a request.

Options are always set as dictionary keys with dotted names, and all options can now be set from .ini files too, overwriting the ones in the blueprint.

All options now come from their specific component, so it’s easy to see which options are available just by looking at the components documentation: https://turbogears.readthedocs.io/en/development/reference/config-options.html

The Components

As the whole configuration process is components based, the FullStackApplicationConfigurator is in fact just a specialised ApplicationConfigurator that enables all the components that are commonly needed in a full stack application and sets some default options.

class FullStackApplicationConfigurator(...):
    def __init__(self):
        super(FullStackApplicationConfigurator, self).__init__()
        self.update_blueprint({
             'use_dotted_templatenames': True
        })         
        self.register(I18NConfigurationComponent)
        self.register(SimpleAuthenticationConfigurationComponent)
        self.register(SessionConfigurationComponent)
        self.register(CachingConfigurationComponent)
        [... more components here ...]

So it’s possible to replace some parts of the framework, disable them or add new features by adding new components.

For example the TurboGears documentation itself shows how to replace the SQLAlchemy component, with a custom one that supports multiple databases for vertical partitioning.

Registering and replacing your own components it’s also possible and simple.

The Upgrade Path

As TurboGears has been around for years (since ~2005) it already had its chance to face a major upgrade path from TurboGears1 to TurboGears2 and we learnt a lot from it.

We wanted to ensure that there was an easy and known upgrade path for everyone and we didn’t want to split the community in two incompatible sub-communities like it happened with TG1 and TG2.

So a major effort was invested in ensuring that most TurboGears2.3.12 applications can upgrade to TurboGears2.4.0 with no changes.

For this reason a backward compatible tg.AppConfig class is still available and applications can continue to import and use it. But it’s now implemented on top of ApplicationConfigurator .

Even though the changes required to make an application work on 2.4 are minimal (a newly quickstarted 2.3 application starts on 2.4 with no changes at all) there is a guarantee that there is no difference in behaviour between a 2.4 application relying on ApplicationConfigurator and one relying on AppConfig . Because AppConfig it’s just a wrapper to ApplicationConfigurator .

As the change in TurboGears is major, this is the first time TurboGears pushed out a pre-release.

2.4.0a1 can only be installed if --pre option is used with pip. So that existing setups are not broken and people can take their time upgrading to 2.4. Especially authors of pluggable applications and extensions.

Currently tgext.pluggable has already been upgraded to support TurboGears2.4 and a new release will happen soon to allow all pluggable authors to slowly port their pluggable apps to the new configuration system.

I’m very excited that 2.4 is finally out in the open world! This pre release will allow us to start gathering feedbacks from users and ensure that there is a flawless upgrade path for everyone, including those companies that have been building products on TurboGears for the past decade.