Automatic page publication from Pelican to GitHub Pages

2021-02-21T04:04:51-05:00

I want to automatically generate and publish the website when I push a modification to the master branch of the GitHub repository.

The tasks.py script already includes a task to help publishing to gh-pages. To use it, we need to install invoke. The gp-pages task also need the ghp-import package. Finally, while we do not need it for the publication task, the livereload package is useful for development, to allow calling invoke livereload.

Here is my requirements.txt file, will all those dependencies, and a few others I use for my site:

ghp-import
invoke
livereload
markdown
pelican
typogrify

On GitHub, the master branch contains the sources of the site, and the gh-pages branch contains the published website. There are other ways to do it, for instance by publishing a subdirectory as the website, but I prefer to work with different branches. The ghp-import package makes it easy to manage the gh-pages branch.

I tweaked a little bit the tasks.py file. First, I set the name of the branch where to publish:

    'github_pages_branch': 'gh-pages',

I also changed some options passed to the ghp-import script:

@task
def gh_pages(c):
    """Publish to GitHub Pages"""
    preview(c)
    c.run('ghp-import -n -b {github_pages_branch} '
          '-m {commit_message} '
          '{deploy_path}'.format(**CONFIG))

I added the -n option to create the .nojekyll file. This tells GitHub to not invoke Jekyll on the generated contents. I removed the -p option, as I need to include credentials to publish the pages.

Finally, here is the GitHub Actions configurations:

name: Publish Website
on:
push:
  branches: [ master ]
jobs:
publish:
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v2
    with:
      submodules: true
  - name: Set up Python
    uses: actions/setup-python@v1
    with:
      python-version: '3.8'
  - name: Install dependencies
    run: pip install -r requirements.txt
  - name: Build docs
    run: invoke gh-pages
  - name: Publish docs
    run: git push -f "https://${GITHUB_ACTOR}:${{secrets.TOKEN}}@github.com/${GITHUB_REPOSITORY}.git" gh-pages:gh-pages

The action run when I push to master branch.

The checkout action tells to checkout submodules as well, as I use submodules for the Flex theme, and for pelican-plugins.

I install Python packages from the requirements.txt file.

I call the invoke task to create the (local) gh-pages branch with the published contents.

Finally, I force push the gh-pages branch to GitHub, using the required credentials.

Yet Another Markup Language

2021-02-21T04:04:51-05:00

I recently needed to choose a file format for a complex software configuration. I decided to use YAML. In this post, I explaned why I choose it, and what I learned about it.

Choosing the right format for configuration file

There are plenty of file formats that exist and can be used to store a software configuration. If its not enough, it is easy to create a custom format. But who wants another standard?

(Source: https://xkcd.com/927/)

The ini file is very common, and used very frequently. In Python, it is easy to manipulate using configparser. In Qt, it can be used with the QSettings class. It is so common that each software has its own interpretation of what the ini file specification should be. It can be a good choice if you need a simple key/value storage, but if your configuration becomes more complex, you end up creating conventions over the ini file, and you finish by creating a new file format based on ini.

TOML can be see as a superset of the ini file. But the syntax is well defined, and it allow more complex representations. In fact, it almost looks like Python code. This format was chosen to store build system dependencies in Python.

JSON is a very popular format these days, mainly because it is the standard (a native) format for JavaScript. Therefore, it can be parsed very quickly in the context of a JavaScript application. However, it is less human-friendly, because you cannot really add comments to it, and because you need a tool to format it if you want to be able to understand its structure.

YAML it a superset of JSON. It means that a YAML parser can read JSON (but not the opposite: a JSON parser cannot read YAML). While more complex (and powerful) than the other format, I do not agree that it is complicated and error prone for humans. You only need to be able to indent text, and know the difference between a list and a map. With those two concepts, you should be able to work with yaml. We see YAML as the config language of many CI, such as GitHub Action, GitLab CI, Travis…

Finally, I put XML here for historical reasons. But you don’t want to work with XML, do you?

When working with Python, another option is to use native Python script as a configuration file. This is what Sphinx uses, and also what Pelican uses. But in the context of a C++ application, or with any other compiled language, this is not really a option…

	ini	toml	json	yaml	xml
Well defined		✓	✓	✓	✓
Allow comments	✓	✓		✓	✓
~~Easy~~ Fun to edit by a human	✓	✓		✓
Allow nested structures		✓	✓	✓	✓

Libraries to parse YAML

With Python, pyyaml does a pretty decent job. However, it doesn’t fully support YAML 1.2 yet.

In C++, there is yaml-cpp. It support YAML 1.2. However, it has a few problems:

It causes warnings in C++17 because it derives from std::iterator [ issue #946 ]
It cannot interfer scalar types correctly, because it doesn’t distinguish between quoted and unquoted scalars [ issue #261 ]

There is also Rapid YAML that seems promising. ~~However, it is not really usable in a projet yet since it is not versionned and has no official releases.~~ It now has a first official release, but API is not stable yet.

Charles Brunet

Automatic page publication from Pelican to GitHub Pages

Yet Another Markup Language

Choosing the right format for configuration file

Libraries to parse YAML