One of the most important aspects of the iGEM experience is the team wiki, more so this year than ever. As it occured to us that the Jamboree this year was going to be online, we realized how important wikis were going to be, and decided to make it easier for teams across the world to build their wikis.
Currently, building iGEM wikis requires teams to write code in a text box in their browsers, due to which they miss out on the wide variety of features code editors and integration tools offer these days. These tools have become the de facto standard in the web development industry, optimizing various parts of the process and ultimately making it much easier and faster.
Over the years, iGEM teams have come up with several ways to improve this experience. These solutions have focused on multiple aspects of building wikis - from making it easier to write code to uploading it and even implementing continuous integration. iGEM Toronto 2015, Waterloo 2017 and Virginia 2018 did excellent work in this area, with their wiki generators, upload APIs and CI tools. Some of these tools allowed teams to write code locally, in a code editor of their choice, and then upload it to iGEM servers for deployment.
However, we felt that there was a need for a holistic solution - something that would cover every aspect of iGEM wikis. This solution had to be comprehensive enough that teams wouldn't need to find other software to build common components, but it would still allow teams to easily integrate other libraries, should they need to do so. We identified three key areas to focus on: content, design and deployment.
We wanted to design something in line with the latest trends and best practices in the web development industry, but we also realized that these things change rapidly. This is why the software that previous years' iGEM teams built got outdated so quickly. In order to get around this, we decided to split our software into modules.
The way teams design and add content to their wikis usually varies widely, but the deployment process is common, since all iGEM wikis are powered by the MediaWiki engine. They write code in a code editor, and then manually copy and paste it in the text box on the page where it belongs. A typical iGEM wiki is anywhere between 15 to 20 pages, so it can quickly become cumbersome to copy and paste for every small change, and sometimes a few changes might just be left out in a hurry.
Deployment is one of the parts of this process that we could automate, and this is something each team would need, irrespective of which frontend web development library they use. These days in the web development industry, it is common to set up a version control system like Git, and link that to an automatic deployment service, like Github Actions or Travis. This ensures that every time a change is made in the codebase, it is automatically pushed to servers for deployment.
Github Actions and Travis are free for open source repositories, and most website hosting services offer integration with these. However, in order for iGEM teams to use such a service, an integration tool was required, that would allow CI/CD software to upload files to iGEM servers, so we decided to make one!
iGEM WikiSync
Our Python package, iGEM WikiSync allows iGEM teams to set up and continuous deployment for their wikis, making the wiki freeze a little less dreadful. It lets teams work on their wiki locally, using whichever web development framework they're familiar with, and upload the code to iGEM servers for deployment, without uploading a single file manually.
It does this in the following way:
- WikiSync looks for all assets (images, videos, PDFs, etc.), renames them according to iGEM specification, and uploads them to iGEM servers.
- It then goes through the code (HTML, CSS, JavaScript), scanning it for links.
- It replaces all local links to iGEM.org URLs where the files have been uploaded.
- Finally, all code files are also uploaded to iGEM servers and the team wiki is deployed.
WikiSync keeps track of the files uploaded in each run, so only changes are uploaded the next time it runs. It also checks code for internal broken links and shows warnings.
In order to access files on iGEM servers, WikiSync requires the user ID and password of a team member. These credentials are taken in as environment variables, so that they never have to be stored in plain text, and are discarded when the terminal session ends. When run as part of a CI/CD workflow on Github Actions or Travis, user credentials are stored as "secrets" in the repository, and CI providers ensure that they are never displayed publicly and cannot be accessed by anyone apart from the user themselves.
WikiSync is available at the following links:
- Github repository: https://github.com/igembitsgoa/igem-wikisync
- Documentation: https://igem-wikisync.readthedocs.io
- PyPI package: https://pypi.org/project/igem-wikisync/
iGEM WikiSync Github Action
By automating uploads, iGEM WikiSync allows teams to save dozens of hours over the course of their iGEM experience. This is made possible by using WikiSync to set up a continuous integration and continuous deployment routine on the wiki's version control repository, such as on Github. For example, for our own wiki, we set up continuous deployment using a Github Action we wrote for iGEM WikiSync that runs every two days, or as requested manually.
iGEM WikiSync Github Action is a Github Actions plugin that allows teams to easily set up a Github Actions workflow and integrate iGEM WikiSync into their wiki, so that everytime changes are made in the codebase, they're automatically sent to iGEM servers. All that is required is a git push.
Setting it up is a really simple process. Teams can either write their own CI workflows and include the plugin in just a few lines of code, or they can use our readymade recipes for wikis built with webpack, such as those made with our iGEM Wiki Starter Pack. All they have to do is download and include a file in their wiki repository and enable Github Actions for their wiki.
Since this Actions plugin behind-the-scenes runs iGEM WikiSync itself, it also needs access to iGEM.org credentials of a team member. These credentials are stored as "secrets" in the Github repository, so that they are never accessible to anyone apart from the user themselves. In public build logs, they are replaced with asterisks (*).
The Action can be found here along with usage instructions and common use cases.