Handy scripts for testing your Hugo website
Whilst developing this website and porting it over to Hugo, I have written a few handy scripts to help me test the site for errors. I am going to present the scripts with a little explanation about what each script achieves so maybe you can use them too.
Whilst my site is developed using Hugo, the scripts themselves just expect to be able to connect to a website. As such, the scripts are not tied to Hugo in any way. If you are not using Hugo on your site, you may still get some value from the scripts.
checklinks.sh
A script for validating all of the internal links in the website as well as images and the like. The script uses the linkchecker tool to validate the links.
Output of the checklinks.sh script
The script first starts up the site using docker and a Hugo container. It then waits for the server to be ready and then starts the linkchecker tool. The script is quite slow so I tend not to run it too often but it is invaluable to validate all of those pesky internal links.
The script also shows how to check external links as well.
The script now uses a linkchecker docker container so you don’t need to install linkchecker in order to use the script.
checkredirects.sh
This website has been around since late 2006 in one guise or another. In that time it has accumulated quite a lot of page moves and the like. The checkredirects.sh script checks to make sure that each page exists.
Output of the checkredirects script
Each line in the redirect file looks like this:
/xyz/ /post/xyz/
The first part is the old link, the second part is the new link. In other words, when the web server sees the first URL it sends the visitor to the second URL. The redirect file is specific to the Netlify hosting provider though it does look a lot like the nginx redirect format so I assume that is where it comes from.
rundevserver.sh
When writing new content or updating the website design, it can be helpful to see your changes immediately. The rundevserver.sh script runs the built-in Hugo server in development mode.
Output of the rundevserver script
One of the nice features of the built-in server in development mode is that it can keep track of changes whilst you are doing them and then update itself and your browser. It isn’t completely foolproof, but it works pretty well.
server_1 | Change detected, rebuilding site. server_1 | 2020-11-19 14:05:05.839 +0000 server_1 | Source changed “/src/content/posts/handy-hugo-website-test-scripts/index.md”: WRITE server_1 | Total in 415 ms
You will notice that there are two parts to running the development server. One part is the docker-compose.yml configuration file. This file tells docker compose what services it needs to run. Whilst the Hugo server only needs a single service, you could have say a web server plus a database server in the configuration file and then docker compose would then start both services for you. The second part is a very simple script that executes docker-compose.
To stop the server, you simply press CTRL C in the shell where the script is executing.
checksitemap.sh
Each website usually has a sitemap that tells the search engines about all of the content on the site as well as when the content was last changed and how often the content is likely to change.
One big error with sitemaps is to list content in the sitemap that does not exist on the website. You’d think that would be difficult to achieve on a Hugo based website but you’d be wrong. I definitely managed it when I first converted this site over to Hugo.
The script reads the sitemap from the website and then validates each link in turn. If a link is missing on the site, the script prints it out.
Conclusion
Some of these scripts will save you a ton of time. I can particularly recommend the checklinks.sh script. It is an absolute life saver when editing your site. It would take days to manually check all of the links on your site, but this little script can do the whole lot in a few minutes.
Update 2022-11-15: If you wish to check your links, there is now a much easier option available if you are deploying your site using Netlify. Netlify have a plugin called netlify-plugin-checklinks you can use that checks your website links after each build. If one or more bad links are found, your build will fail and the new version of your site will not go live. There is enough documentation on the project github page to get you started.
Update 2022-02-24: Updates checklinks.sh script to use the checklinks docker container saving you from having to install the linkchecker program.
