VK4PK Maintaining the LyonsComputer Web Site

  1. Prerequisites
  2. Setting up the Platform Environment
  3. The bldidx.sh script
  4. Setup the sitecopy configuration file
  5. Using "checksum" State to Control Uploads
  6. Initialising Sitecopy
  7. scripts
  8. The "bldidx.sh" Script echo "
  9. $item_no_ext$SP4[Home]" >>$OUTFILE echo "
  10. $item$SP4[Home]" >>$OUTFILE
  11. Recovering from "sitecopy" Database Corruptions
  12. Link Checkers
  13. Usefull Linkchecker Commands
  14. References



  1. ARADIO enviorment variable set to the Amateur Radio Folder
  2. $ARADIO/bin on Path
  3. The application linkchecker and sitecopy must be installed
  4. "bldisx.sh" and "h3.sh" scripts in $ARADIO/bin folder
  5. Diskspace and permission to create temporary folder $HOME/www-build
  6. Create and populated $HOME/.sitecopyrc

Setting up the Platform Environment

Add an environment variable, "ARADIO" to .bashrc file in the home directory


The bldidx.sh script

The bldidx.sh script requires linkchecker and sitecopy applications.

Run the bldidx.sh script to make a temporary copy of the html file tree to the ~/www-build folder. Only required html files and their dependant files (images, pdf's, etc) will be copied.


Setup the sitecopy configuration file

The files state is stored in the files in ~/.sitecopy folder and is used to discover when a file has been changed. Two methods are supported, and can be selected using the state option, with either parameter: "timesize" (the default), and "checksum".

"timesize" uses the last-modification date and the size of files to detect when they have changed.
"checksum" uses an MD5 checksum to detect any changes to the file contents.

Note that MD5 checksumming involves reading in the entire file, and is slower than simply using the last-modification date and size. It may be useful for instance if a versioning system is in use which updates the last-modification date on a 'checkout', but this doesn't actually change the file contents.
Source: "man sitecopy"

*** NOTE the .sitecopy can only be implemented on one workstation for the site state
to be current. ***

Setup sitecopy's configuration file, ~./sitecopyrc


Using "checksum" State to Control Uploads

Important Note: Because the bldidx script copies all web page files to a temporary build directory if the "timesize" method is used, sitecopy will copy every file to the live website every time. Therefore, the "checksum" method is required if only modified files are to be uploaded.


Initialising Sitecopy

Sitecopy must be initialises before use. This will create a status file in the ~/.sitecopy folder with the domain name of the website.

Initialise Sitecopy:

Run bldidx.sh to populate the www-build folder with files. This will only be files required to run the web page. That is all files below index.html.



Two scripts have to be created.
  1. bldidx.sh - Populates ~/www-build with only those files required for the web site.
  2. h3.sh - create Contents from level 3 headings
  3. h3-all.sh
  4. bartab.sh - generate a table
Cut and paste the textareas below to $ARADIO/bin folder.


The "bldidx.sh" Script


Recovering from "sitecopy" Database Corruptions

If the the uploaded site becomes out of sync with the local database run the commands:

/home/glenn/www-build This can occur when an sitecopy update is interrupted or fails to complete. The fetch option can take some time as it has to recalculate the checksum for every file and folder on the server and update the production servers sitecopy database. The update may have to be run a few times to complete the process.


Link Checkers

W3C - Check links and anchors in Web pages or full Web sites:

LinkChecker - Check websites for broken links:


Usefull Linkchecker Commands



sitecopy - maintain remote copies of web sites:
Man page: https://linux.die.net/man/1/sitecopy

Maintaining remote Websites with Sitecopy on Ubuntu 18.04

Glenn Lyons VK4PK
Ver:gnl20181201 - pre published v0.9