Added SiteDiff to development environment (#72)

samlikins · web-flow · commit ed3af15ec30e · 2023-04-03T16:23:14.000-07:00
- Added Docker service for SiteDiff
- Added configuration for SiteDiff to dedicated `dev` directory
- Moved `Dockerfile` for Jekyll service to dedicated `dev` directory
- Added BPKG commands for SiteDiff processes
- Updated GIT ignore to handle SiteDiff runtime files
- Updated README for SiteDiff processes
- Updated README for missing LESSC command
diff --git a/.gitignore b/.gitignore
@@ -5,6 +5,9 @@ _site/*
 _theme_packages/*
 .jekyll-cache/*
 
+/dev/sitediff/*
+!/dev/sitediff/config.yaml
+
 Thumbs.db
 .DS_Store
 
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-# bpkg.github.io
+# bpkg.github.io (bpkg.sh)
 
 `bpkg` is a lightweight bash package manager.
 This repository contains the source code of it's homepage.
@@ -23,28 +23,37 @@ In the remainder of this document, if there is a accompanying `bpkg` _command_ i
 
 ## Docker Environment
 
-This environment is not necessary to use if you'd rather run everything directly on your host system.  This environment is provided however, to facilitate all the required components for building and development of `bpkg` site.  There are associated `bpkg` commands to interact with the Docker environment.
+This environment is not necessary to use if you'd rather run everything directly on your host system.  This environment is provided however, to facilitate all the required components for building and development of `bpkg` site.
+
+A `docker-composer.yml` file exists in the root directory of the project to specify the Docker services
+
+* `jekyll` &ndash; Perform Jekyll and Rake processes
+  > **NOTE:** _A custom `Dockerfile` exists to generate the image for this service, in the `dev/jekyll` directory_
+* `lessc` &ndash; Implements LESS preprocessing
+* `sitediff` &ndash; Host SiteDiff for performing site comparison
+
+There are associated `bpkg` commands to interact with the Docker environments, accompanying Docker commands will be listed like the other `bpkg` commands.
 
 ## Dependencies
 
 This website was made with the [Jekyll][jekyll] engine, so it depends on a few
-Ruby Gems. To install them, run the following command.
+Ruby Gems. To install them, run the following command:
 
 ```bash
 $ gem install jekyll
 ```
 
 It might take a while to finish, but once it does you're ready to go.
 
-## How to edit
+## Edit Site Pages
 
 To **make changes** to the page or **run it locally**, clone this GitHub
 repository and make sure you have _installed the dependencies_ above.
 
 Then, it's a matter of editing pages and running `rake` tasks.  Here's a rundown
-of possible commands (thanks to [this great quickstart on Jekyll][tuto]):
+of possible commands (thanks to [this great quickstart on Jekyll][tuto]).
 
----
+### Preview Site
 
 ```bash
 $ rake preview
@@ -56,12 +65,12 @@ $ rake preview
 Builds the entire site to a local folder `_site` and launches a webserver to
 preview it.
 
-To see the full site, point your browser to `localhost:4000`.
+To see the full site, point your browser to [`localhost:4000`][preview-url].
 
 If you make any changes on any files, it will regenerate the website
 automatically.
 
----
+### Create Post
 
 ```bash
 $ rake post title="Hello, World!"
@@ -76,7 +85,7 @@ the date is the current, by default.
 No further changes are required, the post will get automatically inserted on the
 site.
 
----
+### Create Page
 
 ```bash
 $ rake page name="about"
@@ -96,6 +105,47 @@ $ rake page name="about.html"
 
 Alternative way to create a new page, on this case it will be `./about.html`.
 
+## Update Style Sheets
+
+This project uses [LESS (Leaner Style Sheets)][less] files to generate the CSS for this site.  When modifying the site style sheets under the `/assets` directory, update the LESS files (ie: `*.less`).  To update the CSS file after making changes to any of the LESS files, run the following command (from the root of the project):
+
+```bash
+$ lessc ./assets/themes/the-program/css/style.less ./assets/themes/the-program/css/style.css
+```
+
+> **BPKG:** `lessc`<br />
+> **BPKG:** `docker-lessc`
+
+## View Site Changes
+
+When working on the site, sometimes it's helpful to see changes between the production copy and the output of local changes.  A report of changes can be generated with the [SiteDiff][sitediff] tool, to simplify this process a Docker service has been included with a few `bpkg` commands.  This is a multi-step process (each step has an associated 'bpkg' command), but a single command can be used to run all the steps.  Before performing these steps the local version needs to be in [preview mode](#preview-site).
+
+### SiteDiff - Crawl Sites
+
+The SiteDiff utility will crawl the production and the preview versions of the site prior to performing the comparison:
+
+> **BPKG:** `site-crawl`
+
+### SiteDiff - Diff Crawled Sites
+
+After crawling the versions of the sites, SiteDiff will compare the preview against the production version and generate a change report:
+
+> **BPKG:** `site-diff`
+
+### SiteDiff - Serve Report
+
+The generated report can be served for viewing by SiteDiff:
+
+> **BPKG:** `site-serve`
+
+To see the report, point your browser to [`localhost:13080`][report-url].
+
+### SiteDiff
+
+The preceding command performs the following steps (which can be run individually):
+
+> **BPKG:** `sitediff`
+
 ## Notes
 
 * When producing content (writing pages/posts) keep in mind
@@ -111,13 +161,17 @@ Alternative way to create a new page, on this case it will be `./about.html`.
 This site uses [Jekyll Bootstrap][boots] with a heavily customized version of
 [the_program][theme] theme, originally made by [Yuya Saito][saito].
 
-[home]:    https://bpkg.sh/
-[hub]:     https://github.com/bpkg/bpkg
-[org]:     https://github.com/bpkg
-[jekyll]:  https://jekyllrb.com/
-[tuto]:    http://jekyllbootstrap.com/usage/jekyll-quick-start.html
-[posts]:   https://jekyllrb.com/docs/posts/
-[intro]:   http://jekyllbootstrap.com/lessons/jekyll-introduction.html
-[boots]:   http://jekyllbootstrap.com/
-[theme]:   https://github.com/jekyllbootstrap/theme-the-program
-[saito]:   https://studiomohawk.com/
+[home]:        https://bpkg.sh/
+[hub]:         https://github.com/bpkg/bpkg
+[org]:         https://github.com/bpkg/
+[jekyll]:      https://jekyllrb.com/
+[tuto]:        http://jekyllbootstrap.com/usage/jekyll-quick-start.html
+[preview-url]: http://localhost:4000/
+[less]:        https://lesscss.org/
+[sitediff]:    http://sitediff.io/
+[report-url]:  http://localhost:13080/
+[posts]:       https://jekyllrb.com/docs/posts/
+[intro]:       http://jekyllbootstrap.com/lessons/jekyll-introduction.html
+[boots]:       http://jekyllbootstrap.com/
+[theme]:       https://github.com/jekyllbootstrap/theme-the-program/
+[saito]:       https://studiomohawk.com/
diff --git a/bpkg.json b/bpkg.json
@@ -10,6 +10,11 @@
     "page":           "rake page $@",
     "lessc":          "lessc ./assets/themes/the-program/css/style.less ./assets/themes/the-program/css/style.css",
 
+    "site-crawl":     "docker-compose run --rm --service-ports sitediff crawl --directory=./ ./config.yaml",
+    "site-diff":      "docker-compose run --rm --service-ports sitediff diff --directory=./ ./config.yaml",
+    "site-serve":     "docker-compose run --rm --service-ports sitediff serve --directory=./ ./config.yaml",
+    "sitediff":       "bpkg run site-crawl; bpkg run site-diff; bpkg run site-serve",
+
     "docker":         "docker-compose run --rm --service-ports jekyll $@",
 
     "docker-shell":   "bpkg run docker /bin/bash",
diff --git a/dev/jekyll/Dockerfile b/dev/jekyll/Dockerfile
diff --git a/dev/sitediff/config.yaml b/dev/sitediff/config.yaml
@@ -0,0 +1,85 @@
+## Documentation:
+## https://github.com/evolvingweb/sitediff
+
+## Include other configuration files, merging them with this file.
+includes:
+  # - extra-rules.yaml
+
+## Settings.
+##
+## If you use "sitediff init" with the right parameters, it will generate
+## this section for you.
+settings:
+  ## Crawl 4 levels deep.
+  depth: 4
+
+  ## Wait for 250ms between requests.
+  # interval: 250
+
+  ## Make only 1 request at a time - no simultaneous requests.
+  ## Concurrency has to be one when an interval is set.
+  # concurrency: 1
+
+  ## Don't follow links to XML files.
+  exclude: !ruby/regexp /^\/feed\//
+
+  ## Curl options, if any.
+  curl_opts:
+    # max_recv_speed_large: 10000
+
+## Rules under this element apply only to the 'before' site.
+before:
+  ## URL of the 'before' version of the site.
+  url: https://bpkg.sh
+
+  ## Sanitizations and DOM transformations, just like the general ones
+  ## demonstrated below, but applied only to the 'before' site.
+  sanitization:
+    - title: Remove Google analytics script (added in production)
+      pattern: !ruby/regexp /\n[ \t]*<script[^<]+\.google-analytics\.com\/ga\.js[^<]+<\/script>[ \t]*/
+
+  dom_transform:
+
+## Rules under this element apply only to the 'after' site.
+after:
+  ## URL of the 'after' version of the site.
+  url: http://host.docker.internal:4000
+
+  ## Sanitizations and DOM transformations, just like the general ones
+  ## demonstrated below, but applied only to the 'after' site.
+  sanitization:
+    - title: Remove JavaScript development variable (added in development)
+      pattern: !ruby/regexp /\n[ \t]*var disqus_developer = 1;/
+
+  dom_transform:
+
+## The root element to compare.
+##
+## Usually, sitediff compares the HTML of the entire page. If you'd rather
+## check just a subset of the page, specify a selector here. For example, the
+## line below causes only the body to be compared, ignoring the HTML head.
+# selector: 'body'
+
+## Ignore whitespace when doing the diff.
+# ignore_whitespace: true
+
+## General regular expression rules, applied to both versions of the site.
+sanitization:
+  - title: Remove timezone offset from publish date-time (can differ from local and production)
+    selector: date.date-pub
+    pattern: !ruby/regexp /(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}).\d{2}:\d{2}/
+    substitute: '\1_TZ_OFFSET_'
+
+## General DOM transforms, applied to both versions of the site.
+dom_transform:
+
+## Report settings allow you to display helpful details on the report.
+report:
+  title: BPKG Packages Site - for the bpkg project
+  details: This report outlines visual differences between the published (production) version and the locally generated version.
+  before_note: Production Site
+  after_note: Locally Generated
+  # before_url_report: https://bpkg.sh
+  # after_url_report: http://localhost:13080/cache/after
+  # before_url_report: false
+  after_url_report: http://localhost:4000
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -3,14 +3,27 @@ version: "3"
 services:
   jekyll:
     build:
-      context: ./dev/
+      context: ./dev/jekyll/
     volumes:
       - ./:/srv/jekyll:Z
     ports:
       - "4000:4000"
     stdin_open: true
     tty: true
+
   lessc:
     image: finalgene/lessc
     volumes:
       - ./:/app
+
+  sitediff:
+    image: evolvingweb/sitediff
+    volumes:
+      - ./dev/sitediff:/sitediff
+    # network_mode: host
+    ports:
+      - "13080:13080"
+    stdin_open: true
+    tty: true
+    entrypoint: sitediff
+    command: help
