KeyError: 'domain' exception being thrown from PropagateDescDomain.run

[bneradt]

Describe the bug
Apache Traffic Server docs are built using Sphinx. We noticed that when we upgraded to the 4.x release the build breaks with the following output:

# Sphinx version: 4.0.2
# Python version: 3.8.3 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 3.0.1
# Last messages:
#   writing output... [  2%] admin-guide/configuration/transparent-proxy/wccp-service-config.en
#   writing output... [  2%] admin-guide/configuring-traffic-server.en
#   writing output... [  2%] admin-guide/files/cache.config.en
#   writing output... [  3%] admin-guide/files/hosting.config.en
#   writing output... [  3%] admin-guide/files/index.en
#   writing output... [  3%] admin-guide/files/ip_allow.yaml.en
#   writing output... [  3%] admin-guide/files/logging.yaml.en
#   writing output... [  3%] admin-guide/files/parent.config.en
#   writing output... [  4%] admin-guide/files/plugin.config.en
#   writing output... [  4%] admin-guide/files/records.config.en
# Loaded extensions:
#   sphinx.ext.mathjax (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/alabaster/__init__.py
#   sphinx.ext.graphviz (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/graphviz.py
#   sphinx.ext.intersphinx (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/intersphinx.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.todo (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/todo.py
#   sphinx.ext.coverage (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/coverage.py
#   sphinx.ext.viewcode (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/viewcode.py
#   sphinxcontrib.plantuml (unknown version) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/plantuml.py
#   traffic-server (unknown version) from /var/tmp/trafficserver/doc/ext/traffic-server.py
#   sphinx.ext.imgmath (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/imgmath.py
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/application.py", line 350, in build
    self.builder.build_update()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 292, in build_update
    self.build(to_build,
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 356, in build
    self.write(docnames, list(updated_docnames), method)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 530, in write
    self._write_serial(sorted(docnames))
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 537, in _write_serial
    doctree = self.env.get_and_resolve_doctree(docname, self)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/environment/__init__.py", line 527, in get_and_resolve_doctree
    self.apply_post_transforms(doctree, docname)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/environment/__init__.py", line 573, in apply_post_transforms
    transformer.apply_transforms()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/__init__.py", line 86, in apply_transforms
    super().apply_transforms()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/post_transforms/__init__.py", line 42, in apply
    self.run(**kwargs)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/post_transforms/__init__.py", line 252, in run
    node['classes'].append(node.parent['domain'])
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/docutils/nodes.py", line 625, in __getitem__
    return self.attributes[key]
KeyError: 'domain'

If we pin the Sphinx package to 3.5.4, however, the docs build fine.

To Reproduce
Steps to reproduce the behavior:

docker pull controller.trafficserver.org/ats/centos:8
docker run --name build_ats_docs --init --cap-add=SYS_PTRACE --network=host -d -t controller.trafficserver.org/ats/centos:8
docker exec -it build_ats_docs /bin/bash

cd /var/tmp
git clone https://github.com/apache/trafficserver.git
cd trafficserver

# Checkout the following commit as a Sphinx package pin workaround will soon be merged.
git checkout 1177cc7056294a3b1aaa506456a76cdf6dcc9b52

source /opt/rh/gcc-toolset-9/enable
yum install -y java
cd doc

#-------------------------------------------------------
# Note: at the present time this will install Sphinx 4.0.2:
#-------------------------------------------------------
pipenv install


pipenv shell

cd ..
autoreconf -fi
./configure --enable-docs
cd doc
make html

Some environment information:

(doc) [root@docker-desktop doc]# pip freeze
alabaster==0.7.12
Babel==2.9.1
certifi==2021.5.30
chardet==4.0.0
click==8.0.1
docutils==0.16
idna==2.10
imagesize==1.2.0
Jinja2==3.0.1
lxml==4.6.3
MarkupSafe==2.0.1
packaging==20.9
polib==1.1.1
Pygments==2.9.0
pyparsing==2.4.7
pytz==2021.1
requests==2.25.1
snowballstemmer==2.1.0
Sphinx==4.0.2
sphinx-intl==2.0.1
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-plantuml==0.21
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
urllib3==1.26.5
(doc) [root@docker-desktop doc]# python --version
Python 3.8.3

Expected behavior
If the Sphinx package is pinned to 3.5.4, the build succeeds. Instead, with 4.0.2, it fails with the 'domain' KeyError exception.

Your project
https://github.com/apache/trafficserver/

(See the above clone and checkout git commands to reproduce this.)

Screenshots
Not applicable.

Environment info
The above replay steps show how to reproduce the environment I'm seeing this with using docker.

• OS: CentOS 8, docker image: controller.trafficserver.org/ats/centos:8
• Python version: 3.8.3
• Sphinx version: 4.0.2
• Sphinx extensions:
• Extra tools:

Additional context

From Traffic Server's perspective, this is being tracked via the following Apache Traffic Server issue which will soon be worked around by pinning our Sphinx release to 3.x:

• Building the sphinx docs fails with the latest sphinx version apache/trafficserver#7938

[tk0miya]

The current implementation of PropagateDescDomain expects a parent of the desc_signature node is a desc node and it has "domain" attribute. But it's not a restriction. So I'll fix the implementation later.