Synchronize files with the official upstream source.

Author: mdadams

.github/workflows/docker_ci.yml

@@ -10,13 +10,16 @@ on:
 
 jobs:
   build-job:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     strategy:
       matrix:
-        #image: [ "ghcr.io/uvic-frodo/clang_libs-ubuntu_24-llvm_20:latest", "ghcr.io/uvic-frodo/clang_libs-fedora_41-llvm_20:latest" ]
+        # NOTE: The repository is assumed to have been granted access to the
+        # package corresponding to each of the images below.
         image:
-          - "ghcr.io/mdadams/clang_libs-ubuntu_24-llvm_20:latest"
-          - "ghcr.io/mdadams/clang_libs-fedora_41-llvm_20:latest"
+          #- "ghcr.io/uvic-aurora/clang_libs-ubuntu_24-llvm_21:latest"
+          #- "ghcr.io/uvic-aurora/clang_libs-fedora_42-llvm_21:latest"
+          - "ghcr.io/mdadams/clang_libs-ubuntu_24-llvm_21:latest"
+          - "ghcr.io/mdadams/clang_libs-fedora_42-llvm_21:latest"
         compiler:
           - gcc
           - clang

NEWS.txt

@@ -4,37 +4,54 @@ NEWS
 This document highlights some of the more significant changes from one
 release of the lecture slides (and companion Git repository) to the next.
 
+Edition 0.4.0 (2025-08-15)
+==========================
+
+Some of the more significant changes in going from Edition 0.3.0 to Edition
+0.4.0 are as follows:
+
+- The slides and code examples have been updated to be based on
+  LLVM/Clang 21.  Some of the code examples may work with earlier
+  versions of LLVM/Clang, but this is not officially supported.  The API
+  differences between different versions of LLVM/Clang are handled by
+  conditional compilation in the case of the non-slide code examples and
+  by using multiple separate source files in the case of code examples
+  that appear on slides (so as to avoid clutter).
+
+- Made a number of improvements to the code examples and added a new
+  code example.
+
 Edition 0.3.0 (2025-03-07)
 ==========================
 
 Some of the more significant changes in going from Edition 0.2.0 to Edition
 0.3.0 are as follows:
 
-* The slides and code examples have been updated to be based on
+- The slides and code examples have been updated to be based on
   LLVM/Clang 20.  Some of the code examples may work with earlier
   versions of LLVM/Clang, but this is not officially supported.  The API
   differences between different versions of LLVM/Clang are handled by
   conditional compilation in the case of the non-slide code examples and
   by using multiple separate source files in the case of code examples
   that appear on slides (so as to avoid clutter).
 
-* Added new content related to (amongst other things):
+- Added new content related to (amongst other things):
   - ASTUnit and AST serialization/deserialization
   - various node visitor classes (e.g., DeclVisitor, StmtVisitor, etc.)
   - ASTNodeTraverser
   - VFS interface
   - IntrusiveRefCntPtr
   - DynamicRecursiveASTVisitor
 
-* Added several new code examples, including (but not limited to):
+- Added several new code examples, including (but not limited to):
   - a simple text-based AST dumper based on ASTNodeTraverser
   - a simple text-based AST dumper based on RecursiveASTVisitor
   - a AST serialization and deserialization example using ASTUnit
 
-* Improved the way in which the Clang include directory is being determined
+- Improved the way in which the Clang include directory is being determined
   in various places in the code.
 
-* Made numerous smaller improvements to the slides (e.g., revised wording,
+- Made numerous smaller improvements to the slides (e.g., revised wording,
   added clarifying comments).
 
 Edition 0.2.0 (2024-01-25)
@@ -43,39 +60,39 @@ Edition 0.2.0 (2024-01-25)
 Some of the more significant changes in going from Edition 0.1.0 to Edition
 0.2.0 are as follows:
 
-* The slides and code examples have been updated to be based on
+- The slides and code examples have been updated to be based on
   LLVM/Clang 17, instead of Clang 15 (as was the case with the previous
   edition of the slides).  The code examples still continue to work with
   LLVM/Clang 15 and 16, however.  The API differences between LLVM/Clang
   15, 16, and 17 are handled by conditional compilation in the case of the
   non-slide code examples and by using multiple separate source files in
   the case of code examples that appear on slides (so as to avoid clutter).
 
-* Numerous smaller improvements to the slides were made, including adding
+- Numerous smaller improvements to the slides were made, including adding
   more hyperlinks to external LLVM/Clang documentation.
 
-* A Dockerfile was added to the Companion Git repository in order
+- A Dockerfile was added to the Companion Git repository in order
   to provide an easy-to-use environment in which to build and run the
   code examples.  This is expected to be helpful to beginners who do not
   already have a working LLVM/Clang environment.
 
-* Some bugs fixes and other improvements were made to the Clang Auxiliary
+- Some bugs fixes and other improvements were made to the Clang Auxiliary
   Library (CAL), including fixing some problems in the function used to
   determine the Clang include directory.
 
-* A new CMake find module was introduced for LLVM and Clang in an
+- A new CMake find module was introduced for LLVM and Clang in an
   attempt to reduce boilerplate needed in many CMakeLists.txt files.
 
-* Build-time introspection was added to determine whether the custom
+- Build-time introspection was added to determine whether the custom
   Fmt library is needed for std::format support.
 
-* Many improvements were made to the scripts used (either directly or
+- Many improvements were made to the scripts used (either directly or
   indirectly) for running the various code examples for demo purposes.
 
-* Many changes were made to the scripts and configuration files used
+- Many changes were made to the scripts and configuration files used
   for the GitHub CI workflow.
 
-* A formal JSON schema for compilation databases was added for reference
+- A formal JSON schema for compilation databases was added for reference
   purposes.
 
 Edition 0.1.0 (2023-08-03)
@@ -84,14 +101,14 @@ Edition 0.1.0 (2023-08-03)
 Some of the more significant changes in going from Edition 0.0 to Edition
 0.1.0 are as follows:
 
-* Approximately 120 new slides were added along with a number of new code
+- Approximately 120 new slides were added along with a number of new code
   examples in the Git repository.
 
-* Numerous improvements were made to the old content as well.
+- Numerous improvements were made to the old content as well.
 
 Edition 0.0 (2022-12-31)
 ========================
 
-* This was the initial release of the lecture slides.
+- This was the initial release of the lecture slides.
 
-* This edition of the slides is based on LLVM/Clang 15.
+- This edition of the slides is based on LLVM/Clang 15.

README.md

@@ -4,9 +4,9 @@ This repository contains all of the code examples that are associated
 with the following slide deck:
 
   - Michael D. Adams.
-    *Lecture Slides for the Clang Libraries [LLVM/Clang 20]*.
-    Edition 0.3.0,
-    Mar. 2025.
+    *Lecture Slides for the Clang Libraries [LLVM/Clang 21]*.
+    Edition 0.4.0,
+    Aug. 2025.
 
 ## Obtaining the Slide Deck
 
@@ -41,20 +41,16 @@ provided build script (which invokes these two CMake superbuilds).
 This repository employs a CI workflow based on GitHub Actions.  Each time
 a new commit is pushed, the code examples in the repository are built
 and run as a basic sanity test.  The workflow employs Podman/Docker
-container images based on Ubuntu and Fedora Linux.  The Ubuntu container
-image is based on the Ubuntu APT packages provided by the LLVM Project
-(at <https://apt.llvm.org/>).  The Fedora container image is based on
-the author's own build of LLVM (since LLVM 20 was not yet packaged
-in Fedora Linux at the time that the container image was generated).
-The code examples are built using both Clang and GCC.
+container images based on Ubuntu and Fedora Linux.  The code examples
+are built using both Clang and GCC.
 
 ## Prerequisites to Building the Software
 
 The code examples have the following software dependencies:
 
   - CMake
   - Make
-  - version 20 of LLVM/Clang
+  - version 21 of LLVM/Clang
   - GCC (if application programs are to be built with GCC)
   - Boost
   - Python
@@ -177,7 +173,7 @@ For this reason, a prebuilt version of the Podman/Docker container
 image has been made available via the following repository in the GitHub
 Container Registry:
 
-  - `ghcr.io/mdadams/clang_libs-fedora_41-llvm_20`
+  - `ghcr.io/mdadams/clang_libs-fedora_42-llvm_21`
 
 Instructions are given below on how to use this containerized environment.
 Although these instructions use (rootless) Podman, the `podman` and `docker`
@@ -203,13 +199,13 @@ be an absolute path.
    - Option 1 (Use the prebuilt image).
      To retrieve the prebuilt image, use the following command:
 
-         podman pull ghcr.io/mdadams/clang_libs-fedora_41-llvm_20
+         podman pull ghcr.io/mdadams/clang_libs-fedora_42-llvm_21
 
    - Option 2 (Build the image from scratch).
      To build the image from scratch, use the following command:
 
-         podman build --tag clang_libs-fedora_41-llvm_20 \
-           -f $TOP_DIR/podman/Dockerfile-fedora_41-llvm_20 $TOP_DIR/podman
+         podman build --tag clang_libs-fedora_42-llvm_21 \
+           -f $TOP_DIR/podman/Dockerfile-fedora_42-llvm_21 $TOP_DIR/podman
 
      Note that building the container image from scratch involves
      building LLVM, which takes a considerable amount of time.
@@ -219,7 +215,7 @@ be an absolute path.
 
        podman run -i -t --rm -v $TOP_DIR:$TOP_DIR:rw -w $TOP_DIR \
          --cap-add=SYS_PTRACE --security-opt label=disable \
-         clang_libs-fedora_41-llvm_20 /bin/bash
+         clang_libs-fedora_42-llvm_21 /bin/bash
 
    Note that the `--cap-add` and `--security-opt` options may not be
    needed.  The `SYS_PTRACE` capability is typically needed in order
@@ -234,6 +230,9 @@ be an absolute path.
 
        ./build --defaults
 
+   If you want to control the number of parallel jobs (used for
+   building), you can add the `--num-jobs` option.
+
 ## Remarks on the Use of Address Sanitizer (ASan)
 
 Sometimes the use of Address Sanitizer (ASan) can be problematic, due,

slides/examples/ast_serialization_1/CMakeLists.txt

@@ -1,21 +1,31 @@
 cmake_minimum_required(VERSION 3.14)
+
 project(ast_serialization_1 LANGUAGES CXX C)
 
 set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../cmake")
 include(CheckCXXCompilerFlag)
 include(Sanitizers)
+include(ParseVersion)
 
 #set(CMAKE_VERBOSE_MAKEFILE TRUE)
 set(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)
 
 set(CMAKE_CXX_STANDARD 20)
+set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
 
 find_package(ClangFoo REQUIRED)
+parse_version_string("${LLVM_VERSION}" LLVM_MAJOR_VERSION LLVM_MINOR_VERSION
+  LLVM_PATCH_VERSION)
 include(CheckStdFormat)
 import_std_format()
 
 add_executable(load_ast)
-target_sources(load_ast PRIVATE load_ast.cpp)
+if(LLVM_MAJOR_VERSION GREATER_EQUAL 21)
+	set(load_ast_sources load_ast.cpp)
+else()
+	set(load_ast_sources clang-20/load_ast.cpp)
+endif()
+target_sources(load_ast PRIVATE ${load_ast_sources})
 
 add_executable(save_ast)
 target_sources(save_ast PRIVATE save_ast.cpp)

slides/examples/ast_serialization_1/clang-20/load_ast.cpp

@@ -0,0 +1,109 @@
+#include <memory>
+#include <string>
+#include <clang/AST/ASTContext.h>
+#include <clang/AST/RecursiveASTVisitor.h>
+#include <clang/Basic/Diagnostic.h>
+#include <clang/Basic/DiagnosticOptions.h>
+#include <clang/Basic/FileSystemOptions.h>
+#include <clang/Basic/SourceManager.h>
+#include <clang/Frontend/ASTUnit.h>
+#include <clang/Frontend/FrontendOptions.h>
+#include <clang/Frontend/TextDiagnosticPrinter.h>
+#include <clang/Lex/Lexer.h>
+#include <clang/Serialization/PCHContainerOperations.h>
+#include <llvm/ADT/IntrusiveRefCntPtr.h>
+#include <llvm/ADT/StringRef.h>
+#include <llvm/Support/CommandLine.h>
+#include <llvm/Support/raw_ostream.h>
+
+namespace lc = llvm::cl;
+
+static lc::opt<std::string> InputASTFile(lc::Positional,
+  lc::desc("input AST file"), lc::Required);
+
+std::string getSourceText(const clang::SourceManager &sourceManager,
+  const clang::LangOptions& langOpts, clang::SourceRange sourceRange) {
+	assert(sourceRange.isValid());
+	clang::SourceLocation beginLoc = sourceRange.getBegin();
+	clang::SourceLocation endLoc = sourceRange.getEnd();
+	if (beginLoc.isMacroID() || endLoc.isMacroID()) {return "@@SOURCE::MACRO@@"; }
+	assert(beginLoc == sourceManager.getSpellingLoc(beginLoc));
+	assert(endLoc == sourceManager.getSpellingLoc(endLoc));
+	endLoc = clang::Lexer::getLocForEndOfToken(endLoc, 0, sourceManager,
+	  langOpts);
+	assert(beginLoc.isValid() && endLoc.isValid());
+	clang::FileID beginFileId = sourceManager.getFileID(beginLoc);
+	clang::FileID endFileId = sourceManager.getFileID(endLoc);
+	if (beginFileId != endFileId) {return "@@SOURCE::SPANS_FILES@@";}
+	assert(beginLoc.isValid() && endLoc.isValid());
+	unsigned beginOffset = sourceManager.getFileOffset(beginLoc);
+	unsigned endOffset = sourceManager.getFileOffset(endLoc);
+	std::size_t length = endOffset - beginOffset;
+	llvm::StringRef buffer = sourceManager.getBufferData(beginFileId);
+	return std::string(buffer.substr(beginOffset, length));
+}
+
+class MyASTVisitor : public clang::RecursiveASTVisitor<MyASTVisitor> {
+public:
+	explicit MyASTVisitor(clang::ASTUnit &astUnit) :
+	  sourceManager_(&astUnit.getSourceManager()),
+	  langOpts_(&astUnit.getLangOpts()) {}
+	std::string makeHeading(const std::string& title)
+	  {return std::format("{:=^80}\n{}\n", "", title);}
+	bool VisitDecl(clang::Decl *decl) {
+		llvm::outs() << makeHeading(std::format("Decl: {}",
+		  std::string(decl->getDeclKindName())));
+		clang::SourceRange sourceRange = decl->getSourceRange();
+		if (sourceRange.isValid()) {
+			llvm::outs() << getSourceText(*sourceManager_, *langOpts_,
+			  sourceRange) << '\n';
+		}
+		return true;
+	}
+	bool VisitStmt(clang::Stmt *stmt) {
+		llvm::outs() << makeHeading(std::format("Stmt: {}",
+		  std::string(stmt->getStmtClassName())));
+		clang::SourceRange sourceRange = stmt->getSourceRange();
+		if (sourceRange.isValid()) {
+			llvm::outs() << getSourceText(*sourceManager_, *langOpts_,
+			  sourceRange) << '\n';
+		}
+		return true;
+	}
+private:
+	clang::SourceManager* sourceManager_;
+	const clang::LangOptions* langOpts_;
+};
+
+int main(int argc, const char** argv) {
+	lc::ParseCommandLineOptions(argc, argv, "AST deserializer\n");
+	llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> diagOpts(
+	  new clang::DiagnosticOptions());
+	llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> diagIDs(
+	  new clang::DiagnosticIDs());
+	llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diagEngine(
+	  new clang::DiagnosticsEngine(diagIDs, diagOpts,
+	  new clang::TextDiagnosticPrinter(llvm::errs(), &*diagOpts), true));
+	clang::PCHContainerOperations pchContainerOps;
+	clang::FileSystemOptions fileSysOpts;
+	std::unique_ptr<clang::ASTUnit> astUnit = clang::ASTUnit::LoadFromASTFile(
+	  InputASTFile, pchContainerOps.getRawReader(),
+	  clang::ASTUnit::LoadEverything, diagEngine, fileSysOpts, nullptr);
+	if (!astUnit) {
+		llvm::errs() << "cannot load AST file\n";
+		return 1;
+	}
+	clang::ASTContext &astContext = astUnit->getASTContext();
+	clang::TranslationUnitDecl *tuDecl = astContext.getTranslationUnitDecl();
+	llvm::outs() << std::format("main file name: {}\n",
+	  std::string_view(astUnit->getMainFileName()));
+	llvm::outs() << std::format("language: {}\n",
+	  astUnit->getInputKind().getLanguage() == clang::Language::CXX ? "C++" :
+	  "other");
+	const clang::LangOptions& langOpts = astUnit->getLangOpts();
+	llvm::outs() << std::format("language standard unspecified: {}\n",
+	  langOpts.LangStd == clang::LangStandard::Kind::lang_unspecified);
+	MyASTVisitor visitor(*astUnit);
+	visitor.TraverseDecl(tuDecl);
+	return 0;
+}