Skip to content

Commit 01c08ce

Browse files
maartenSXMmaartenSXM
committed
doc updates
1 parent e73479b commit 01c08ce

File tree

3 files changed

+114
-144
lines changed

3 files changed

+114
-144
lines changed

‎INSTALL.md

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,32 @@
11
To install cpphash on Linux or Darwin, clone it from github
2-
and then cd into cpphash and run ./install.sh -y
2+
and then cd into cpphash and run ./install.sh
33

4-
To confirm each step, do not specify the -y option.
4+
To skiphaving to confirm each step, specify the -y option.
55

66
If cpphash is only needed for text files that are not esphome
7-
yaml, teh installation of esphome depencies can be skipped using
8-
the -s option like this:
7+
yaml, the installation of some packages that cpphash depends on
8+
can be skipped using the -s option like this:
99

10-
./install.sh -s -y
10+
./install.sh -s
1111

12-
cpphash/install.sh will install its dependencies. If -s is not
13-
specified t also creates a virtual python environment containing
14-
the latest esphome release and esphome dependencies.
12+
If -s is not specified install.sh also creates a virtual python
13+
environment containing the latest esphome release and esphome dependencies.
1514

16-
Once installed, cpphash can be activated using 'source Bashrc' for
17-
those using it with esphome. If cpphash is being used without
18-
esphome, do noyt source the Bashrc and instead set environment
19-
variable CH_HOME to the cpphash install directory using, for example:
15+
cpphash scipts can be run without any environment variables, once
16+
install.sh has comleted successfullly.
2017

21-
export CH_HOME=~/git/cpphash
18+
To use Makefile.cpphash in a project, CH_HOME has to be set to the local
19+
installation directory of cpphash. That can be done by creating an
20+
exported CH_OME environment in your shell of choice or by defining it
21+
in the project Makefile.
2222

23-
Alternatively, it can be set in Makefile.cpphash.
23+
To use Makefile.esphome in a project, CH_HOME is also needed. Sourcing
24+
cpphash/Bashrc automatically sets CH_HOME and is the recommended way
25+
to use cpphash with esphome projects.
2426

25-
Makefile.cpphash and Makefile.esphome use ./build as the build directory.
26-
That can be changed by editting the Makefile.cpphash or (Makefile.esphome
27-
and Bashrc).
27+
Makefile.cpphash and Makefile.esphome use ./build as the default
28+
build directory. That can be changed by exporting the environment
29+
variable CH_BUILD before sourcing Bashrc.
2830

2931
esp_help will list the convenience aliases defined by Bashrc.
3032

@@ -62,5 +64,5 @@ set by cpphash/Bashrc:
6264
_ESPMAKE_IP0=192.168.248.10
6365
_ESPMAKE_IP1=192.168.248.11
6466
_ESPMAKE_DEV0=/dev/ttyACM0
65-
_ESPMAKE_DEV0=/dev/ttyACM1
67+
_ESPMAKE_DEV1=/dev/ttyACM1
6668

‎README.md

Lines changed: 91 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
1-
# cpphash.sh
1+
This project enables the C pre-processor to be run on files containing hash-style comments. It provides shell scripts that can be used to integrate with any build system and it also provides Makefiles to enable make. It has optional support for esphome projects, for which it was originally written.
2+
3+
cpphash.sh
4+
==========
25

36
run the C preprocessor (cpp) on files with hash-style comments
47

5-
## Usage
8+
Usage
9+
-----
610
```
711
./cpphash.sh [-f] [-C] [-v] [-h] [-t <dir>] [-D define|define=<x>] [-I includedir] cppFile [extraFiles]...
8-
-t|--tempdir argument is a directory for temporary files. Defaults is .cpphash
12+
-t|--tempdir argument is a directory for temporary files. Default is .cpphash
913
-D|--define> add the argument as a define to pass to cpp
1014
-I|--include add the argument as an include file directory to pass to cpp
1115
-o|--outfile argument is a filename to write to. Default is <cppFile>.cpp
@@ -18,13 +22,17 @@ run the C preprocessor (cpp) on files with hash-style comments
1822
cppFile the main file to run cpp on which optionally includes <extraFiles>.
1923
[extraFiles] extra files to process that are #included by <cppFile>
2024
21-
./cpphash.sh overwrites the file specified by -o (default is <cppFile>.cpp)
25+
cpphash.sh processes <cppFile> and <extraFiles> using dehash.sh. The dehashed <extraFiles> are stored in <tempDir>. The dehashed <cppFile> is stored in <cppFile>.cpp. Then <cppFile> is run through the C pre-processor with -I <tempDir> as the first include directory path.
26+
27+
Note: ./cpphash.sh overwrites the file specified by -o (default is <cppFile>.cpp)
2228
```
2329

24-
# dehash.sh
25-
dehash.sh removes '#'-style comments
30+
dehash.sh
31+
=========
32+
dehash.sh removes '#'-style comments
2633

27-
## Usage
34+
Usage
35+
-----
2836
```
2937
dehash.sh [-c] [-b] [-h] filename
3038
-c|--cpp keep CPP directives
@@ -34,10 +42,9 @@ dehash.sh [-c] [-b] [-h] filename
3442
filename file to dehash to stout or - for stdin
3543
```
3644

37-
dehash was originally written to remove hash-style comments in text files
38-
such as yaml that require the C-preprocessor for macro features. cpphash.sh
39-
was added as a front-end for dehash.sh to do exactly that for any text files,
40-
including yaml.
45+
dehash.sh was removes hash style comments from one file. Use cpphash.sh
46+
to run dehash.sh on multiple files and to run teh C-preprocessor on one of
47+
them that includes one or more of the others.
4148

4249
Some possible dehash command variants are:
4350
```
@@ -46,136 +53,97 @@ Some possible dehash command variants are:
4653
./dehash.sh -b example.txt
4754
./dehash.sh -c -b example.txt
4855
```
49-
Note: cpphash uses GNU sed, md5sum and yq.
5056

51-
If missing, yq can be installed on Linux with:
57+
espmerge.sh
58+
===========
59+
espmerge.sh: reads esphome yaml and outputs merged esphome yaml.
5260

53-
sudo apt install yq -y
54-
or
55-
snap install yq --channel=v3/stable
56-
or
57-
apk add yq
58-
59-
On MacOS, usually all three are missing and can be installed as follows:
60-
```
61-
brew install gsed md5sha1sum yq
61+
Usage
62+
-----
6263
```
63-
## How to use this repo
64-
65-
"makefile" and "Makefile" are functional makefile templates that will clone
66-
this repo into a project.
67-
To use them, just copy them both into a project directory that does not
68-
already use makefiles. If your project already does, then you likely
69-
already know how to integrate them to your project.
70-
71-
There is an example in example/make that shows how to use make and cpphash
72-
to assist with managing multiple project variants that share text files.
73-
That example shows how to share constant #defines and configuration
74-
#defines between text files and C / C++ files. See the files main.yaml,
75-
config.h and pins.h in example/make for more details.
76-
77-
## User variables
78-
79-
These variables can be changed from their defaults by editting
80-
"Makefile" or overriding them with a CLI argument to make such as
81-
```bash
82-
make MAIN=init.yaml
64+
espmerge.sh <input.yaml >output.yaml
8365
```
8466

85-
### OUTDIR
86-
87-
Where to write generated files, including the cpphash repo itself.
88-
Unlike the other user variables, it is set in "makefile".
89-
90-
### MAIN
91-
92-
The initial/main text file that "#include"s the others. It defaults
93-
to "main.yaml".
94-
95-
### SUFFIX
96-
97-
The suffix of source files that will be dehash-ed so that they can be
98-
included by <MAIN>. <MAIN> may not end up including them all however they
99-
will be dehash-ed regardless. See user variables DIRS and SRCS. SUFFIX
100-
defaults to the file suffix of <MAIN>, including the '.'.
101-
102-
### DIRS
103-
104-
The directories that are searched for files ending in <SUFFIX> to dehash.
105-
See user variable SRCS. It defaults to ".".
106-
107-
### SRCS
108-
109-
The list of files to dehash. It defaults *.<SUFFIX> wildcards in
110-
the directories dpecified by DIRS
111-
112-
### PREFIX
113-
114-
cpphash.mk generates a single filename named <PREFIX><PROJTAG>.
115-
PREFIX defaults to "./myProj_"
116-
117-
### PROJTAG
118-
119-
PROJTAG is used to uniquely declar the project name. It defaults to "0"
120-
but it can be any character string of any length. To build different
121-
project variants from the same project directory, specify a different
122-
PROJTAG for each by overriding it on the make command line using, for
123-
example, make PROJTAG=1.
67+
espmerge.sh parses yaml into common blocks and merges sections that
68+
are repeated *and* named.
69+
70+
Common blocks are merged backwards in the output by referencing tags
71+
specified in "id: <tag>" yaml lines. The first common block to
72+
specify a unique <tag> is the destination for subsequent references.
73+
lines in subsequent blocks are merged at the end of the block
74+
that declared "id: <tag>" first. Array elements are each treated
75+
as common blocks so it is possible to merge into an array element
76+
as long as it declares itself using "id: <tag>".
77+
78+
After running espmerge.sh, it is advised to merge remaining repeated
79+
sections that are not named with "id: <tag>" by piping the espmerge.sh
80+
output through this pipe which sets up each section as a separate yaml
81+
document and then removes yaml comments;
82+
```
83+
awk '/^[[:alnum:]_]/{print "---"} | yq '... comments=""' esphome.yaml
84+
```
85+
and then piping its out into yq to merge the multiple documents using:
12486

125-
Argument -D_PROJTAG_$(PROJTAG) is passed to the C-preprocessor so that
126-
text sources can vary the generated file using #if directives such as:
127-
```code
128-
#if _PROJTAG_foo
129-
# yaml code only for project foo goes here
130-
#endif
131-
Some other C preprocessor defines are passed as well. They can be
132-
found by reviewing the CPPDEFS definition in cpphash/cpphash.mk.
87+
```
88+
yq eval-all '. as $item ireduce ({}; . *+ $item)'
89+
```
90+
For an example of how this is done, refer to the yamlmerge.sh implementation.
91+
92+
yamlmerge.sh
93+
============
94+
yamlmerge.sh: merge duplicate map keys in non-compliant yaml
95+
96+
Usage
97+
-----
98+
yamlmerge.sh: [-okseEqh] [-o outfile] <file.yaml>\n
99+
-o|--outfile File to write to, else stdout.
100+
-k|--keep Keep yaml comments.
101+
-s|--sort Sort the map keys.
102+
-e|--esphoist Hoist the esphome: and esp32: map keys to output first.
103+
-E|--espmerge Enable esphome item merging using \"id\": references.
104+
-q|--quiet Do not output the number of merged components.
105+
-h|--help Output this help.
106+
107+
<file.yaml> The yaml file to merge, else stdin.
108+
109+
yamlmerge.sh processes a single yaml files using these steps:
110+
```
111+
First, yaml comments are optionally removed using yq.
112+
Then espmerge.sh is run if requested.
113+
Then map keys are each put in their own yaml document, using awk.
114+
Then map keys are merged using yq.
115+
Then map keys are optionally sorted using yq.
116+
Then esphome map keys are optionally hoisted using yq.
133117
```
134118

135-
## Generated files
136-
cpphash.mk generates output file <OUTDIR>/<PREFIX><PROJTAG>.<SUFFIX>
137-
Intermediate C-preprocessed files used to generate it are stored in
138-
directory <OUTDIR>/<PREFIX><PROJTAG>/
119+
Installation
120+
============
139121

140-
Both can be deleted using 'make clean'.
122+
Please refer to INSTALL.md for information relating to cpphash installation.
141123

142-
## Other
124+
How to use this repo
125+
====================
143126

144-
There are some additional comments describing cpptest features in
145-
Makefile.
127+
Use the cpphash.sh, dehash.sh, espmerge.sh or yamlmerge.sh scripts
128+
individually or optionally use the Makefile.cpphash or Makefile.esphome
129+
makefiles.
146130

147-
You will note that cpphash.mk uses dehash.sh to remove the
148-
hash-style comments before running the files through the c-preprocessor.
149-
It leverages a sed script to do that and sets up dehash.sh flags
150-
to leave the C preprocessor directives.
131+
Makefile.cpphash implements a cpphash project build mechanism using make.
132+
See examples/stackoverflow for an example of how to use it,.
133+
Makefile.cpphash implements a cpphash project build mechanism using make
134+
specifically for esphome yaml files. See examples/esphome for an example
135+
of how to use it.
151136

152-
There are some aliases in file Bashrc also which may be helpful for
153-
issuing esphome commands.
137+
To use either Makefile.cpphash or Makefile.esphome, copy one of them into
138+
a project directory that does not already use make. If your project already
139+
does, then you likely already know how to integrate them into your project.
154140

155-
There is an optional esphome.mk that may be useful to those
156-
using this project with esphome projects, such as the example project.
157-
YMMV.
141+
For an example of a larger project that uses cpphash and has customized
142+
Makefile.esphome, see github.com/maartenSXM/GrowOS
158143

159144
# Credits
160145

161146
Thank you to Landon Rohatensky for the exemplary esphome yaml file
162147
https://github.com/landonr/lilygo-tdisplays3-esphome used to demonstrate
163148
configuration & build and also as used in the test subdirectory.
164149

165-
# Disclaimers
166-
167-
The author has not attempted to use cpphash with Visual Studio.
168-
169-
# MacOS Note
170-
171-
Note (repeated intentionally): on MacOS, you need GNU sed to run dehash.sh,
172-
which dehash.sh invokes. To install GNU sed, please do this:
173-
```
174-
brew install gsed
175-
```
176-
and then add this line to your .bashrc:
177-
```
178-
export PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH"
179-
```
180-
and then 'source .bashrc' or logout and log back in.
181-

‎yamlmerge.sh

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ Usage: $me: [-okseEqh] [-o outfile] <file.yaml>\n
1818
-s|--sort\tSort the map keys.
1919
-e|--esphoist\tHoist the esphome: and esp32: map keys to output first.
2020
-E|--espmerge\tEnable esphome item merging using \"id\": references.
21-
-q|--quiet\tdo not output the number of merged components.
22-
-h|--help\toutput this help.
23-
<file.yaml>\tyaml file to operate, else stdin.
21+
-q|--quiet\tDo not output the number of merged components.
22+
-h|--help\tOutput this help.
23+
<file.yaml>\tThe yaml file to merge, else stdin.
2424
2525
This script is from git repo github.com/maartenSXM/cpphash.
2626
Note: This script does not vet arguments securely. Do not setuid or host it.

0 commit comments

Comments
 (0)