Continue the documentation effort!

- Many, many updates to the public header files.
- tidyenum.h was reorganized substantially in order to better generate
  documentation with Doxygen.
- This was also a good time to clean up all of the various enums for languages
  and strings. Everything is simple and in a single enum now, other than a
  couple of cases (TidyOptionId, for example, doesn't need to be redefined).
- A full and complete audit of the strings meant some opportunities to delete
  useless strings.
- Reorganized the order of the strings in language_en.h in order to better
  find things when programmers want to make changes. There are a lot fewer
  internal "sections" now, and everything has been painstakingly sorted within
  the remaining sections.
- Consequently rebased all of the PO's, POT, and other language files.
- Updated several of the READMEs with the newest information.
- Made the READMEs easier to copy into the Doxygen project by changing some of
  the code format for compatibility, mainly the use of tildes instead of
  backslashes for code blocks.
- Added tidyGetMessageCode() to message API. Despite the huge diff, this is the
  only externally-visible change, other than removing some enums (but not their
  values!).
- Passing `next` tests on Mac, Linux, Win10.

Author: balthisar

README/API_AND_NAMESPACE.md

@@ -66,7 +66,7 @@ All of the information above is useful for anyone who wants to browse Tidy's sou
 
 ## Behind the scenes
 
-The first thing we need to do is have the internal version of the function that we want to add. Tidy has module that handles localization: `language.h/c`. In the header is where we define the interface to LibTidy, which should be namespaced according to the discussion above. We can declare:
+The first thing we need to do is have the internal version of the function that we want to add. Tidy has a module that handles localization: `language.h/c`. In the header is where we define the interface to LibTidy, which should be namespaced according to the discussion above. We can declare:
 
 ~~~
 ctmbstr TY_(tidyLocalizedString)( uint messageType );
@@ -78,7 +78,7 @@ Now you have a decision to make: if you plan to use this function internally, yo
 
 ## The API
 
-Once implemented, we want a pretty, public-facing name for our `tidyLocalizedString()` function, which appropriate is `tidyLocalizedString()`. Add the declaration to `tidy.h`:
+Once implemented, we want a pretty, public-facing name for our `tidyLocalizedString()` function, which appropriately is `tidyLocalizedString()`. Add the declaration to `tidy.h`:
 
 ~~~
 TIDY_EXPORT ctmbstr TIDY_CALL tidyLocalizedString( uint messageType );
@@ -102,4 +102,3 @@ For a more complicated example that demonstrates how to use opaque types (and al
   - implement iteration for structures with multiple records.
   - write a function in `tidylib.c` that converts between the exposed, opaque type and the internal, implementation type.
   - further reinforce how functionality is added to the API.
-

README/ATTRIBUTES.md

@@ -11,13 +11,13 @@ Tidy’s large number of attributes are supported via number of files:
 
 So, to add a new `attribute=value`, on one or more existing tags, consists of the following simple steps -
 
- 1. `tidyenum.h` - Give the attribute an internal name, like `TidyAttr_XXXX`, and thus a value. While there were some initial steps to keep this `TidyAttrId` enumeration alphabetic, now just add the new `TidyAttr_XXXX` just before the last entry `N_TIDY_ATTRIBS`.
+ 1. `tidyenum.h` - Give the attribute an internal name, like `TidyAttr_XXXX`, and thus a value. Please try to keep this enumeration in alphabetical order.
 
  2. `attrs.c` - Assign the string value of the attribute. Of course this must be unique. And then assign a `function` to verify the attribute value. There are already a considerable number of defined functions to verify specific attribute values, but maybe this new attribute requires a new function, so that should be written, and defined.
 
  3. `attrdict.c` - If this attribute only relates to specific tags, then it should be added to their list. There are some general attributes that are allowed on every, or most tags, so this new attribute and value should be added accordingly.
 
- 4. `tags.c` - Now the new attribute will be verified for each tag it is associate with in the `tag_defs[]` table. Like for example the `<button ...>`, `{ TidyTag_BUTTON, ...` has `&TY_(W3CAttrsFor_BUTTON)[0]` assigned.
+ 4. `tags.c` - Now the new attribute will be verified for each tag it is associated with in the `tag_defs[]` table. Like for example the `<button ...>`, `{ TidyTag_BUTTON, ...` has `&TY_(W3CAttrsFor_BUTTON)[0]` assigned.
 
 So, normally, just changing 3 files, `tidyenum.h`, `attrs.c`, and `attrdict.c`, will already adjust `tags.c` to accept a new `attribute=value` for any tag, or all tags. Simple...
 

README/BUILD.md

@@ -45,18 +45,18 @@ See the `CMakeLists.txt` file for other CMake **options** offered.
 Due to API changes in the PHP source, `buffio.h` needs to be renamed to `tidybuffio.h` in the file `ext/tidy/tidy.c` in PHP's source.
 
 That is - prior to configuring PHP run this in the PHP source directory:
-```
+~~~
 sed -i 's/buffio.h/tidybuffio.h/' ext/tidy/*.c
-```
+~~~
 
 And then continue with (just an example here, use your own PHP config options):
 
-```
+~~~
 ./configure --with-tidy=/usr/local
 make
 make test
 make install
-```
+~~~
 
   [1]: http://git-scm.com/book/en/v2/Getting-Started-Installing-Git
   [2]: http://www.cmake.org/download/

README/MESSAGES.md

@@ -4,18 +4,23 @@ Tidy has quite complex warning/error messaging system. This is all about adding
 
 First assign the message a key value. This is done in `tidyenum.h`, in one of the two enumerations that are listed there.
 
- 1. `tidyMessageCodes` - starts with the value `tidyMessageCodes_first = 500`, and it must be first. These are messages that appear in Tidy's report list, the list that's emitted telling you what Tidy did to your code. **However** don't modify this enum directly. You'll modify a preprocessor macro instead.
+ 1. `tidyStrings` - starts with the value `TIDYSTRINGS_FIRST = 500`, and it must be first. This is the list of all strings available in Tidy with the exception of strings provided by other enumerations. **However** don't modify this enum directly. You'll modify a preprocessor macro instead.
 
- 2. `tidyMessagesMisc` - starts with the value `tidyMessagesMisc_first = tidyMessageCodes_last`. These are messages that are emitted that tell you general information, such as further advice.
+ 2. `TidyOptionId` - You probably won't need this unless you're adding new options, and there's another readme for that.
+ 
+ 3. `TidyConfigCategory` - You probably won't need this, either, unless you're adding a whole new category for options.
+ 
+ 4. `TidyReportLevel` - And you probably won't need this, either.
+
 
-All enum values are only ever used by name within **libTidy** (and incidentally, should only ever be used by name in your client applications; never trust the value!), so feel free to enter new strings wherever they make the most sense. There are already existing categories (marked by comments), or feel free to create a new category if that's best.
+All enum values are only ever used by name within **libTidy** (and incidentally, should only ever be used by name in your client applications; never trust the value!), so feel free to enter new strings wherever they make the most sense. 
 
-As mentioned above, `tidyMessageCodes` messaged must be defined in one of the existing macros named like `FOREACH_...(FN)`, such as `FOREACH_MSG_ENTITIES(FN)`. These macros ensure that another data structure used for localization and key lookup is updated automatically any time strings are added or removed, thus limiting the possibility of developer error.
+As mentioned above, `tidyStrings` messages must be defined in one of the existing macros named like `FOREACH_...(FN)`, such as `FOREACH_DIALOG_MSG(FN)`. These macros ensure that another data structure used for localization and key lookup is updated automatically any time strings are added or removed, thus limiting the possibility of developer error.
 
 
 ## Step 1
 
-So in this case I want to add 3 warning messages: `BAD_SURROGATE_PAIR`, `BAD_SURROGATE_TAIL`, and `BAD_SURROGATE_LEAD`. Because these are error messages, they belong in the `tidyErrorCodes` enum, and they fit into nicely into the macro beginning `FOREACH_MSG_ENCODING(FN)`. 
+So in this case I want to add 3 warning messages: `BAD_SURROGATE_PAIR`, `BAD_SURROGATE_TAIL`, and `BAD_SURROGATE_LEAD`. Because these are error messages, they belong in the `tidyStrings` enum, and they fit into nicely into the macro beginning `FOREACH_DIALOG_MSG(FN)`. Please look at the comments for the category which indicates which output routine is used to generate the error, and keep this list in order!
 
 
 ## Step 2
@@ -24,7 +29,7 @@ The next step is adding a `format` string to `language_en.h`. This string may la
 
 Where to add this seems a bit of a mess, but in general things are grouped by where they're used in `libTidy`, and often in alphabetical order within those groups. Here I've added them relative to where they were placed in the other enums and structs.
 
-Depending on which of the output routines you use (consult `message.c`) you may be able to use parameters such as `%u` and `%s` in your format strings. The available data is currently limited to the available message output routines, but perhaps generalizing this in order to make more data available will be a nice focus of Tidy 5.5. Please don't use `printf` for message output within **libTidy**.
+Depending on which of the output routines you use (consult `message.c`) you may be able to use parameters such as `%u` and `%s` in your format strings. The available data is currently limited to the available message output routines. Please don't use `printf` for message output within **libTidy**.
 
 
 eof;

README/OPTIONS.md

@@ -4,32 +4,32 @@ Tidy supports a quite large number of configuration options. The full list can b
 
 The options can also be listed in xml format. `-xml-help` will output each option plus a description. `-xml-config` will not only output the option and a desciption, but will include the type, default and examples. These xml outputs are used, with the aid of `xsltproc` and `doxygen`, to generate the [API Documentation](http://api.html-tidy.org/).
 
-These options can also be used by application linking with **`libtidy`**. For each option there is a `TidyOptionId` enumeration in the `tidyenum.h` file, and get/set functions for each option type.
+These options can also be used by application linking with `libtidy`. For each option there is a `TidyOptionId` enumeration in the `tidyenum.h` file, and get/set functions for each option type.
 
 This file indicates how to add a new option to tidy, here adding an option `TidyEscapeScripts`. In essence it consists of 4 steps:
 
- 1. Add the option **`ID`** to `tidyenum.h`.
- 2. Add to the **`table`** `TidyOptionImpl option_defs[]` in `config.c`
- 3. Add the id, with a **`description`** to `language_en.h`
+ 1. Add the option `ID` to `tidyenum.h`.
+ 2. Add to the `table` `TidyOptionImpl option_defs[]` in `config.c`
+ 3. Add the id, with a `description` to `language_en.h`
  4. Use the option in the code.
 
 #### 1. Option ID
 
-In `tidyenum.h` the `TidyOptionId` can be in any order, but normally a new option would be added just before the last `N_TIDY_OPTIONS`, which must remain the last. Choosing the id name can be any string, but by convention it will commence with `Tidy` followed by brief descriptive text.
+In `tidyenum.h` the `TidyOptionId` can be in any order, but please try to keep things alphabetical, and keep in mind that `N_TIDY_OPTIONS` must remain the last. Choosing the id name can be any string, but by convention it will commence with `Tidy` followed by brief descriptive text.
 
 Naturally it can not be the same as any exisitng option. That is, it must be unique. And it will be followed by a brief descriptive special doxygen formatted comment. So for this new option I have chosen -
 
-```
+~~~
   TidyEscapeScripts,       /**< Escape items that look like closing tags */
-```
+~~~
 
 #### 2. Table Definition
 
 In `config.c`, added in `TidyOptionImpl option_defs[]`. Again it can be in any order, but normally a new option would be added just before the last `N_TIDY_OPTIONS`, which must remain the last.
 
 The structure definition of the table entries is simple -
 
-```
+~~~
 struct _tidy_option
 {
     TidyOptionId        id;
@@ -41,93 +41,94 @@ struct _tidy_option
     const ctmbstr*      pickList;   /* pick list */
     ctmbstr             pdflt;      /* default for TidyString */
 };
-```
+~~~
 
-Naturally it will commence with the above chosen unique **`id`**.
+Naturally it will commence with the above chosen unique `id`.
 
-The **`category`** will be one of this enumeration -
+The `category` will be one of this enumeration -
 
-```
+~~~
 typedef enum
 {
   TidyMarkup,          /**< Markup options: (X)HTML version, etc */
   TidyDiagnostics,     /**< Diagnostics */
   TidyPrettyPrint,     /**< Output layout */
   TidyEncoding,        /**< Character encodings */
-  TidyMiscellaneous    /**< File handling, message format, etc. */
+  TidyMiscellaneous,   /**< File handling, message format, etc. */
+  TidyInternalCategory /**< Option is internal only. */
 } TidyConfigCategory;
-```
+~~~
 
 Care, each of these enumeration strings have been equated to 2 uppercase letters. If you feel there should be another `category` or group then this can be discussed, and added.
 
-The **`name`** can be anything, but should try to be somewhat descriptive of the option. Again this string must be unique. It should be lowercase alphanumeric characters, and can contain a `-` separator. Remember this is the name places on the command line, or in a configuration file to set the option.
+The `name` can be anything, but should try to be somewhat descriptive of the option. Again this string must be unique. It should be lowercase alphanumeric characters, and can contain a `-` separator. Remember this is the name places on the command line, or in a configuration file to set the option.
 
-The **`type`** is one of the following enumeration items -
+The `type` is one of the following enumeration items -
 
-```
+~~~
 typedef enum
 {
   TidyString,          /**< String */
   TidyInteger,         /**< Integer or enumeration */
   TidyBoolean          /**< Boolean flag */
 } TidyOptionType;
-```
+~~~
 
 Care, each of these enumeration strings have been equated to two uppercase letters. If you feel there should be another `type` then this can be discussed, but would require other additional things. And also note the `TidyTriState` is the same as a `TidyInteger` except uses its own parser.
 
-The next item is the **`default`** value for a boolean, tristate or integer. Note tidy set `no=0` and `yes=1` as its own `Bool` enumeration.
+The next item is the `default` value for a boolean, tristate or integer. Note tidy set `no=0` and `yes=1` as its own `Bool` enumeration.
 
-There are a number of **`parser`** for the options. Likewise a number of **`pickList`**. Find another option similar to your new option and use the same values.
+There are a number of `parser` for the options. Likewise a number of `pickList`. Find another option similar to your new option and use the same values.
 
-Presently no options have the final **`default`** string, and it is left out of the table. The compiler will add a NULL.
+Presently no options have the final `default` string, and it is left out of the table. The compiler will add a NULL.
 
 The final table entry added. Note in here the spacing has been compressed, but in the actual code the current column settings should be maintained if possible -
 
-```
+~~~
   { TidyEscapeScripts, PP, "escape-scripts", BL, yes, ParseBool, boolPicks[, NULL] }, /* 20160227 - Issue #348 */
-```
+~~~
 
 #### 3. Option Description
 
-In `language_en.h`, in the section labelled **Options Documentation**. It can be anywhere, but usually a new option would be added just before the next section labelled **Console Application**.
+In `language_en.h`, in the section labelled **Options Documentation**. Please try to keep this in alphabetical order.
 
 Each entry is a structure with 3 members -
-```
+~~~
 typedef struct languageDictionaryEntry {
     uint key;
     uint pluralForm;
     ctmbstr value;
 } languageDictionaryEntry;
-```
+~~~
 
-The **`key`** is the option **`ID`**; The **`pluralForm`** is not used for options, and should be `0`; The **`value`** is the description string.
+The `key` is the option `ID`; The `pluralForm` is not used for options, and should be `0`; The `value` is the description string.
 
 Some care has to be taken with the description string. The only html allowed here is `<code>...</code>`, `<var>...</var>`, `<em>...</em>`, `<strong>...</strong>`, and `<br/>`. Entities, tags, attributes, etc., should be enclosed in `<code>...</code>`. Option values should be enclosed in `<var>...</var>`. It's very important that `<br/>` be self-closing! This string is processed to build the API documentation.
 
 This is the desription added for this new option.
 
-```
+~~~
     {
       TidyEscapeScripts,          0,
         "This option causes items that look like closing tags, like <code>&lt;/g</code> to be "
         "escaped to <code>&lt;\\/g</code>. Set this option to 'no' if you do not want this."
     },
-```
+~~~
 
 #### 4. Use in Code
 
 This can be added anywhere in the code to change the current code action. While the testing of the option depends on the option type, the most common is `cfgBool( doc, id )`. Here is an example of where this new option is used -
 
-```
+~~~
     /*\ if javascript insert backslash before / 
      *  Issue #348 - Add option, escape-scripts, to skip
     \*/
     if ((TY_(IsJavaScript)(container)) && cfgBool(doc, TidyEscapeScripts))
     {
-```
+~~~
 
 #### Summary
 
-That's about it. Just 4 places. Obviously the best idea it to search for an existing option **`ID`**, and follow where it is all defined and used, and copy that. It is not difficult.
+That's about it. Just 4 places. Obviously the best idea it to search for an existing option `ID`, and follow where it is all defined and used, and copy that. It is not difficult.
 
 ; eof 20160310

README/TAGS.md

@@ -8,7 +8,7 @@ Note, there are a group of configuration options to add **tags** not yet approve
 
 So, adding a new HTML **tag** consists of the following simple steps:
 
- 1. `tidyenum.h` - Give the element an internal name, like `TidyTag_XXXX`, and thus a value. While there were some initial steps to keep this `TidyTagId` enumeration alphabetic, now just add the new `TidyTag_XXXX` just before the last entry `N_TIDY_TAGS`.
+ 1. `tidyenum.h` - Give the element an internal name, like `TidyTag_XXXX`, and thus a value. Please keep this list in alphabetical order.
 
  2. `tags.c` - Add a line to the `tag_defs[]` table. This assigns the unique string value of the element. Then the html versions that support the element, a pointer to the attributes supported by that elelment, and a bit field of the elements characteristics, inline, block, etc.
 

README/VERSION.md

@@ -58,41 +58,41 @@ The **libTidy** version is controlled by the contents of `version.txt` in the ro
 
 This file consists of two lines of dot (.) separated items. The first being the **major**, **minor**, and **patch** version values, and the second string is a date. Example:
 
-```
+~~~
 5.3.15
 2017.01.29
-```
+~~~
 
 When **CMake** is run, this file is read and two macros are added to the compile flags:
 
-```
+~~~
 add_definitions ( -DLIBTIDY_VERSION="${LIBTIDY_VERSION}" )
 add_definitions ( -DRELEASE_DATE="${tidy_YEAR}/${tidy_MONTH}/${tidy_DAY}" )
-```
+~~~
 
 And in `CMakeLists.txt` there is the posibility to define another macro, when and if required:
 
-```
+~~~
 # add_definitions ( -DRC_NUMBER="D231" )
-```
+~~~
 
 These macros are put in `static const char` strings in **libTidys**’s internal- only `src/version.h` file:
 
-```
+~~~
 static const char TY_(release_date)[] = RELEASE_DATE;
 #ifdef RC_NUMBER
 static const char TY_(library_version)[] = LIBTIDY_VERSION "." RC_NUMBER;
 #else
 static const char TY_(library_version)[] = LIBTIDY_VERSION;
 #endif
-```
+~~~
 
 These strings are returned respectively by the **libTidy** API functions:
 
-```
+~~~
 TIDY_EXPORT ctmbstr TIDY_CALL     tidyLibraryVersion(void);
 TIDY_EXPORT ctmbstr TIDY_CALL     tidyReleaseDate(void);
-```
+~~~
 
 ### Git branches