Tidy should not output unguarded CDATA sections #660

dechamps · 2017-12-16T14:38:34Z

On current HEAD (f0438bd):

$ tidy <<EOF
<?xml version="1.0" ?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title></title><meta charset="utf-8" /></head><body>
<script>foo</script>
</body></html>
EOF

Returns:

Info: Document content looks like XHTML5
No warnings or errors were found.

<?xml version="1.0"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta name="generator" content=
"HTML Tidy for HTML5 for Linux version 5.7.0" />
<title></title>
<meta charset="utf-8" />
</head>
<body>
<script>
<![CDATA[
foo
]]>
</script>
</body>
</html>

Tidy decided to wrap the contents of the <script> element inside a CDATA section. This is fine from a pure XML perspective, but in practice, browsers tend to not like that at all since they will typically parse the document as HTML, not XHTML (that's true for at least modern Firefox). They will not recognize the <![CDATA[ sequence and will try to parse it as JavaScript, leading to parse errors and a broken script.

I think the most straightforward way to solve this problem is to forget about CDATA sections and simply escape the <script> contents as entity-escaped XML (related: #659).

Alternatively, the standard trick is to "guard" the CDATA opening and closing tags by commenting them out in JavaScript, like so:

<script>
// <![CDATA[
foo
// ]]>
</script>

For now, the only workaround is to output HTML (-ashtml), which doesn't have this problem.

The text was updated successfully, but these errors were encountered:

utrenkner · 2018-05-14T11:01:31Z

I would like to confirm: This is a problem also for us. Last week, we switched an old website to the new(er) way of including Google Analytics, which is via Google's tag manager. This is part of the code to include in each web page:

<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-12345678-1');
</script>

And all of the sudden, Google Analytics worked only intermittently. I then noticed that the code between <script>.. </script> had been escaped with a CDATA section. And both, Chrome and Firefox, reported this is a JavaScript error. It took me quite some time to figure out which tool included this CDATA code: tidy-html5.

I followed @dechamps suggestion to add -ashtml which effectively solved the problem for us. However, it would be great if tidy could get an option to not add this CDATA code in the first place.

jcubic · 2018-10-28T19:59:38Z

@utrenkner With JavaScript it's no problem you just have to use type="text/javascript".

EDIT:

The problem I'm having is JSON-LD

  <script type="application/ld+json">
  {
  "@context": "http://schema.org",
  "@type": "BlogPosting",
  "headline": "15 Pytań na rozmowę rekrutacyjną z CSS",
  "image": {
    "@type": "ImageObject",
    "url": "/img/css-interview.jpg",
    "width": 800,
    "height": 464
  }
  }
  </script>

and I want to have xhtml self closing tags on output html, so -ashtml option don't work.

markafoltz · 2019-04-05T22:51:08Z

tidy-html5 also breaks inline script used to customize Respec (W3C spec tooling). -ashtml still inserts the tags on version 'next'.

Ref: https://github.com/w3c/presentation-api/blob/gh-pages/Makefile

geoffmcl · 2019-04-07T19:31:26Z

@dechamps sorry for the rather belated response... somehow I overlooked this... perhaps related to #659...

This certainly seems an issue related to DOCTYPE... diff HTML5 and previous versions, and HTML versus XHTML... AND diff PCDATA, Parsed Character Data and CDATA, Unparsed Character Data...

Tidy sees the input sample as XML, as XHTML5 is its declaration...

The -ashtml option requests HTML output, so the CDATA guard will not be added... this seems correct...

Then this request to stop tidy inserting CDATA in XHTML seems a problem! Why do that? Spec/Ref links needed...

And maybe even more of a problem, further inserting // javascript comments, as well... again Spec/Ref links needed...

Saw a suggestion somewhere where this too could be in a comment, like , to maintain XHTML... wow... do not think so...

In HTML5, you can put just about anything you like in <script>, like -

<script>
var v = 5;
if ( v < 10 && ((v & 0x1) == 1) ) {
        // do something
}
</script>

But NOT if you want XHTML conformance... without escaping, at least the < and & chars...

Unless I am mis-reading this request, or have missed some W3C REF, the best solution, as @mfoltzgoogle points to, is to accept HTML output...

Again sorry for the delay... @mfoltzgoogle comment jogged the memory...

Look forward to further feedback, comments, patches, PRs, etc to add to this... thanks...

markafoltz · 2019-04-11T17:50:04Z

Unless I'm missing something, the document I'm tidying [1] is not XHTML:

<!DOCTYPE html>

So I don't expect tidy to process it as XML and insert CDATA markup for inline script.

[1] https://w3c.github.io/presentation-api/

Magiweb · 2019-04-17T15:12:24Z

Confirm I'm getting the same thing happening. It actually breaks Facebook's standard tracking pixel script. Commenting the added CDATA tags just causes a JSON parse error from FB, so that doesn't work, and as @geoffmcl pointed out, is not a good solution.

Agree with @mfoltzgoogle - my doctype is set to html, so surely it shouldn't be processing it as XHTML?

geoffmcl · 2019-04-17T18:39:59Z

@mfoltzgoogle well I too seem missing something... ;=))

When I wget the link you gave, and process it with tidy, it correctly treats it as HTML5 ...

Some warning that do not appear exactly right... link-for... but that can be a separate issue...

And does not add any <![CDATA[ ... ]]> to the script, or style, tags... so can not repeat the problem, with this sample... need a small, consistent, sample that shows this...

However, Tidy will add the <![CDATA[ ... ]]> blocks, if requested with an option - -asxml, -asxhtml, output-xhtml: yes ie convert HTML to well formed XHTML - then you get the XHTML you asked for... what can be wrong with this? What am I missing???

Just reading the parser.c code, the criteria for setting output to XHTML, is also finding like <html xmlns="http://www.w3.org/1999/xhtml">... as in @dechamps first sample... and not just the DOCTYPE... which it seems can just about be anything...

@Magiweb, agree that just the DOCTYPE should not cause a document to be output as xhtml... can you supply a small sample where it seems to?

So still in the dark as to exactly what is the problem, the request, here... need more info, explanation... samples, with config used, etc, etc...

As stated, look forward to further feedback, comments, patches, PRs, etc to add to this... thanks...

petko · 2020-05-04T10:30:53Z

It can be a problem is situations where for example you need both the XML parsing of an XHTML file (so we use the output-xhtml: yes option), but you also need the same file to be read properly by some older off-screen readers, which do not understand the CDATA sections properly.

So maybe something like an option guard-cdata can be useful here?

ghost · 2022-09-21T21:44:48Z

Additionally, there are cases where, as a developer, you want to process an HTML document using XSLT, which requires a well-formed XML document. However, you don't want to include the <![CDATA section because it breaks browsers. This results in a situation that is seemingly impossible to resolve: you can't output HTML because the HTML is not well-formed (e.g., the <br> and <meta> tags are not self-closing) and you can't output as XML because of the <![CDATA section.

It would be nice to have a bit more control over whether <![CDATA is written when using XML (or XHTML) as the output format.

Aside, in our situation, we have full control over the <style> section of the HTML page, so we know the <![CDATA instruction is superfluous. Meaning, although using // <![CDATA would solve the problem, we'd prefer to suppress it altogether.

linusjf · 2024-04-09T07:46:30Z

This is additionally an issue when using type="importmap". That really screws up my React components.

geoffmcl mentioned this issue Dec 26, 2017

escape-cdata doesn't work on <script> tags #659

Open

geoffmcl added the Technical Discussion label Apr 7, 2019

balthisar added this to the 5.9 milestone Jul 9, 2021

egekorkan mentioned this issue May 25, 2023

Publication Prep w3c/wot-binding-templates#290

Merged

jgm mentioned this issue May 14, 2024

Syntax Highlighting not working in HTML (CSS not being picked up) jgm/pandoc#9752

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tidy should not output unguarded CDATA sections #660

Tidy should not output unguarded CDATA sections #660

dechamps commented Dec 16, 2017

utrenkner commented May 14, 2018

jcubic commented Oct 28, 2018 •

edited

Loading

markafoltz commented Apr 5, 2019

geoffmcl commented Apr 7, 2019

markafoltz commented Apr 11, 2019

Magiweb commented Apr 17, 2019

geoffmcl commented Apr 17, 2019

petko commented May 4, 2020 •

edited

Loading

ghost commented Sep 21, 2022 •

edited by ghost

Loading

linusjf commented Apr 9, 2024

Tidy should not output unguarded CDATA sections #660

Tidy should not output unguarded CDATA sections #660

Comments

dechamps commented Dec 16, 2017

utrenkner commented May 14, 2018

jcubic commented Oct 28, 2018 • edited Loading

markafoltz commented Apr 5, 2019

geoffmcl commented Apr 7, 2019

markafoltz commented Apr 11, 2019

Magiweb commented Apr 17, 2019

geoffmcl commented Apr 17, 2019

petko commented May 4, 2020 • edited Loading

ghost commented Sep 21, 2022 • edited by ghost Loading

linusjf commented Apr 9, 2024

jcubic commented Oct 28, 2018 •

edited

Loading

petko commented May 4, 2020 •

edited

Loading

ghost commented Sep 21, 2022 •

edited by ghost

Loading