Math and Code

[kervendurdy]

Hi, thanks for this great editor!

Is there any plugin or extension that can be used with django-prose-editor for:

• LaTeX/math support (e.g., KaTeX or MathJax)?
• Code syntax highlighting (e.g., with Prism.js or highlight.js)?

If not, are there any recommended ways to integrate these?

Thanks!

[matthiask]

Hi

Tiptap 3.0 will ship a mathematics extension: https://next.tiptap.dev/docs/editor/extensions/nodes/mathematics

It also ships a code block with syntax highlighting: https://next.tiptap.dev/docs/editor/extensions/nodes/code-block-lowlight

The problem is that these extensions cannot be used 1:1 with django-prose-editor without me bundling them. I haven't done that yet because of the code size increase that these packages bring when used without code splitting. I'm very interested in supporting these use cases though. So, it would be nice if someone beat me to integrating those!

[vovansrnd]

Как добавить поддержку вставки кода в django-prose-editor. Да она там и так есть.

extensions={
        "Bold": True,
        "Italic": True,
        "BulletList": True,
        "Link": True,
        "Code": True,
        "CodeBlock": True,
    },

Теперь код вставляется по Ctrl + Alt + C или по тройному апострофу (тройной тильде) и пробел или Enter. Если в одну строку, то выделяется одним апострофом с двух сторон.

В итоге мы получаем:

<pre><code class="language-python">
...
</code></pre>

А во фронтэнде добавить в шаблон до закрывающего тега </body> что-то вроде:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/highlight.min.js"></script>

Но есть один большой недостаток - если я делаю Edit HTML, то всё форматирование в <pre>....</pre> слетает. Точнее, удаляются все проблемы слева. Это недопустимое поведение! Блоки <pre> трогать нельзя!!!

Итак, мы получили возможность ввода кода и подстветку синтаксиса во фронтенде. Ок!

Добавить подсветку в редактор статей (django-prose-editor) несложно.

в base.py или settings.py вашего проекта добавляем:

DJANGO_PROSE_EDITOR_EXTENSIONS = [
    {
        "js": ["/static/js/prose-codeblock-lowlight.js"],
        "extensions": {
            "CodeBlockLowlight": "django_prose_editor.config.process_code_block",
        },
    },
]

в models.py у меня сделано так:

ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES = {
    "Bold": True,
    "Italic": True,
    "Heading": {"levels": [2, 3]},
    "Image": True,
    ...
    "Code": True,
    "CodeBlock": False,  # Отключаем стандартный
    "CodeBlockLowlight": { # Включаем наш
        "languageClassPrefix": "language-",
    },
}

....

class BlogPage(ThemedTemplateMixin, Page):
    # body = RichTextField(blank=True)
    body = ProseEditorField(
        extensions=ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES,
        blank=True,
        sanitize=False,
    )
    
    ...
    
    content_panels = [
        ...
        FieldPanel(
            "body",
            widget=AdminProseEditorWidget(
                preset="configurable",
                config={"extensions": ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES},
            ),
        ),

В wagtail_hooks.py примерно такой код или как вам угодно добавить css подсветки синтаксиса:

@hooks.register("insert_global_admin_css")
def editor_highlight_css():
    return format_html(
        '<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">'
    )

И теперь самое главное, скрипт prose-codeblock-lowlight.js:

// /static/js/prose-codeblock-lowlight.js

// 1. Импортируем Tiptap расширение и lowlight
import { createLowlight } from 'https://esm.sh/lowlight@3?bundle';
// Важно: переименовываем импорт, чтобы он не конфликтовал с нашим экспортом
import TiptapCodeBlockLowlight from 'https://esm.sh/@tiptap/extension-code-block-lowlight@2?bundle';

// 2. Импортируем языки
import bash from 'https://esm.sh/highlight.js@11/lib/languages/bash?bundle';
import python from 'https://esm.sh/highlight.js@11/lib/languages/python?bundle';
import css from 'https://esm.sh/highlight.js@11/lib/languages/css?bundle';
import javascript from 'https://esm.sh/highlight.js@11/lib/languages/javascript?bundle';
import html from 'https://esm.sh/highlight.js@11/lib/languages/xml?bundle';
import c from 'https://esm.sh/highlight.js@11/lib/languages/c?bundle';
import json from 'https://esm.sh/highlight.js@11/lib/languages/json?bundle';

// 3. Создаем инстанс lowlight
const grammars = { bash, python, css, javascript, html, c, json };
const lowlight = createLowlight(grammars);

// 4. ГЛАВНОЕ: Экспортируем расширение Tiptap под нужным именем.
//    Именно эту константу `CodeBlockLowlight` ищет `configurable.js`.
//    Мы сразу конфигурируем его с нашим `lowlight` инстансом.
export const CodeBlockLowlight = TiptapCodeBlockLowlight.configure({
  lowlight,
});

Теперь подсветка появляется и в админке сразу при редактировании (по-моему, это лишнее, но если оно будет сразу из коробки, то почему и нет)

Но на данный момент подсветка получится очень урезанной, потому что в overrides.css строка color: #fff !important; перебивает ваши стили подсветки. Это не всё, но это главное.

Всё, теперь подсветка есть везде, код леко вставляется, выглядит красиво, но есть несколько коментов, которые хотелось бы устранить в django-prose-editor. Фундаментальная проблема, можно её считать багом, это удаление пробелов слева в блоке <pre> при Edit HTML. Если вам понадобится подправить код статьи, всё ваше форматирование кода слетит напроч.

Проверялось на версиях 0.16-0.17a2

[vovansrnd]

How to add code-insertion support in django-prose-editor.
Well, it’s already there out of the box.

extensions={
        "Bold": True,
        "Italic": True,
        "BulletList": True,
        "Link": True,
        "Code": True,
        "CodeBlock": True,
    },

Now code can be inserted with Ctrl + Alt + C or by typing triple back-ticks (triple tildes) followed by a space or Enter.
For a single-line snippet, wrap it with single back-ticks on both sides.

As a result we get:

<pre><code class="language-python">
...
</code></pre>

On the frontend, add something like this to your template right before the closing </body> tag:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/highlight.min.js"></script>

But there’s one big drawback — when I click Edit HTML, all whitespace inside <pre>....</pre> is stripped (all leading spaces are removed).
That’s unacceptable! <pre> blocks must remain untouched!!!

So, we now have code input and syntax highlighting on the frontend. OK!

Adding highlighting inside the article editor (django-prose-editor) is straightforward.

In your project’s base.py or settings.py add:

DJANGO_PROSE_EDITOR_EXTENSIONS = [
    {
        "js": ["/static/js/prose-codeblock-lowlight.js"],
        "extensions": {
            "CodeBlockLowlight": "django_prose_editor.config.process_code_block",
        },
    },
]

In models.py I have:

ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES = {
    "Bold": True,
    "Italic": True,
    "Heading": {"levels": [2, 3]},
    "Image": True,
    ...
    "Code": True,
    "CodeBlock": False,  # Disable the default one
    "CodeBlockLowlight": {  # Enable ours
        "languageClassPrefix": "language-",
    },
}

....

class BlogPage(ThemedTemplateMixin, Page):
    # body = RichTextField(blank=True)
    body = ProseEditorField(
        extensions=ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES,
        blank=True,
        sanitize=False,
    )
    
    ...
    
    content_panels = [
        ...
        FieldPanel(
            "body",
            widget=AdminProseEditorWidget(
                preset="configurable",
                config={"extensions": ADMIN_EXTENSIONS_WITH_WAGTAIL_IMAGES},
            ),
        ),

In wagtail_hooks.py, insert the CSS for syntax highlighting (or do it however you prefer):

@hooks.register("insert_global_admin_css")
def editor_highlight_css():
    return format_html(
        '<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">'
    )

And finally, the script prose-codeblock-lowlight.js:

// /static/js/prose-codeblock-lowlight.js

// 1. Import Tiptap extension and lowlight
import { createLowlight } from 'https://esm.sh/lowlight@3?bundle';
// Important: rename the import so it doesn’t clash with our export
import TiptapCodeBlockLowlight from 'https://esm.sh/@tiptap/extension-code-block-lowlight@2?bundle';

// 2. Import languages
import bash from 'https://esm.sh/highlight.js@11/lib/languages/bash?bundle';
import python from 'https://esm.sh/highlight.js@11/lib/languages/python?bundle';
import css from 'https://esm.sh/highlight.js@11/lib/languages/css?bundle';
import javascript from 'https://esm.sh/highlight.js@11/lib/languages/javascript?bundle';
import html from 'https://esm.sh/highlight.js@11/lib/languages/xml?bundle';
import c from 'https://esm.sh/highlight.js@11/lib/languages/c?bundle';
import json from 'https://esm.sh/highlight.js@11/lib/languages/json?bundle';

// 3. Create a lowlight instance
const grammars = { bash, python, css, javascript, html, c, json };
const lowlight = createLowlight(grammars);

// 4. MAIN PART: export the Tiptap extension under the required name.
//    `configurable.js` specifically looks for the constant `CodeBlockLowlight`.
//    We configure it with our `lowlight` instance right away.
export const CodeBlockLowlight = TiptapCodeBlockLowlight.configure({
  lowlight,
});

Now highlighting shows up in the admin interface while editing (IMO a bit over-the-top, but if it works out of the box, why not).

However, at the moment the highlighting is very limited because the line color: #fff !important; in overrides.css overrides your highlight styles.
That’s not the only issue, but it’s the main one.

That’s it: highlighting everywhere, code is easy to paste, looks nice, but there are still a few things I’d like to see fixed in django-prose-editor.
The fundamental issue—call it a bug—is that left-hand spaces inside <pre> blocks are removed when you hit Edit HTML. If you need to tweak the article’s code, all formatting is lost.

Tested on versions 0.16–0.17a2.

[matthiask]

However, at the moment the highlighting is very limited because the line color: #fff !important; in overrides.css overrides your highlight styles.

Yes, that sounds ... not good. I wonder if there's a better way to override Django's styles? The Django admin CSS is a bit ugly with all the element styles and so on. I am sure we're overusing !important though.

The fundamental issue—call it a bug—is that left-hand spaces inside <pre> blocks are removed when you hit Edit HTML. If you need to tweak the article’s code, all formatting is lost.

I didn't really want to introduce beautification of HTML in the HTML extension. But now it's there. We should definitely be skipping <pre> tags in there though.

[vovansrnd]

Anything can live inside a pre—even poetry, like this:

Я волком бы
       выгрыз
          бюрократизм.
К мандатам
      почтения нету.
К любым
     чертям с матерями
               катись
любая бумажка.
        Но эту…
По длинному фронту
           купе
              и кают
чиновник
     учтивый движется.
Сдают паспорта,
        и я
          сдаю
мою
  пурпурную книжицу.

What really matters is keeping the original formatting intact.
Syntax highlighting in the admin panel is, in my opinion, secondary and optional.
But preserving the formatting of pre blocks is absolutely critical.

[matthiask]

Thanks for the poem :)

I changed the HTML extension to keep whitespace intact inside <pre>. Also, prettification now only runs when requested explicitly.

[vovansrnd]

The first thing that came to mind regarding complex prose formatting was Mayakovsky. And it wasn't easy to find a good example. The formatting is mangled everywhere online. This was the first one I came across; there wasn't much to choose from. But in reality, you occasionally need to preserve some tricky formatting, and these edge cases aren't obvious—you don't notice them when everything is working fine. But when it doesn't work, it's a constant stumbling block. Thank you so much for the quick fix!

I'm currently migrating my blog from a PHP-based CMS to Wagtail 7. I also added image uploading to your editor—it's a feature that's needed all the time. I'll try to package this as a Wagtail extension and publish it soon. I have no experience with this, so it will be my first extension.

I also made it so that images are saved to media/YEAR/SLUG to make them easier to navigate. Additionally, I implemented filesystem directory navigation in the admin panel. For some reason, Wagtail decided to just dump all uploaded images into one big mess. As a fallback, the images are also added to corresponding collections (YEAR->SLUG) just in case.

I've also done a huge amount of work on the comments. I made them threaded and completely rewrote the admin interface for managing them. The original implementation was unintuitive and inconvenient.

The blog is still in a rough state, but the first thing I want to do is create extensions for django-prose-editor and Wagtail, and then write an article about why I chose Wagtail and the modifications I've made, complete with screenshots and descriptions. To do that, I needed to figure out how to publish code blocks. And this behavior with the pre tag was really frustrating because it automatically broke the ability to edit the raw HTML of articles if they contained any formatted code. It was my first disappointment with Wagtail—the fact that you can't edit the article's source code in the default editor. A lot of manual edits are required after migrating from the old blog. And then, after spending a ton of time, I found I had hit the same wall with django-prose-editor. But you solved this issue in just a few minutes. A huge thank you for that, and for creating such a great and simple text editor for Django

[matthiask]

Thank you very much! I'm not using Wagtail (yet?), but I'm wishing you all the best with your project. I'm glad that this project is useful to you.