Wanted: 'insert=fullpage' for tables

[lemzwerg]

lyluatex always uses \insertpdf to handle LilyPond snippets with the insert=fullpage option. This doesn't work with tables. Here is my work-around solution that uses \includegraphics – I would like to have this directly supported, for example, with an option insert=full.

\newcommand{\lilypondmyfile}[2][]{%
  \begingroup
    % Compensate top and bottom margin values inserted by lyluatex.
    % Due to rounding, however, we get slightly inaccurate values; we
    % add a small amount to avoid lyluatex emitting values near zero in
    % real number format like `-1.3671879628419e-08\pt' in the
    % temporary `.ly file (this is a bug in lyluatex).
    \newdimen\myextratopmargin
    \newdimen\myextrabottommargin
    \myextratopmargin = \dimexpr 0.1pt - (1in
                                          + \voffset
                                          + \topmargin
                                          + \headheight
                                          + \headsep) \relax
    \myextrabottommargin = \dimexpr -0.1pt - (\paperheight
                                              + \myextratopmargin
                                              - \textheight) \relax
    \includely[raw-pdf,
               extra-top-margin=\myextratopmargin,
               extra-bottom-margin=\myextrabottommargin,
               #1]{#2}%
    \includegraphics{\lyscore{}}%
  \endgroup}

A typical call could be

\begin{table}
  \centering

  \lilypondmyfile[line-width=0.9\linewidth,
                  system-count=5,
                  insert=fullpage,
                  twoside=false,
                  paperwidth=\textwidth,
                  paperheight=0.9\textheight]{foo.ly}

  \caption{bla bla}
  \label{tab:foo}
\end{table}

where the file foo.ly would also contain something like

\paper {
  system-system-spacing.basic-distance = #18
}

to get decent spacing.

[uliska]

I'm not sure I fully understand the issue.

1. Why exactly do you want/need to put a score in a table environment?
2. What do you need the insert=fullpage for?

If you are using the table just as a floating environment and for the caption handling it might be worth turning your attention to lyluatexmp, which is very far from a releasable state right now, but aims at building on top of lyluatex to manage the handling of examples and (floating) environments etc. It was a rather clear decision by @jperon to keep lyluatex as low-level as possible, he didn't want the stuff I needed for my work to clutter the lyluatex concept.

If you want I can give you access to my private repository where I make use of lyluatexmp so can have a better idea what to look for. Right now the package basically does what I currently need, and I could need any encouragment, feedback, support or even contribution to get it to a further point so I can hopefully release it some day.

[lemzwerg]

Ad (1): as a floating environment, with captions and the like.
Ad (2): I want to have lilypond's vertical spacing for multiple staves, since line-by-line handling gives poor results for my use case.

I will have a look at your lyluatexmp stuff, thanks.

What I want is lyluatex to be even more low-level! Right now it adjusts the margins and page size for the fullpage option. I would like to have a new pass-through mode that makes lyluatex just call lilypond, without any behind-the-scene adjustments at all (besides the absolutely necessary setup to pass default values). It is OK with me if lyluatex provided this mode for raw-pdf only.

Here is an image showing lyluatex's systems formatting; the vertical space between systems was increased with

\newcommand{\betweenLilyPondSystem}[1]{\\[6ex]}

Note the ugly big gap between the second and third system. I know that I could improve the call to \betweenLilyPondSystem, but this would be a hack IMHO.

And here the same again with lilypond's native formatting, using the above described setup to undo lyluatex's margin settings.

[lemzwerg]

I now had a look at lyluatexmp: This package is at a far higher level than what I suggest above. In general I believe what you are going to implement is partially a work-around for lyluatex's use of the very inflexible \includepdf command. In addition, AFAICS, it doesn't cover the case where you want lilypond's native vertical formatting for a non-full-page float.

I also tried to experiment with koma-script's \captionof functionality; however, contrary to you, I failed since I lack the necessary lua knowledge...

[uliska]

I now had a look at lyluatexmp: This package is at a far higher level than what I suggest above.

Yes, I had already understood this after your previous comment.

In general I believe what you are going to implement is partially a work-around for lyluatex's use of the very inflexible \includepdf command.

That's not correct, but I assume the actual goal of the lyluatexmp package is not quite visible yet.
What it will provide is:

• a dedicated environment with its own counter and "list of music examples"
• commands to easily create LilyPond-based music examples using that environment
• additional commands for sideways-, rotated etc. examples
• full/multipage examples with caption (printed in the page header/footer)
• Proper support for indicating missing or failed examples instead of causing LaTeX errors (see images below)
• Maybe some more I don't recall right now

AFAICS, it doesn't cover the case where you want lilypond's native vertical formatting for a non-full-page float.

Correct, my package doesn't even bother with that question, which gets us back to your original request.

Similar to #263 I'd say this is not a genuine lyluatex issue but a case where lyluatex exposes LilyPond behaviour: What you are describing is the result of lilypond-book-preamble.ly slicing and cropping systems. However, unlike #263 I would accept the issue as something we should do in lyluatex:

• Any "fix" in LilyPond would probably not be available for existing LilyPond versions, and it would surely not make it into 2.20 so it would only become available in a "stable" release with 2.22 - whenever that will be ...
• There may be other use cases for non-full-size fullpage examples that warrant a lyluatex solution: e.g. "thumbnail"-style smaller scores

Of course (just to be clear), the current behaviour of adjusting margins and including the results with \includepdf in fullpage insertion mode is absolutely correct and what lyluatex should do, it just doesn't cover valid additional use cases.

My suggestion:

• a new insert= option, with a name to be discussed (e.g. raw, thumbnail (frame option #270), page, smallpage)
• do not adjust the page margins
• create a paper size:
  • defaults could be: current \textwidth and the aspect ratio of the document's paper size (as I don't think LaTeX provides a way to get a reasonable value for a "currently available height"
  • of course the paper size can be given manually
• pass the other options (e.g. fonts, indent, staffsize ...) as usual, with a way to override them.
• for staffsize there should be an option similar to inline-staffsize. Maybe the default value could set the staffsize relative to the main staffsize proportional to the example's paper size.

Nevertheless I think we should investigate this on the LilyPond level as well, as I think it's essentially an issue with lilypond-book-preamble. There should be a way of compiling that

• does the layout just as normal
  • (maybe lilypond-book-preamble does not perform the page-breaking algorithms as at least the result doesn't have a notion of pages anymore)
  • do the vertical distribution
• slices systems
• does not crop them (vertically!)

That way one could insert the systems one after another and retain the original layout.
Ideally there should be two files per system, one cropped and one uncropped. When inserting such systems in a LaTeX document at the top or the bottom of the page I would want to have cropped systems.

---

lyluatexmp examples:

Automatic reporting of problem with score. Here it's a missing LilyPond file, it could also be a failed compilation (of course it will be possible to change the behaviour to cause an error instead):

Here's the list of music examples, with the entry printed in red to indicate the problem:

[lemzwerg]

Of course (just to be clear), the current behaviour of
adjusting margins and including the results with
\includepdf in fullpage insertion mode is absolutely
correct and what lyluatex should do, it just doesn't
cover valid additional use cases.

Agreed.

My suggestion:

• a new insert= option, with a name to be discussed
  (e.g. raw, thumbnail (frame option #270), page, smallpage)
• do not adjust the page margins
• create a paper size:
  • defaults could be: current \textwidth and the aspect
    ratio of the document's paper size (as I don't think
    LaTeX provides a way to get a reasonable value
    for a "currently available height"

It does, see \pagetotal.

• of course the paper size can be given manually

• pass the other options (e.g. fonts, indent, staffsize ...)
  as usual, with a way to override them.
• for staffsize there should be an option similar to
  inline-staffsize. Maybe the default value could set
  the staffsize relative to the main staffsize proportional
  to the example's paper size.

I agree with all of your suggestions.

lyluatexmp examples:

Automatic reporting of problem with score. [...]

This would be a very welcome addition, which should be part of lyluatex proper, IMHO.

[uliska]

OK, this is an example that shows why some scores may not reasonably be inserted with insert=systems at all:

\version "2.19.82"

\paper {
  indent = 0
}

\score {
  \new PianoStaff <<
    \new Staff {
      c''1
      \break
      r4
      \voiceOne
      c''''''4
      r2
    }
    \new Staff {
      \voiceOne
      c'2 c,
      r4
      \crossStaff c'4
      r2
    }
  >>
}

\layout {
  \context {
    \PianoStaff
    \consists Span_stem_engraver
  }
}

LilyPond will push the staves apart as necessary but will at the same time respect the skylines. Even if we could convince LilyPond to somehow tell us about all the relevant distances we would have to overlay the resulting images, and it seems rather complicated to do so.

When compiled with lilypond-book-preamble.ly this is what LilyPond engraves:

However, I think we should keep this idea in mind and try to find a solution because @lemzwerg 's wish has the considerable drawback that it can't flow over page borders.

Well, actually there's only one number we would have to get from LilyPond: how much the systems have been additionally pushed apart for the slicing (although I have now idea whether this is possible ...)

[uliska]

The other way round it's

\paper {
  indent = 0
}

{
  c'1 \break
  c'1

Regular compilation looks like this:

and with lilypond-book-preamble the inter-system space is cropped:

(which doesn't look like a big deal in this example but can be really ugly in real-world scores, especially when multiple systems are very different or with ragged-bottom = ##f)

[uliska]

lyluatexmp examples: Automatic reporting of problem with score. [...]

This would be a very welcome addition, which should be part of lyluatex proper, IMHO.

@lemzwerg try lyluatex's showfailed option and tell me if that's sufficient or not.

[uliska]

Discussion in https://lists.gnu.org/archive/html/lilypond-devel/2019-07/msg00166.html.
David K's last response https://lists.gnu.org/archive/html/lilypond-devel/2019-07/msg00169.html makes clear that we can't expect LilyPond to provide the necessary information in the foreseeable future, so we have to bury that idea and think of a way to implement @lemzwerg 's request (along the lines of my suggestion)

[lemzwerg]

It's a bit unfortunate that comments in this issue mix up two completely different requests. I suggest that you open another issue (probably in the LilyPond tracker) that discusses how to improve lilypond-book-preamble.ly (and LilyPond itself); it probably makes sense to open another lyluatex issue, too, to discuss how lyluatex would benefit.

This issue should be restricted to any progress w.r.t. a new lyluatex option to get full-page LilyPond output as described in the OP.

[uliska]

Distributed over #272 and #273