A newbie’s experiences with Lilypond, Lilypond-book, LATEX and Perl

Joe McCool
Southern Regional College, Ireland
mccoolj (at) src dot ac dot uk
http://benburb.demon.co.uk/apache2-default/joe.html

Abstract
The author is an active Irish traditional musician. He is also a keen inland boater.
He is having a lot of fun composing a book on “Traditional Music for Boaters”.
In this paper he describes his successes and frustrations using Lilypond,
Lilypond-book, LATEX and ABC musical notation. Lilypond and LATEX have a lot
in common. Neither are WYSIWYG, neither demand GUI’s. Both compile simple
flat files to produce beautiful graphical output.
Lilypond’s original manifestations produced output directly for LATEX, but of
late users writing books have been encouraged to use Lilypond-book. This looks
for Lilypond code within LATEX source files and produces graphics and associated
instructions which can then be processed by LATEX.
Most joy has come from automating these processes via GNU/Linux and Perl.

1 What’s out there C:Franz Schubert

O:Austria
By definition classical music has an inherent connec-
M:C|
tion with books and text. Traditional music does not.
L:1/8
Classical music is written down in scores. Histori- Q:1/4=160
cally, traditional music is not. It is largely an aural K:C
medium and tunes are learnt by ear. Having said G2|"C"c2c2e2e2|"C"c4G2G2|"G"G3G dcBA|"G"
that, musical scores do now have an important role G4 z2G2|"C"c2c2 e2e2|"C"c4G2c2|"G"B2AB
to play. More and more young traditonal musicians ....
are learning how to read scores. Printed material The K: field represents the tune’s key. All lines
allows us to store pieces that would otherwise be lost below this contain the music. Lines above are header
and pencil and paper is a useful aid to composition. fields, with X: representing an index for this partic-
Musicians like to share pieces and the arrival ular tune in a file of other ABC’s. Notes in quotes
of the personal computer has made this easier than represent accompaniment chords.
ever. But, where years ago we popped manuscripts in At first sight ABC seems ideal for what I wish
the post, we now need to share scores electronically. to do. It has a simple input format. It can output
This has brought about a plethora of programs and PostScript files that I can incorporate into LATEX
systems that enable us to do so. documents. ABC programs are open source and,
1.1 ABC notation
most importantly, there are huge collections of ABC
source files available on the internet.
In the 1980’s Chris Walshaw, then at the Univer- But, I have a few issues with ABC:
sity of Cambridge, began writing out fragments of
• there is a gross lack of standardisation. What
folk/traditional tunes using letters to represent the
standardisation exists is often ignored by the
notes. This became gradually formalised into what
authors of ABC files.
is known as the ABC “standard”.1 Numerous small
programs have appeared to convert ASCII files of • I need a system comparable to LATEX in terms
ABC to printed scores.2 There are also programs to of typesetting quality. ABC does not have the
convert ABC code to midi. fine grained control of LATEX.
Here’s an example of an ABC file: • It is difficult to avoid clashing problems: note
X: 1 heads clashing with bar numbers, or grace notes
T:The trout clashing with accidentals.
1 www.walshaw.plus.com/abc
• At the time I started my project, the ABC mail-
2 for example, moinejf.free.fr ing list seemed to vanish!

376 TUGboat, Volume 29 (2008), No. 3 — Proceedings of the 2008 Annual Meeting
 A newbie’s experiences with Lilypond, Lilypond-book, LATEX and Perl

1.2 Commercially available software 1.5 Example Lilypond file

Under Microsoft Windows, several commercial pro- \version "2.11.33"
grams are available for typesetting music. Finale,3 \header {
Sibelius,4 and Cakewalk5 are well known. Notewor- composer = "Joe Mc Cool"
thy Composer is available as shareware.6 title = "The Eight Lock"
dedication = ""
1.3 MusicTEX and MusiXTEX }
voicedefault = {
Both of these, based on what I could find out about \relative c’
them,7 appeared too complex for me. They did \clef treble
not seem to have an active development or support \key g \major
community. \time 4/4
\repeat volta 2 {
1.4 Lilypond \time 4/4
I rejected the commercial products and Notewor- \clef treble
thy because they use proprietary file formats and d’4 d8 e d c b4
they rely on GUI interfaces. I am a traditional- ....
}
ist in more ways than one. Long ago I realised the
\repeat volta 2 {
power, elegance and beauty of plain ASCII files under a’4 a8 a b c4.
GNU/Linux. Hence my final choice of Lilypond.89 ....
• Lilypond’s originators have objectives very simi- }
lar to those of LATEX: “to print music in the best }
traditions of classical engraving with minimum \include "../new.score.ly"
fuss”. Notice the indentation of code, similar to pro-
• Lilypond uses plain ASCII, not dissimilar to ABC. gramming languages. voicedefault is a musical object
• Lilypond enjoys ongoing development. that will subsequently process as a score (see below),
• Its documentation is excellent. or a midi file.10
• Very active user support via mailing lists. As my project gathered weight I got tired of
• Very fast keying of source files: having to edit individual Lilypond files in order to
– Note durations need stating once only. In change the overall look of my book, hence my use of
the input a4 b, the notes a and b have the the include statement. new.score.ly consists of:
same duration. \score{
– Notes can be raised an octave using a fol- <<
lowing ’ or lowered by a following ,. Lily- {
\voicedefault
pond also provides a relative mode in
}
which it will position notes on the scale in >>
a common sense, reasonable fashion. \layout{
\relative c’’ { #(layout-set-staff-size 20)
b c d c b c bes a }
} }
These notes will all be placed within the In order to change the overall look of all my
scale, rather than climbing higher upward. pieces, for example to change the staff size, all I have
• key transposition is easy: \transpose d e ... to do is edit the above.
• The excitement and delight I found putting to-
gether my first Lilypond scripts was matched 2 My approach
only by that of my first LATEX scripts. Lilypond My approach is constrained by the following goals:
and LATEX really are first cousins! • Each page should contain an integer number of
3 www.finalemusic.com tunes. Classical musicians are happy (or at least
4 www.sibelius.com
5 www.cakewalk.com
willing) to turn a page in the middle of a piece,
6 www.noteworthysoftware.com traditional musicians are not.
7 www.tex.ac.uk/cgi-bin/texfaq2html?label=music • Traditional music is often played in sets. Two
8 www.lilypond.org
or three jigs will be played one after the other,
9 In rural Ireland we have a saying: “If you think a donkey

will do the job, use a horse!” 10 though midi support is regrettably not good in Lilypond.

TUGboat, Volume 29 (2008), No. 3 — Proceedings of the 2008 Annual Meeting 377
Joe McCool

Some text before a musical snippet.

or a group of hornpipes. When new tunes are
added to the collection, they must be kept as à j ¹ ¹ ¹ ¹
close together as possible to other members of
Another snippet:
their set, ideally on the same page, or on the
same spread. à j ¹ ¹ ¹ ¹
• Brief texts and footnotes must appear on the Some more text.
page of the tune to which they refer. Figure 1: small.pdf
• Index entries must have the form ‘name : type
: page number ’. This reveals the page number Here the file small.tex constitutes the final
and the tune type for each entry. output. This is then processsed by LATEX in the
• The build process should produce midi files. normal way, with the result shown in Fig. 1.
Lilypond-book creates a graphic for each line of
3 Combining Lilypond and LATEX music. It also creates a graphic for the whole snip-
Here is a simple example of a LATEX file (small.ly) pet — in our small example, 2b589ef505-1.eps and
containing Lilypond code: 2b589ef505.eps. My project has currently gath-
ered about 200 tunes and the number of small files
\documentclass{article} in OUTPUT hovers around 3500!
\begin{document}
Contents of OUTPUT/small.tex:
\noindent
Some text before a musical snippet.\\ \documentclass{article}
\begin[quote,fragment]{lilypond} \usepackage{graphics}
{ \begin{document}
c’ e’ g’ e’ \noindent
} Some text before a musical snippet.\\
\end{lilypond} {%
Another snippet:\\ \parindent 0pt%
\begin[quote,fragment]{lilypond} \ifx\preLilyPondExample \undefined%
{ \relax%
f’ g’ a’ b’ \else%
} \preLilyPondExample%
\end{lilypond} \fi%
Some more text.\\ \def\lilypondbook{}%
\end{document} \input lily-2b589ef505-systems.tex%
\ifx\postLilyPondExample \undefined%
This is processed by the command line:
\relax%
lilypond-book -f latex --psfonts \else%
--output OUTPUT small.tex \postLilyPondExample%
\fi%
And in the OUTPUT directory Lilypond creates
}
the following files: Another snippet:\\
lily-2b589ef505-1.eps {%
lily-2b589ef505.eps \parindent 0pt%
lily-2b589ef505.ly \ifx\preLilyPondExample \undefined%
lily-2b589ef505-systems.tex \relax%
lily-2b589ef505-systems.texi \else%
lily-2b589ef505.txt \preLilyPondExample%
lily-eb070afcf7-1.eps \fi%
lily-eb070afcf7.eps \def\lilypondbook{}%
lily-eb070afcf7.ly \input lily-eb070afcf7-systems.tex%
lily-eb070afcf7-systems.tex \ifx\postLilyPondExample \undefined%
lily-eb070afcf7-systems.texi \relax%
lily-eb070afcf7.txt \else%
small.dep \postLilyPondExample%
small.tex \fi%
snippet-map.ly }
snippet-names Some more text.\\
tmpMQ9ShM.aux \end{document}

378 TUGboat, Volume 29 (2008), No. 3 — Proceedings of the 2008 Annual Meeting
 A newbie’s experiences with Lilypond, Lilypond-book, LATEX and Perl

If LATEX can fit the snippet into a page in its but lilypond-book complained about not being able
entirety it uses the whole graphic, otherwise it uses to find files. It is just not that clever. It is not able
individual lines, placing some of the lines on the fol- to process my \newcommand.
lowing page. This breaks my first requirement: pages
should contain only an integer number of snippets. 6 Source collections
Ironically the largest collections of traditional music
4 Overcoming the first limitation from all over the world are held in ABC files and there
My first approach to this problem was to wrap the are quite a few search engines tuned specifically for
snippet in a Figure environment: searching ABC sites.12 There is also a Python script
\begin{figure} available that converts ABC to Lilypond (abc2ly).
.... Again, possibly because of the lack of ABC stan-
lilypond code dards, abc2ly does not produce very tidy code and
.... sometimes gets the repeats plain wrong. It is often
\end{figure} brought to its knees by idiosyncratic ABC.
An integer number of tunes then appeared on a
7 And then there was Perl
page, but I found that the positioning of the graphics
was inconsistent, particularly at the end of chapters. Perl is ideal for processing text. Both LATEX and
Google reported that lots of people had suffered from Lilypond are text based, so the marriage is obvi-
this same problem, but I could find no solutions. ous. My collecting of ABC files and their subsequent
Indeed the suggestion was that I abandon LATEX placement in my book is now almost completely au-
altogether and use only lilypond and a particular tomatic:
stylesheet.11 • ABC file arrives in target folder (often via email)
I also tried using the standard utility grep to • A Perl daemon:
find a relation between my LATEX file, the Lilypond
1. makes a backup
file and the lilypond-book -generated eps files. I in-
tended then to use conventional \includegraphics 2. cleans up ABC code
commands to position the graphics manually. This 3. creates an index entry
proved too cumbersome. 4. merges text to precede or follow this item
My final code ended up as: 5. runs abc2ly
\noindent 6. adds name of lily file to compilation list
\begin{minipage}{\columnwidth} • a make invocation puts together book version
\index{mytune:reels}
\lilypondfile{mytune.ly}\\\\ I think of this process as resembling a trout: the
\include{mytune.tex} Perl daemon watches for an ABC file arriving as a
\end{minipage} result of an Internet search — just as a trout watches
Here mytune.tex contains notes pertaining to for minnows! It is not perfect, but I have good error
this particular piece. Wrapped within the minipage, reporting in place and mistakes are easily fixed by
it was guaranteed to appear on the same page. It hand.
might also contain footnotes. When sufficient new tunes have been added to
This example also shows that if the Lilypond the repository, another Perl script employs lilypond-
script is long, it can be stored in a file and referred to book, latex, dvips and ps2pdf to produce the final
with \lilypondfile; lilypond-book then processes copy.
its argument. 8 Subversion
5 Clever includes Small changes are made to this project daily and
Ideally I would have liked code such as: sometimes the editing is done using machines on
different sites. A small change can have a disastrous
\newcommand{\lily}[1]{ effect on the end product. For this reason the whole
\lilypondfile{#1}{#1}}
project is controlled using the Subversion 13 version
.....
\lily{lilys/my.tune.ly}
control system.

11 lsr.dsi.unimi.it/LSR/Item?id=368 12 for example, trillian.mit.edu/~jc/cgi/abc/tunefind

13 subversion.tigris.org

TUGboat, Volume 29 (2008), No. 3 — Proceedings of the 2008 Annual Meeting 379