7.1.10.1. Standard Bibliography methods in LyX

Both standard methods have in common that they make use of the Bibliography environment (Section 5.1) to list references. When you first open a Bibliography environment, LyX adds a large vertical space, followed by the heading “Bibliography” or “References,” depending on the document class. The heading is in a large boldface font. Each paragraph of the Bibliography environment is a bibliography entry.

At the beginning of the first line of each paragraph, you will see a grayish box showing a number. If you click on it, you will get a dialog in which you can set a key and a label. The key is the symbolic name by which you will refer to this bibliography entry. For example, suppose your first entry in the bibliography was a book about LaTeX. We could choose the key “latexguide” for that entry. You can also give a label, which will be displayed in the gray inset box.

The key field isn't useless. You can refer to your bibliography entries using the Insert->Citation Reference command. Just choose the key inside in the panel Bibliography keys, then transfer it to the Inset Keys panel with the left arrow. Multiple references can be placed by selecting more than one key. This is the first standard way of creating a bibliography in LyX.

The second standard way of creating a bibliography with LyX uses BibTeX. BibTeX is, it is a system for creating a large database of your most used journal references. For all future articles you write, you only need to include this standard database and reference the appropriate key to each reference.

To use BibTeX for inserting references in your document, proceed as follows: at the very end of your document, select Insert->Lists & TOC->BibTeX Reference. In the resulting dialog, fill out the dialog boxes as follows:

For each citation, assuming that the source is in the .bib file, just call Insert->Citation Reference at the correct location in the text, and enter the appropriate reference key.

7.1.10.2. The RefDB method

The problem with both methods is that they don't “speak” SGML. Of course, you can use BibTeX to create a DVI document containing citations and bibliography and from there take the route to PS and PDF, or even to HTML through a tool like LaTeX2HTML, but this is not the route we want to take. We want to take the route through SGML - and that's where both methods fail to follow us.

The general principle of the RefDB method is straightforward: each citation that you want to be treated as a RefDB citation needs to have a role attribute with the value "REFDB". Each citation defines at least one xref element. The value of the linkend attribute encodes the ID of the required reference in the database (if you need references in several databases, this attribute can additionally specify the database). RefDB uses this information to generate a DocBook bibliography element. This contains an entry for each requested reference. These entries are labelled with ID attributes that match the xref linkend attributes in the text. Each RefDB-generated reference entry defines a xreflabel attribute which holds the text that is to be displayed at the position of the corresponding xref elements.

This is all it takes for single and unique citations, i.e. with one xref element per citation element and only one occurrence throughout the text. Both multiple occurrences of the same citation in the text and multiple citations (more than one xref elements per citation element) make things a bit more difficult.

Some output formats require a different formatting for the first citation of a publication in the text and all subsequent citations of the same publication. The first citation is identical with the above mentioned default case. All following citations of the same publication need an additional xref endterm attribute which points to an additional bibliomset element which in turn contains the text to be displayed for subsequent citations. The endterm attribute has the same value as the linkend attribute except that the letter "S" (as in subsequent) is appended to the attribute. See Processing expectations for the refdb DocBook bibliography output.

Compare the way you would use BibTeX with LaTeX (and LyX) to the way you use RefDB (10) to produce a Bibliography (see RefDB and BibTeX comparison):

With the BibTeX method, you proceed as follows:

• You enter your LaTeX references into a flat-file database in the BibTeX format.

• You use style files in the powerful but somewhat cryptic BibTeX format for your LaTeX documents.

• In a LaTeX document, you specify with the \bibliography command which external bibliography file to use. You specify with \cite commands which references you want to cite (and appear in your bibliography). LyX does this automatically for you.

• You run latex on your LaTeX document. This will create an .aux file which contains (among other stuff) a list of all citations.

• Then you run bibtex on your LaTeX document. This will use the bibliography style you specified in the document and will create a cooked bibliography in a .bbl file.

• Finally you run latex once or twice again to finalize your document.

With the RefDB method, you proceed as follows:

• You enter your references for a SGML or XML document into a SQL database. Input can be either RIS, DocBook, or BibTeX. As RefDB is Unix-style, you can write other import filters in any language that can send output to either stdout or to a file. Using a SQL database means better scalability for large collections and added benefit if you share your data with colleagues (think workgroups, departments, access control...)

• You specify the bibliography and citation styles for SGML and XML documents in XML files which are essentially templates for the sequence and appearance of bibliography and citation elements. These are also loaded into a database. This means they are pre-parsed, which speeds up the formatting of bibliographies.

• In an SGML or XML document, you specify an external entity with the name of the SMGL or XML file that will contain your bibliography. In DocBook documents you specify with <citation><xref..></citation> constructs which references you want to cite. Parenthetical and textual citations are supported.

• RefDB uses a DSSSL script to extract a list of all citations from SGML or XML documents into an XML document (which you can edit to add other, not cited references)

• Then you run a RefDB tool on your SGML or XML document. This will use the bibliography style you specified on the command line and will create a cooked bibliography in a SGML or XML file. It will also create a small stylesheet driver file specific for your bibliography style.

• Finally you run Jade or an XSLT processor on your document to transform it to the final output. This step uses the RefDB-created driver files to format the RefDB bibliographies (leaving alone potential other bibliographies) and the RefDB citations (leaving alone potential other citations). The stylesheet driver files essentially take care of character properties like font weight, posture etc. for various parts of the citation or bibliography. The citations are neatly hyperlinked with the references in the bibliography in all output formats that support this.

Your sources remain intact!

Please note that neither BibTeX nor RefDB do any "search-and-replace"-style mangling of your sources. The cooked bibliography is kept in an external file in both cases. This way it is easy to reformat your document for a different bibliography style without touching the document source. And the whole thing works for DocBook, TEI, and any other reasonable DTD (with a little stylesheet tweaking, that is).

Let's see how lyxtox handles all this complexity in the background for you:

When sedscr is run through runsed from lyxtox:

$RUNSED $SEDSCR $1.sgml

it makes a lot of changes to the SGML file $1.sgml that was previously exported by LyX (see Section 7.1.4.1 for an in-depth description of them). The following code in sedscr takes care of citations that were entered as described in Section 5.19.2:

s/<xref linkend="cit:\([^"]*\)">/\
<citation role="REFDB">\1<\/citation>\
/g

This sed code will substitute every cross-reference to a label of the form “cit:IDsome” with a citation of the form:

<citation role="REFDB">"IDsome"</citation>

in the exported SGML file. Then, lyxtox will call refdbxp:

$REFDBXP -t db31 < $1.sgml > $1.full.sgml
mv $1.full.sgml $1.sgml

to transform all citations of the above form to one of the form:

<citation role="REFDB"><xref linkend="IDsome"></citation>

But how are the citations and references going to be formatted? We need special DSSSL instructions for those parts of our document! Fortunately, RefDB provides us with the aproppriate stylesheet. It is generated dynamically each time, using as input the style name we entered in lyxtox as the value of the REFDB_style variable (see Section 4.13), with a call to runbib from lyxtox:

$RUNBIB -d $RefDB_db -S $REFDB_style -t db31 $1.sgml

This will produce the stylesheet ${REFDB_style}dsl (notice the absence of a dot before the dsl ending). For example, if REFDB_style is “J.Biol.Chem.”, then the above call to runbib will create J.Biol.Chem.dsl.

J.Biol.Chem.dsl looks like this:

<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN" [
<!ENTITY docbook-refdb-html.dsl PUBLIC "-//Markus Hoenicka//DOCUMENT 
RefDB DocBook html stylesheet//EN" CDATA DSSSL>
<!ENTITY docbook-refdb-print.dsl PUBLIC "-//Markus Hoenicka//DOCUMENT 
RefDB DocBook print stylesheet//EN" CDATA DSSSL>
]>
<style-sheet>
<style-specification id="html" use="docbook-refdb-html">
<style-specification-body>
(define ABSTJOURNALNAMESTYLE "ITALIC")
(define ABSTVOLUMESTYLE "BOLD")
(define CHAPBOOKTITLESTYLE "ITALIC")
(define BOOKBOOKTITLESTYLE "ITALIC")
(define INPRJOURNALNAMESTYLE "ITALIC")
(define JOURJOURNALNAMESTYLE "ITALIC")
(define JOURVOLUMESTYLE "BOLD")
(define NEWSJOURNALNAMESTYLE "ITALIC")
(define NEWSVOLUMESTYLE "BOLD")
</style-specification-body>
</style-specification>
<style-specification id="print" use="docbook-refdb-print">
<style-specification-body>
(define ABSTJOURNALNAMESTYLE "ITALIC")
(define ABSTVOLUMESTYLE "BOLD")
(define CHAPBOOKTITLESTYLE "ITALIC")
(define BOOKBOOKTITLESTYLE "ITALIC")
(define INPRJOURNALNAMESTYLE "ITALIC")
(define JOURJOURNALNAMESTYLE "ITALIC")
(define JOURVOLUMESTYLE "BOLD")
(define NEWSJOURNALNAMESTYLE "ITALIC")
(define NEWSVOLUMESTYLE "BOLD")
</style-specification-body>
</style-specification>
<external-specification id="docbook-refdb-html" document="docbook-refdb-html.dsl">
<external-specification id="docbook-refdb-print" document="docbook-refdb-print.dsl">
</style-sheet>

This means the following:

For online output (id=”html”), take those define's into account and then proceed to use the part of the stylesheets with ID “docbook-refdb-html” (use=”docbook-refdb-html”). Openjade will look at the external specification with id “docbook-refdb-html” at the end:

<external-specification id="docbook-refdb-html" document="docbook-refdb-html.dsl">

and will see that is is the document whose name is the "docbook-refdb-html.dsl" SGML entity (document="docbook-refdb-html.dsl"). It will then consult the entity declarations at the start of the stylesheet:

<!ENTITY docbook-refdb-html.dsl PUBLIC "-//Markus Hoenicka//DOCUMENT 
RefDB DocBook html stylesheet//EN" CDATA DSSSL>

and find out that the docbook-refdb-html.dsl entity is the one with the public identifier "-//Markus Hoenicka//DOCUMENT RefDB DocBook html stylesheet//EN", it will search the catalog files (see Section 4.5) and, since the lyxtox script stores the RefDB catalog file in its SGML_CATALOG_FILES variable:

SGML_CATALOG_FILES="$SGML_CATALOG_FILES:/usr/share/refdb/refdb.cat"

openjade will finally be able to locate the document containing the rest of the DSSSL instructions necessary for processing the Bibliography. It turns out that, on my system, this is the /usr/share/refdb/dsssl/html/docbook-refdb.dsl, but this depends on how you compiled RefDB - on the value of the --prefix option precisely. The same procedure is followed for the print output (the id=”print” part).

But wait a minute! That /usr/share/refdb/dsssl/html/docbook-refdb.dsl, does it contain any special code for Mathematics, like the one in Section 10.3.2.1? No. Does it contain any of the other special DSSSL processing as described in Section 7.1.5? Again, no. It is a “driver” file containing only the RefDB-specific DSSSL instructions - for the rest, it jumps directly to the standard DocBook stylesheets of Norman Walsh, through the use=”docbook” attribute, just as our J.Biol.Chem.dsl jumps to /usr/share/refdb/dsssl/html/docbook-refdb.dsl through the use="docbook-refdb-html" attribute.

Of course, you can tweak the /usr/share/refdb/dsssl/html/docbook-refdb.dsl to include both the Mathematics (DBTeXMath) and all the other specific code, but this is not acceptable. Clearly, we need an automatic solution.

Fortnunately, there is one: use the ${REFDB_style}dsl stylesheet that RefDB created automatically for us, to create two new DSSSL driver files, one for HTML and one for print output. these stylesheets will take care to jump first to our other DSSSL files - the latter ones will then jump to the standard DocBook stylesheets.

To implement this solution, two new scripts are necessary, awkscr_refdb_html and awkscr_refdb_print. The lyxtox script calls them as follows:

  $AWKSCR_REFDB_HTML  ${REFDB_style}dsl > $HTML_DSL
  $AWKSCR_REFDB_PRINT ${REFDB_style}dsl > $PRINT_DSL

Both take as input the newly created ${REFDB_style}dsl stylesheet (J.Biol.Chem.dsl in our example) and produce a new HTML and print DSSSL stylesheet respectively. Here is the awkscr_refdb_html script :

#! /bin/sh
AWK="/usr/bin/awk"
cat <<-EOF 
<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN" [
<!ENTITY docbook.dsl PUBLIC "-//Norman Walsh//DOCUMENT DocBook HTML 
Stylesheet//EN" CDATA DSSSL>
<!ENTITY lyxtox-html.dsl SYSTEM "lyxtox-html.dsl" CDATA DSSSL>
<!ENTITY lyxtox-onehtml.dsl SYSTEM "lyxtox-onehtml.dsl" CDATA DSSSL>
]>
<style-sheet>
EOF
for id in "html" "onehtml"; do
  echo "<style-specification id=\"$id\" use=\"lyxtox-"$id"\">"
  echo "<style-specification-body>"
  $AWK '$1 ~ /<style-specification/ && $2 ~ "id=\"html\"",$1 ~ /<\/style-specification/ 
  {if ($1 !~ /^</) {print}}' "id=$id" $1
  echo "</style-specification-body>"
  echo "</style-specification>"
done
cat <<-EOF 
<external-specification id="lyxtox-html" document="lyxtox-html.dsl">
<external-specification id="lyxtox-onehtml" document="lyxtox-onehtml.dsl">
</style-sheet>
EOF

This script

1. Prints the necessary entity declarations that have to come at the top of the new DSSSL stylesheet.

2. For every HTML id (there are currently two of them, “html” and “onehtml”), print as style specification containing all lines that do NOT start with “<” in the style specification for the “html” id in the given file, i.e. the newly created ${REFDB_style}dsl stylesheet. In our example, it prints every line not starting with “<” in the following part of J.Biol.Chem.dsl that comprises only the style specification for the id=”html” part:

   <style-specification id="html" use="docbook-refdb-html"> <style-specification-body> (define ABSTJOURNALNAMESTYLE "ITALIC") (define ABSTVOLUMESTYLE "BOLD") (define CHAPBOOKTITLESTYLE "ITALIC") (define BOOKBOOKTITLESTYLE "ITALIC") (define INPRJOURNALNAMESTYLE "ITALIC") (define JOURJOURNALNAMESTYLE "ITALIC") (define JOURVOLUMESTYLE "BOLD") (define NEWSJOURNALNAMESTYLE "ITALIC") (define NEWSVOLUMESTYLE "BOLD") </style-specification-body> </style-specification>

   Obviously, the lines that do NOT start with a “<” are those define's - and those are exactly the lines we are interested in, everything else will be replaced by our own code.

3. Prints aproppriate external specifications that point to our own stylesheets that contain the rest of our customization.

The output looks like this (see also refdb-html.dsl):

<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN" [
<!ENTITY docbook.dsl PUBLIC "-//Norman Walsh//DOCUMENT DocBook HTML 
Stylesheet//EN" CDATA DSSSL>
<!ENTITY lyxtox-html.dsl SYSTEM "lyxtox-html.dsl" CDATA DSSSL>
<!ENTITY lyxtox-onehtml.dsl SYSTEM "lyxtox-onehtml.dsl" CDATA DSSSL>
]>
<style-sheet>
<style-specification id="html" use="lyxtox-html">
<style-specification-body>
(define ABSTJOURNALNAMESTYLE "ITALIC")
(define ABSTVOLUMESTYLE "BOLD")
(define CHAPBOOKTITLESTYLE "ITALIC")
(define BOOKBOOKTITLESTYLE "ITALIC")
(define INPRJOURNALNAMESTYLE "ITALIC")
(define JOURJOURNALNAMESTYLE "ITALIC")
(define JOURVOLUMESTYLE "BOLD")
(define NEWSJOURNALNAMESTYLE "ITALIC")
(define NEWSVOLUMESTYLE "BOLD")
</style-specification-body>
</style-specification>
<style-specification id="onehtml" use="lyxtox-onehtml">
<style-specification-body>
(define ABSTJOURNALNAMESTYLE "ITALIC")
(define ABSTVOLUMESTYLE "BOLD")
(define CHAPBOOKTITLESTYLE "ITALIC")
(define BOOKBOOKTITLESTYLE "ITALIC")
(define INPRJOURNALNAMESTYLE "ITALIC")
(define JOURJOURNALNAMESTYLE "ITALIC")
(define JOURVOLUMESTYLE "BOLD")
(define NEWSJOURNALNAMESTYLE "ITALIC")
(define NEWSVOLUMESTYLE "BOLD")
</style-specification-body>
</style-specification>
<external-specification id="lyxtox-html" document="lyxtox-html.dsl">
<external-specification id="lyxtox-onehtml" document="lyxtox-onehtml.dsl">
</style-sheet>

We then process our SGML file with this stylesheets for HTML output!

Let's examine for a moment what the new stylesheet does:

1. lyxtox calls openjade and tells it to use the “html” id of this stylesheet. This is because, in case RefDB processing is on, the stylesheet for HTML chunked output is defined by:

   HTML_DSL="refdb-html.dsl" HTML_CHUNKS_DSL="$HTML_DSL#html"

   and HTML_DSL is the name of the file created by awkscr_refdb_html above:

   $AWKSCR_REFDB_HTML ${REFDB_style}dsl > $HTML_DSL

   That “#html” is a special Jade/Openjade construct that tells it to start processing at id=”html” of the given stylesheet.

The new stylesheet (which will bear the name refdb-html.dsl, if HTML_DSL is left to its default value) jumps to id “lyxtox-html” which, by the same reasoning as we saw for the original J.Biol.Chem.dsl file previously, will be found in the lyxtox-html.dsl file that contains all our non-RefDB-specific customization. lyxtox-html.dsl, on the other hand, is constructed in such a way that it will call the standard DocBook stylesheets with a use=”docbook” attribute - and the processing chain is closed!

Processing for print output is done similarly: awkscr_refdb_print creates a new print stylesheet from ${REFDB_style}dsl, refdb-print.dsl, then this new stylesheet is called from lyxtox with an id of “print-pdf”, “print-ps”, “print-rtf” or “print-txt” respectively for each one of the print formats:

  PRINT_DSL="refdb-print.dsl"
  PRINT_PDF_DSL="$PRINT_DSL#print-pdf"
  PRINT_PS_DSL="$PRINT_DSL#print-ps"
  PRINT_RTF_DSL="$PRINT_DSL#print-rtf"
  PRINT_TXT_DSL="$PRINT_DSL#print-txt"

Just as its HTML counterpart above, refdb-print.dsl is constructed in such a way that it will call our lyxtox-print-pdf.dsl, lyxtox-print-pdf.dsl etc. stylesheets with use attributes like use="lyxtox-print-pdf", use="lyxtox-print-ps" etc. respectively. Those stylesheets, in turn, will jump to the standard DocBook stylesheets with a use=”docbook” attribute.

How does a typical RefDB bibliography look like? lyxtox copies the $1.bib.sgml file (wher $1 is the argument passed to it) to the file with the fixed name bibliography.sgml:

mv $1.bib.sgml bibliography.sgml

The &bibliography; entity that sedscr_abi appends to our SGML file is mapped automatically to bibliography.sgml through the line

<!entity bibliography SYSTEM "bibliography.sgml">

in the preample of our document (see Section 4.6). Thus, you only have to look at bibliography.sgml - a typical entry there looks like:

<bibliomixed id="IDKARAKAS1992" role="BOOK">
<bibliomset role="intext" id="IDKARAKAS1992X">
    (<abbrev>11</abbrev>)
</bibliomset>
<bibliomset role="intextsq" id="IDKARAKAS1992S">
    (<abbrev>11</abbrev>)
</bibliomset>
<bibliomset role="authoronly" id="IDKARAKAS1992A">
    <bibliomset relation="author">
        <surname>Karakas</surname>
    </bibliomset>
</bibliomset>
<bibliomset role="authoronlysq" id="IDKARAKAS1992Q">
    <bibliomset relation="author">
        <surname>Karakas</surname>
    </bibliomset>
</bibliomset>
<bibliomset role="yearonly" id="IDKARAKAS1992Y">
    (<abbrev>11</abbrev>)
</bibliomset>
<bibliomset role="bibliography" id="IDKARAKAS1992B">
    <abbrev>11. </abbrev>
    <bibliomset relation="book">
        <bibliomset relation="author">
            <surname>Karakas</surname> 
            <firstname>C.</firstname>
        </bibliomset> 
    </bibliomset>
    <bibliomset relation="book">
        (<pubdate role="primary">1999</pubdate>) 
    </bibliomset>
    <bibliomset relation="book">
        <publishername>BoD GmbH, Norderstedt</publishername>  
    </bibliomset>
</bibliomset>
</bibliomixed>

As you can see, it is based on bibliomixed and bibliomset elements. The content of a bibliomixed element includes all necessary punctuation for formatting - we say that bibliomixed entries are “cooked”. The creator of RefDB, Markus Hoenicka, says the following about the role of bibliomixed elements in RefDB (see Formatting DocBook bibliographies):

RefDB does use bibliomixed on purpose. I think the choice between raw and cooked is a compromise between philosophy and ease of implementation, speed of execution etc. I see the main purpose of auto-generating bibliographies not in creating beautiful and philosophically correct *source* documents, but to help users create correct *formatted* output. The intermediate bibliography element is a means to achieve this. The DocBook DTD explicitly defines the bibliomixed element to create bibliographic output that would be too tedious or complicated to create on the stylesheet level alone.