[om-a] critique of OpenMath

[David Carlisle]

Comments on Richard Fateman's Review

The review makes several adverse comments on the current OpenMath work
but apparently they all are special cases of the remark in the abstract:

> One of them is the (apparently implicit) assumption that the
> OpenMath designers, through dint of mathematical thought and the
> advice of the members of the Open Math Society, have solved
> one of the most pressing problems of software engineering today,


"apparently implicit" here presumably means unstated. I don't think
any of the people working on OpenMath would make such a claim.
OpenMath is (or may be) part of a solution, but if such grandiose
claims are made of any software then of course it would be easy to
knock those claims (which appears to be the point of the rest of the
review).

Having said that, both of us work for a company whose principal products
are re-usable software components.  NAG's numerical libraries will be
thirty years old this year and while none of the code in them is a day
over twenty-five the longevity of these products suggests that we are 
doing something right!  The point is that this code was designed to be
re-usable in certain contexts, and OpenMath is a helpful tool for
facilitating re-use in a broader setting.  Trying to re-use software in
a manner in which it was not intended (e.g. embedding a complete algebra
system in another application) is a different kettle of fish and nobody
is suggesting that OpenMath or anything else can make this trivial.  

> At least as often
> such a publication has no impact whatsoever because the author has
> accidentally or perhaps studiously avoided addressing certain
> assumptions essential to the paper. 

This is of course a real danger as the history of publications in more
or less any subject can testify. Whether or not OpenMath will have any
impact will remain to be seen. (It has already had impact on the design
of MathML, but also of course it is still early days to see if MathML
will succeed as we hope it will.)

> The defect in the OpenMath concept is the
> implicit assumption that software is generally re-usable. 

This statement is a straw man, and complete nonsense.  You can design
software to be re-usable, and OpenMath is a tool to assist in this, 
alongside e.g. Java Beans or Active-X.  The fact that some people have
chosen to use OpenMath to link together existing software does not imply
that they believe that all software is generally re-usable.


> All protestations to the contrary,
> it simply does not have any mandate outside the rather simple
> application of denoting what could be trivially done in any
> programming language capable of representing attributed trees. 

This is true (and stated in several OpenMath documents). OpenMath
models expressions as an attributed tree. How that tree is represented
is relatively unimportant, two possible encodings are highlighted: a specific
binary encoding and an XML one, but other encodings are possible
(and have been used in the past, lisp for example).

So while we agree with the comment, we're not sure of the point you are
trying to make. The same point could be made about any tree structure:
any XML document could be encoded in lisp but no one is really
suggesting that the Web should have done that. Or at least, no one is
suggesting that the Web should change to that starting from where it is
now. So XML is a fairly natural (if painfully verbose) format for any
such tree structure and OpenMath is no exception to this.  You may feel
that XML is overkill and prefer lisp, non-lisp programmers may take the
opposite view.  As people with experience in both lisp and XML we feel that
as representation formats they have a lot in common, but that the XML tools
currently available are more flexible, re-usable, and lightweight than their
lisp equivalents.  

>  We'll set this potential quarrel aside except to note that it is quite
> reasonable to believe that MathML has eaten OpenMath's lunch

Given two languages, it is always entertaining to try to picture a "war"
between opposing camps. But there is a large overlap between people
developing OpenMath and people developing MathML and apart from this
shared personnel the two groups keep fairly regular contact.
OpenMath and MathML have different aims and although these aims do
slightly overlap in the area of "Content" Markup for "K12 mathematics",
there are specified conversions between the two equivalent forms in
this area.

If you read the MathML2 recommendation you will note that MathML mechanism
for going beyond K12 (the csymbol element) is equivalent to OpenMath's OMS
symbol, and almost all the examples in the document use OpenMath.

> One comment on a draft of this paper,
> ``In the tried and tested tradition of EU-funded
> projects the software has been kept hidden, under wraps and in a
> permanent state of development for so long that MathML had time to
> appear, crash onto the world scene, get into Mozilla and Explorer and
> become a W3C Recommendation.''}.

While you are free to make comments about funding bodies, this doesn't
really seem to be appropriate here. The main initial production of the
EU project was the OpenMath standard itself, drafts of which were made
public at all stages. The OpenMath libraries and other software have
also been public for some time. Some other software developed on the
project will appear in commercial products. If comparing with MathML
it is worth noting that one of the activities of this "under wraps"
EU funding was the MathML activity of three members of the MathML
working group.

> ``The nth program must have in its phrasebook a way of understanding
> "the meaning of Y to the program $X_1, X_2, \dots, X_{n-1}$ if it is
> to communicate effectively.  Thus the $n^2$ translations are still
> necessary.''

This is not true.  Each program must understand the meaning of Y in
OpenMath which is not always trivial (see Davenport et al's paper discussing
branch cuts).  Thus there are n translations required.

> ``Any user familiar with more
> than one computer algebra system can testify 

the fact that some of the papers discus problems encountered seems at
odds with your earlier assertion that the OpenMath work assumed
that all problems had been solved?


> The alternatives to OpenMath are not considered
> in the SIGSAM papers, but a technique using \TeX\ macros for semantics
> has been used successfully for the latest edition of GR allows for
> communication and typesetting, as well as testing the mathematical
> validity of formulas.

Of course, for any particular field one just needs to specify the syntax,
and whether you use TeX style \ {} or XML style <> makes no difference.
However if you wish to use existing TeX macros then you have to accept
that you are imposing your own interpretation of their semantics on them,
and cannot necessarily treat the same macros from an independent source in
the same way.  There are also always tradeoffs between using a system that 
aims to be general and extensible (OpenMath) with using a system tailored 
for one particular use. (You give two examples, one of some TeX macros and 
one of a lisp encoding).


>  See for example Norbert Kajler,
> Neil Soiffer: ``A Survey of User Interfaces for Computer Algebra
> Systems''. J.Symb. Comp. 25(2): 127-159 (1998).

> The SIGSAM article by David Carlisle on OpenMath, MathML, and XSL
> simply cloaks the easy part of the problem in obscurity

Glad you liked it:-)

> ...
> The simplicity of this is obscured
> by the demand that instead of writing a program (once) that returns a
> displayable object, we should write, and debug, a collection of formal
> XSL that describes the program that displays the object.  In the
> process of making this indirection, the real difficulties are
> forgotten.  

No, not forgotten, just moved. You _could_ write a one off program
to display OpenMath (and some have done that) there are of course some
advantages to that approach.  We discuss displaying OpenMath by
transforming to MathML partly to give a comparison of the languages but
also because by converting to MathML you get OpenMath displayed in
whatever MathML renderers you have available  (currently there are
systems going from MathML to pdf, native rendering in
Mozilla, in Java applet windows, in the Amaya browser, etc). It is a lot
less effort to write a transform to MathML than it is to try to write
native OpenMath display code in each of these contexts.  Moreover if
you wish to customise your display (since e.g. not every mathematician in
the world uses Anglo-Saxon notation) you need only do it once.

> It ignores (for example) the interesting problem of
> breaking expressions over several lines (at all, much less
> efficiently), 

Yes, I believe that it it is the job of the MathML renderer to line
break the given MathML (which many don't do so well at present,
although Mathematica is not bad, which is hardly surprising given Neil's
longstanding interest in this, as you point out).

> variations in displaying precedence by space or
> position (when are parentheses required), and many other issues
> explored in the previously noted paper by Kajler/Soiffer. 

This in fact is handled by the prototype conversions, even if this
wasn't made explicit in the paper. In particular some particularly
nasty examples of omitting parentheses around unary minus/implied times
combinations that Neil posed to me are (I hope) handled.

This separation isn't just a practical matter to leverage off existing
MathML implementations, I think it is conceptually the right thing to
do. If you are introducing a new function xxxxx in some Content
Dictionary, you need to say whether it displays as F(x,y) or x + y
or  (x)   or whatever, and one way of doing that is to map to a
    (y)
fragment of Presentation MathML that describes the layout scheme.
The problem of line breaking the expression as a whole within the
measure available is a different problem best specified at a different
place. Especially for online systems where the measure may change at any
time by someone resizing a window (unlike the traditional approach
taken in most TeX formats).

> The task of presenting written mathematics is solved rather well by
> \TeX\ which, in combination with the generation of diagrams, curves,
> and figures using associated tools, is now in common use by many
> publications.

TeX though makes no attempt at all at breaking Display
Mathematics. Traditionally it is all done by hand by the TeX author.
(The TeX Book comments that breaking mathematics is hard and rather than
get it wrong TeX doesn't attempt it...)
Packages such as breqn.sty from Michael Downes of the AMS attempt to
encode a line breaking algorithm via TeX macros but this is still highly
experimental. 

>   Can one do better?  

One can do better than TeX as a language for authoring and archiving
documents, yes.

We like TeX and one of us has devoted a large part of his life to supporting
it.  (David is one of the authors of LaTeX2e which is by far the
most commonly used TeX format, including we note the format you used for
this review.)

But take a typical journal accepting author produced TeX files: a
reasonable proportion of them are handled by printing the files to paper
shipping them off to some other country with cheaper labour costs, and
getting the entire document rekeyed. TeX allows authors a lot of
control, but this often makes the files unusable for any purpose other
than printing on the author's system. In particular over clever macros
that break when a journal tries to typeset the document to a different
page size and with a different font set.  Rekeying from printed output
isn't our ideal of electronic document processing.

Having a level that allows some notion of validation and constraints
over document authoring is in general a good thing. Having said that, if
hand writing a document, it makes far more sense to author in a simple
language (like a dialect of TeX), or an application like Stilo's MathWriter,
and then convert to OpenMath and/or MathML. No one is suggesting that either
OpenMath or MathML is directly hand author-able in their native XML
syntax.

One can still use TeX as the typesetting engine.





> Supposing that Mathematica's Mathlink is a special treat allowing
> programs to be written in C and connected to another program via
> TCP/IP, is hardly an advanced notion.

This seems to be a comment about Mathematica rather than OpenMath. The
paper was discussing interfacing OpenMath to Mathematica, so not
unnaturally it used Mathematica's interface to external
programs. Whether that interface is good or bad architecturally is
important for Mathematica users, but not that relevant to OpenMath as
a concept.

> I found disturbing the apparent controversy in introducing scope and
> binding through the lambda mechanism 

We don't know why you say "controversy". There was some discussion about
the exact form the binding operators should take but we don't remember
any controversy about the principle (and adding binding operators was one 
of the first actions taken once the EU project started).

> Clearly the tempo of web-based enterprises 
> requires a faster cycle than is available
> with the processes put in place by the  OpenMath Society.

We have no evidence that any "processes" are holding things up.
Maybe development isn't as fast as you would wish, but unless you have
evidence that there is a whole slew of ideas that are being "hidden"
waiting for "approval" by the Society then we don't understand this
comment at all.


> Can OpenMath be saved?  Perhaps.  

A related question is, does the world need a language for encoding
semantics of Mathematical expressions. If the answer to that is "No", then
OpenMath can't be saved. If the answer to that is "Yes" then perhaps
OpenMath can provide that, and even if not (predicting whether a
language is a success isn't really possible) the effort that has gone
into it won't have been wasted. For example the OpenMath Content
Dictionary effort is mirrored in appendix C of the MathML spec, and any
other similar language will need some similar construct and could if it
chose make use of those files. As they stand they probably need
improvements even for OpenMath use, but that will happen as and when
more people start to use them. 


> If some real, hard,
> applications are attempted, it may be possible to finally
> gather some understanding of what is lacking.
> We already know how hard it is to re-use software. I suspect that each
> new encoding project may require the construction of a substantially
> new CD depending only on the most primitive previous CDs ... because
> the previously constructed ``advanced'' CDs are insufficiently
> re-usable.

This is true. Some CDs will be essentially usable with just one system
and some will be designed for communication between systems.
There will be a tendency for some more advanced/specialised CDs to be of
the former kind, but is that a bad thing?

> Encoding the complete information from a large
> table of integrals.  Encoding a book on functional analysis, a book on
> methods of mathematical physics, a book on combinatorics. 

These would all be good test cases.

> Encoding any number of journals whose contents are presumably already
> encoded in \TeX.

Getting any kind of structured markup out of the TeX in a typical
journal submission is hard. Many Journal publishers will tell you that
getting even print out of the TeX supplied by a typical author is hard:-)  

David Carlisle and Mike Dewar.

_____________________________________________________________________
This message has been checked for all known viruses by Star Internet delivered
through the MessageLabs Virus Control Centre. For further information visit
http://www.star.net.uk/stats.asp
--
om-announce at openmath.org  -  public announcements concerning OpenMath
Post discussion to om at openmath.org
Automatic list maintenance software at majordomo at openmath.org
Mail om-announce-owner at openmath.org for assistance with any problems