The Omphalos Context-Free Language Learning

Recent News

Oct 01
The Omphalos competition is over. Congratulations to the winner Alexander Clark and to the other contestants. The official award ceremony will be held during the ICGI Omphalos session. We hope to see you soon in Athens...
The server remains available for your grammar learning pleasure. Don't hesitate to send us your positive results!
Sept 07
Alexander Clark has also solved problem 4, large test set...
Sept 06
The first large test set (Problem 2)  has been solved by Alexander Clark. Congratulations to our new winner!
Sept 04
The last remaining small test set (Problem 2) has been correctly labelled by Alexander Clark from ISSCO/ETI, University of Geneva. The next winner will have to label a large test set...
June 07
New larger testing sets have been added for problems 1 to 6. It is worth trying to solve problem 1 again...
Apr 14
4 new problems have been added... They are even more difficult!
Mar 22
Problems 3,4,5 and 6 (!) have been solved by Erik Tjong Kim Sang from the CNTS - Language Technology Group at the University of Antwerp in Belgium. Problem 2 is still unsolved but i think that we will have to add new problems...

Feb 20 13:40:04 (Omphalos time)
Problem 1 has been solved by Joan Andreu Sánchez from the Departament de Sistemes Informàtics i Computació, Universitat Politècnica de València ! Congratulations to the first winner of Omphalos...

Feb 18
If you are interested in the Omphalos competition, you may also be interested by the learning DFA from noisy samples contest organized by Simon M. Lucas for GECCO 2004.


Omphalos is a context-free language learning competition that forms part of ICGI-2004 . The task is to infer a model of a context-free language from unstructured examples (both positive and negative) and to use that model to label a set of test sentences as being either in or out of the language.


A context-free grammar (CFG) is a data structure that defines a language, including the phrase structure of sentences in that language. A context-free language is one that can be described using a context-free grammar. Context-free grammars are used in natural language processing, speech recognition, defining computer languages as well modelling natural phenomenon such as DNA. The process of learning grammars or languages from examples is known as grammatical inference .

The Competition Task

We have generated some context-free grammars . For each grammar we have labelled a set of sentences indicating whether or not those sentences can be generated from that grammar. The goal of the competition is to infer a model of each language (such as a grammar) using the training data. You then need to tag new sentences, indicating whether the sentences are in the language or not.

To win this competition you need to do the following:

  1. Download the training data and test data.
  2. Train your system using the training data.
  3. Apply your system to the test data, this should result in a list of 1's and 0's, indicating that each sentence in the test set is either in or is not in the language (respectively).
  4. Submit the list of 1's and 0's to the oracle (on this website).
  5. If your results are not good enough, rethink your system and try again.
  6. If your results are good enough, you've solved the problem. Send an email to with your labelling and the problem name to validate your result. Please include also in your email a description of the technique that you have used (the purpose of this is to aid the ongoing design of the competition and will not be communicated to the other contestants before the end of the competition). 
  7. You may then celebrate.

The Winner

There are currently 10 different problems , ranked in difficulty from 1 to 10. The winner of each individual problem will be the first contestant that submits a correctly labelled test set for that problem to the oracle. The winner of the competition will be the winner of the highest ranking problem if a winner exists for that problem. We reserve the right to add additional problems to the competition if the initial set of problems proves to be too simple or too hard.


Omphalos is sponsored by :

Important Dates



February 15th 2004

Competition begins

October 1st 2004

Competition closes

October 11 th 2004

Competition winner announced

October 11 th – 13 th 2004

Omphalos session at ICGI-2004


You can download the data sets for the problems here . The way in which the target problems, training and testing sets were created is defined here .

Submission procedure

You can submit your data on this page .

Further questions

If you have any further questions on this competition, please take a look at the FAQ .
If your question is not on there, please let the organizers know. 
If you are looking for other grammatical inference benchmarks, you may be interested by this repository...

Intellectual Property: We make no claims on the intellectual property of competition participants. Money: All donated funds will be used solely for awards and reimbursing the award ceremony travel expenses of award winners. If there is any prize money left over after the competition it will be returned to the sponsors, unless they'd prefer that it be rolled over into subsequent competitions. Void Where Prohibited: Participants must obey all laws and regulations that apply to them. If anyone's government or employer prohibits them from participating in this competition or from claiming an award, then they must not do so. Similarly, winners are responsible for paying any taxes that might be required. General Disclaimer: The organizers and administrators of the Omphalos competition (Brad Starkie, François Coste and Menno Van Zaanen) reserve the right to change the competition in any way at any time at their sole discretion, without notice, including the right to change the rules, problems, and awards, and the right to terminate the competition. The Omphalos administrators are the sole arbiters for this competition, and their judgment is final in all matters. Eligibility: The administrators and their immediate relatives and immediate co-workers are not eligible for any award.

Omphalos is being organized and administered by:

Brad Starkie , François Coste and Menno van Zaanen

You can contact them for comments and complaints at

Valid CSS!Valid HTML 4.01!