Toki_Pona-Parser
Toki Pona Parser - A Tool for Spelling, Grammar Check and Ambiguity Check of Toki Pona Sentences
Toki Pona is a constructed, minimal language, designed by the translator and linguist Sonja Lang. Toki Pona favors simplicity over clarity, and touts itself as “the language of good. The simple way of life.”
By virtue of Toki Pona’s extremely small vocabulary, and order-independent syntax, the language is good at talking about feelings and simple relationships, but not about the finer points of politics or silicon-on-insulator microchip fabrication techniques. Tokiponists believe this is exactly as it should be.
Whether you accept the philosophy or not, Toki Pona is fun to speak.
About
This parser analyzes Toki Pona sentences. It ckecks the spelling and grammar. Furthermore, it finds possible grammatical variants. That is, it finds grammatical ambiguities. It helps you to form correct and clear Toki Pona sentences.
This parser is written in SWI-Prolog. SWI-Prolog is a free implementation of the programming language Prolog. Prolog is a general-purpose logic programming language associated with artificial intelligence and computational linguistics. SWI-Prolog supported the Definite Clause Grammar (DCG). DCG is a logic way of expressing grammar, either for natural or formal languages. These scripts contain DCG rules for describing the Toki Pona grammar.
In short: This tool analyzes Toki Pona sentences based on logic and not on “maybe” or “could be”.
It based on the offical Toki Pona book of Sonja Lang ( tokipona.org ), the lessons of jan Pije ( tokipona.net/tp/janpije ), the lessons of jan Lope ( jan-lope.github.io ) and many text examples.
Installation
MS Windows
First you have to download SWI-Prolog for Windows: http://www.swi-prolog.org/download/stable
Install SWI-Prolog:
Select type of install: Typical
Destination Folder: C:\Program Files\swipl
Extension for Prolog files: pro
After this you have to download the Toki Pona parser from this link https://github.com/jan-Lope/Toki_Pona-Parser. Use the button “Clone or download” and then “Download ZIP”. Unzip the file “Toki_Pona-Parser-master.zip”. Change to the new folder “Toki_Pona-Parser-master” and doppel click the file “Toki_Pona.pro”. A new window opens. After the prompt |: you can type Toki Pona sentences.
|:
Linux
First you have to install swi-prolog. Here are the command for Ubuntu/Debian:
$ sudo apt-get install swi-prolog
After this you have to clone the repository with the sources of the Toki Pona parser.
$ git clone https://github.com/jan-Lope/Toki_Pona-Parser.git
$ cd Toki_Pona-Parser
Alternatively you can download the sources as a ZIP file from this link https://github.com/jan-Lope/Toki_Pona-Parser. Use the button “Clone or download” and then “Download ZIP”.
Unzip the file “Toki_Pona-Parser-master.zip” and changed to the new directory.
$ unzip Toki_Pona-Parser-master.zip
$ cd Toki_Pona-Parser-master
Under Linux you can start the script directly.
$ ./Toki_Pona.pro
|:
Mac OS X 10.7 and later
First you have to download and install install swi-prolog: http://www.swi-prolog.org/download/stable
After this you have to download the Toki Pona parser from this link https://github.com/jan-Lope/Toki_Pona-Parser. Use the button “Clone or download” and then “Download ZIP”.
Unzip the file “Toki_Pona-Parser-master.zip”. Change to the new folder “Toki_Pona-Parser-master” and doppel click the file “Toki_Pona.pro”. At the first time you will be asked for the program to open this file. Please select “SWI-Prolog”. A new window opens. After the prompt |: you can type Toki Pona sentences.
|:
Usage
Now you can check the grammar of a Toki Pona sentence. After the prompt |: you can type a Toki Pona sentence. Use neither a space before the sentence nor double spaces.
In this example it is the sentence “mi moku.”.
|: mi moku.
s(dec(subj_p(pronoun(mi)),pred_p(verb_t(moku))))
s(dec(subj_p(pronoun(mi)),pred_p(verb_int(be),adjective(moku))))
s(dec(subj_p(pronoun(mi)),pred_p(verb_int(be),noun(moku))))
|:
You can see this sentence has three grammatical interpretations and at least three meanings. Here “moku” can be a transitive verb (verb_tra), a predicate adjective, or a predicate noun.
Let’s check the next sentence.
|: mi moku e moku.
s(dec(subj_p(pronoun(mi)),pred_p(verb_t(moku),obj_d(noun(moku)))))
The grammar of the sentence “mi moku e moku.” is unambiguous. The first “moku” can only be a transitive verb and the second “moku” is a noun. This sentence (s) is a declarative (dec), simple (sim) sentence. It has a subject phrase (sp) with a subject (sub) and a predicate phrase (pp) with a direct object (obj_d). Here are the abbreviations:
s sentence
dec declarative sentence
exc exclamatory sentence
imp imperative sentence
int interrogative sentence
hdl headline
voc vocativ sentence
inj interjection sentence
salut salutation sentence
com command sentence
des designate sentence
subj_p subject phrase
pred_p predicate phrase
cond_p conditional phrase ("la")
voca_p vocativ phrase
salu_p salutation phrase
obj_d object direct after a transitive verb
obj_i object after a intransitive verb
obj_p object after a prepositiion
obj_a object after "anu"
unofficial_ unofficial word (adjective)
card cardinal number
ord ordinal number
conj conjunction
verb_i verb_intransitive
verb_t verb transitive
verb_p verb pre
be The missing "be", "am", "are", "is" in Toki Pona.
Avoid ambiguous grammar as much as possible! Keep in mind that if a sentence is clear to you, it could still not be clear to other people!
Next example: “ona li pakala e tomo tan jan seme?” This question has two grammatical readings (whom, which_object).
|: ona li pakala e tomo tan jan seme?
s(int(whom(subj_p(pronoun(ona)), (pred_p(verb_t(pakala),obj_d(noun(tomo))),obj_p(preposition(tan),noun(jan),pronoun_question(seme))))))
s(int(which_object(subj_p(pronoun(ona)),pred_p(verb_t(pakala),obj_d(noun(tomo), (adjective(tan),adjective(jan)),pronoun_question(seme))))))
A comma before “tan” turns the sentence into a whom question, because “tan” can only be a preposition here.
|: ona li pakala e tomo, tan jan seme?
s(int(whom(subj_p(pronoun(ona)), (pred_p(verb_t(pakala),obj_d(noun(tomo))),obj_p(preposition(tan),noun(jan),pronoun_question(seme))))))
In the next example a comma can make the grammatical structure unambiguous. But maybe this is not what you mean.
|: jan pona pi meli mi en mi li moku e moku.
s(dec(subj_p(((noun(jan),adjective(pona)), (noun(meli),pronoun(mi)),conj(en),pronoun(mi))),pred_p(verb_t(moku),obj_d(noun(moku)))))
s(dec(subj_p((((noun(jan),adjective(pona)),noun(meli),pronoun(mi)),conj(en),pronoun(mi))),pred_p(verb_t(moku),obj_d(noun(moku)))))
|: jan pona pi meli mi, en mi li moku e moku. s(dec(subj_p((((noun(jan),adjective(pona)),noun(meli),pronoun(mi)),conj(en),pronoun(mi))),pred_p(verb_t(moku),obj_d(noun(moku)))))
If you would like to mark the missing “be”, you can use an apostrophe for the grammar check. This is not an official Toki Pona rule, but it avoids ambiguity.
|: ni li ' moku.
s(dec(subj_p(pronoun(ni)),pred_p(verb_int(be),adjective(moku))))
s(dec(subj_p(pronoun(ni)),pred_p(verb_int(be),noun(moku))))
To reduce the object variants to a noun, you can use “pi”.
|: ni li ' moku pi jan pona.
s(dec(subj_p(pronoun(ni)),pred_p(verb_int(be), (noun(moku),noun(jan),adjective(pona)))))
To mark numbers, you can use a comma or a “#” (unofficial).
|: jan lili li moku e moku.
s(dec(subj_p((noun(jan),adjective(lili))),pred_p(verb_t(moku),obj_d(noun(moku)))))
s(dec(subj_p((noun(jan),card(adjective(lili)))),pred_p(verb_t(moku),obj_d(noun(moku)))))
|: jan, lili li moku e moku.
s(dec(subj_p((noun(jan),card(adjective(lili)))),pred_p(verb_t(moku),obj_d(noun(moku)))))
|: jan # lili li moku e moku.
s(dec(subj_p((noun(jan),card(adjective(lili)))),pred_p(verb_t(moku),obj_d(noun(moku)))))
These scripts can help you to better understand Toki Pona sentences. For example the sentence “sina pona ala pona?” can not mean “Are you ok or not ok?” The verb “are” (or “be”) doesn’t exist in Toki Pona. “pona” can only be a transitive verb here.
|: sina pona ala pona?
s(int(yes_no(subj_p(pronoun(sina)),pred_p(verb_t(pona),adverb(ala),verb_t(pona)))))
This example makes it clearer:
|: sina pona ala pona e ni?
s(int(yes_no(subj_p(pronoun(sina)),pred_p(verb_t(pona),adverb(ala),verb_t(pona),obj_d(pronoun(ni))))))
To say “Are you ok or not ok?” use “sina ‘ pona anu seme?”:
|: sina ' pona anu seme?
s(int(or_what(subj_p(pronoun(sina)), (pred_p(verb_int(be),adjective(pona)),conj(anu),pronoun_question(seme)))))
s(int(or_what(subj_p(pronoun(sina)), (pred_p(verb_int(be),noun(pona)),conj(anu),pronoun_question(seme)))))
This sentence is clearer.
|: sina ' pona pi pilin en sijelo anu seme?
s(int(or_what(subj_p(pronoun(sina)), (pred_p(verb_int(be), (noun(pona),noun(pilin),conj(en),noun(sijelo))),conj(anu),pronoun_question(seme)))))
An other example is the sentence “ona li mama ala mama?”, which cannot mean “Is she a mother?”
|: ona li mama ala mama?
s(int(yes_no(subj_p(pronoun(ona)),pred_p(verb_t(mama),adverb(ala),verb_t(mama)))))
“mama” is a transitive verb here and not a noun. This sentence means “Does she mother (somebody)?” or “Does she wet-nurse (somebody)?”.
The sentence “ona li ‘ mama anu seme?” could mean “Is she a mother?”. But it can also mean “Is she maternal?”.
|: ona li ' mama anu seme?
s(int(or_what(subj_p(pronoun(ona)), (pred_p(verb_int(be),adjective(mama)),conj(anu),pronoun_question(seme)))))
s(int(or_what(subj_p(pronoun(ona)), (pred_p(verb_int(be),noun(mama)),conj(anu),pronoun_question(seme)))))
A “pi” make it clearer. “mama” can only be a noun here.
|: ona li ' mama pi jan ni anu seme?
s(int(or_what(subj_p(pronoun(ona)), (pred_p(verb_int(be), (noun(mama),noun(jan),pronoun(ni))),conj(anu),pronoun_question(seme)))))
You can check more complicated sentences also.
|: ken la jan Lope li wile e ni: sina kama sona e toki pona!
s(exc(cond_p(noun(ken)),subj_p((noun(jan),unofficial_male_name(known_male_name(Lope)))),pred_p(verb_t(wile),obj_d(pronoun(ni))),
subj_p(pronoun(sina)),
pred_p((verb_p(kama),verb_t(sona)),obj_d((noun(toki),adjective(pona))))))
You can check a paragraph with an optional heading and several sentences. But if you use sentences with ambiguous structure, you will get many results. At the end of a heading you have to type the sign “/”, because you can’t use a new line in this version of the script. For example:
en moku
sina moku e moku.
mi moku e moku.
ni li ‘ pona pi sijelo mi.
|: en moku / sina moku e moku. mi moku e moku. ni li ' pona pi sijelo mi.
If you type a wrong sentence, you will get no results. If you press the “enter” key only, the script will stop.
Questions, suggestions, … Please write to me via twitter.
Here an introduction to definite clause grammars (DCG).
For edit the source code you can use the atom editor with the prolog plugin. On MS Windows you can use the the swi-prolog editor also.
To Do
Several unofficial Names, for example “jan Eliku Kulaputon li jan sewi.”
Quotation marks for quotes and unofficial names.
My ideas
Levels of artificial languages
I think there are several levels of artificial languages. These levels depend on the number of active speakers, the level of speakers, the number of good teachers and the number of good texts.
1st level
The language is unknown. Perhaps few people discuss this language. A meaningful application of this language is not possible. There are no good teachers and nobody writes good texts. Most artificial languages are at this level.
2nd level
Hundreds to thousands of lay people discuss the correct application of this language. However, a meaningful application of this language is not possible. There are no good teachers and nobody writes good texts. An example of this level is the “language” Toki Pona.
3rd level
Thousands of amateurs discuss the benefits of this language. Organizations are formed to spread the language. But these organizations are concerned with themselves. The reason is: there are not enough good teachers and there is no strategy to spread the language. There are some good texts, but that is not enough. The number of good speakers is insufficient. People talk about this language, but not in this language. Conversations about everyday topics are not common. Interlingua has this level.
4th level
Hundreds of thousands of people speak this language. People also talk about everyday topics. There are many organizations that want to spread this language. The number of good texts and good teachers is increasing. The language has a long tradition. I think only Esperanto has this level.
5th level
The language has become a normal language. There are enough good texts and good teachers. The organizations that spread this language are becoming redundant. You can find enough partners to practice this language. Dialects of this language are emerging. You can use this language in hotels and airports in at least one country. I don’t think an artificial language could ever reach this level.