Prahok

In my previous post, I wrote a song text down in Khmer. Before I could post it on this weblog, there were some technical issues to resolve like fonts and the like. Some issues were resolved with the help of some of you out there. By this, thank you all.

The last few days, I was struggeling with the following: a text written in Khmer appears as long sentences without word separators. In a western language, we put ‘whitespace’ characters between the words. ‘Whitespace’ characters is the technical term for blanks, tabs, newlines, etc… At first sight, written khmer does not. How would search engines like Google handle this? The basic elements a search engine works with are words (also called terms). Do they break these long sentences into words? Are there any rules one can use to develop some piece of software?

I found a Java program (khwrdbrk.jar) that can do the job, or at least tries to. If you give this program a Unicode file with a Unicode encoded text in khmer, you get an output file in Unicode. But when I opened the output file, the text looked exactly like the original text. The output file on the other hand was bigger than the original?! The reason why is that this program inserts indeed a word break character between each word. This word break caracter is called the ZWSP. It is not visible, or has a zero width! I tested the program on one of my texts. The result is not 100% correct. Some word boundaries were not found.

I was so proud of my previous post, but now I have to admit that it is NOT what it should be. Time for a second try!

The song text from the previous post is from a song sang by a boy. This song is based on an older song, sang by a girl and made popular by the singer Oeun Sreymom. Her song in its turn is the khmer version of an even older Khmer Surin song.

The song is about a girl that saved some money, sold three chickens for money (without telling her mother), just to buy herself a new shirt (shirt: អាវ) which she (and the boys) likes so much that she does not want to take it off (undress: ដោះ). It is this last word (breast: ដោះ) that made others to put new lyrics on the same music. And this introduces a classic word play because of the double meaning of “ឃើញ​ដោះ”. These two words can be interpreted as “wearing a shirt that none seen taking off” or “wearing a shirt that never shows her boobs”.

The text of the song, and now with the necessary ZWSP’s, goes as follows:

​អាវ​ថ្មី​មិន​ខ្ចី​ដោះ

មាន​អាវ​មួយ​សន្សំ​លុយ​យូរ​ខៃ
ស្រលាញ់​ម្លេះ​ទេ    អាវ​ថ្មី​ចេញ​ម៉ូត​ស្រស់
​ពាក់​អោយ​កេ​ដឹង    ថា​មិន​ដែល​ឃើញ​ដោះ
​ទៅ​នេះ​មក​នោះ   ពាក់​តែ​អាវ​មួយ​ហ្នឹង

​អាវ​ជិត​ក​ល្អ​ត្រូវ​ចិត្ត​ស្រី
​លក់​មេ​មាន់​បី មិន​អោយ​ម៉ែ​គត់​ដឹង
​ពាក់​ដើរ​រាល់​ថ្ងៃ ប្រុស​លួច​សម្លឹង
​ខ្លះ​ស្ទើរ​ភ្លឹក​ព្រលឹង សរសើរ​ស្រលាញ់​ខ្ញុំ

​ពាក់​អាវ​មិន​ដែល​ឃើញ​ដោះ ពាក់​អាវ​មិន​ដែល​ឃើញ​ដោះ
​គិត​អីឬ​វាយ៉ា​ង​ណា យ៉ាង​ម៉េច​ស្រីង៉ា ​បង​មិន​ដែល​ឃើញ​ដោះ
​អាវ​ថ្មី​ខ្ញុំ​មិន​ខ្ចី​ដោះ អាវ​ថ្មី​ខ្ញុំ​មិន​ខ្ចី​ដោះ
​កំលោះ​នាំ​គ្នា​ចោម​រោម បើ​សរសើរ​ខ្ញុំ ខ្ញុំ​រិត​តែ​លែង​ដោះ

​រូប​រាង​ស្រី​ទាំ​ង​សម្ដី​វាចា
​ឬ​កពា​ចរិយា អាវ​ថ្មី​ឆើត​ស្រស់
​មាន​អាវ​ថ្មី​មួយ ពាក់​មិន​ខ្ចី​ដោះ
​ប្រុសណា​ស្រណោះ ចូល​ដល់​យាយ​តា

​រូប​បង​ប្រុស​មិន​យល់​សោះ​ចិត្ត​ស្រី
​ស្រុក​សីវីល័យ ស្រី​តែង​ខ្លូន​សង្ហា
​ពាក់​មិន​ឃើញ​ដោះ អាវ​នោះ​យ៉ាង​ណា
​បើ​ប្រុស​សង្ហា ស្វែង​យល់​ខ្លូន​ឯង

​ពាក់​អាវ​មិន​ដែល​ឃើញ​ដោះ ពាក់​អាវ​មិន​ដែល​ឃើញ​ដោះ
​កើត​អី​ឬ​វាយ៉ាង​ណា យ៉ាង​ម៉េច​ស្រីង៉ា ​បង​មិន​ដែល​ឃើញ​ដោះ
​អាវ​ថ្មី​ខ្ញុំ​មិន​ខ្ចី​ដោះ អាវ​ថ្មី​ខ្ញុំ​មិន​ខ្ចី​ដោះ
​កំលោះ​នាំ​គ្នា​ចោម​រោម បើ​សរសើរ​ខ្ញុំ ខ្ញុំ​រិត​តែ​លែង​ដោះ

I checked it carefully: all ZWSP’s are there! I will keep an eye on Google to see if a search on one or more of the above words will hit this weblog.

I will correct the previous post asap.

This entry was posted on Wednesday, April 11th, 2007 at 4:10 pm and is filed under Cambodia, Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.