new freelang software ?

hansman · Post by **hansman** » 31 Aug 2003 14:20

Hi!

I'm a student in electronics engeneering, option ICT. Currently I'm still in my exams (2nd-chance exams if you know what i'm talking about

). I am looking for a thesis to make in my final year, where I hopefully will be in in a couple of weeks.

I allready mailed Beaumont about this, but I guess it wouldn't be a bad idea to ask about this in the forum...

The idea is that I would be writing some new software for the freelang-dictionary. I am allready planning a linux version of the dictionary anyway, thesis or no thesis. But if I plan to propose this as a thesis-project to my teachers it has to be a bit bigger than just that.

These are my ideas:

I don't know how you currently update the word lists with new words, but I think it would be great if there would be a program that itself can connect to the internet and download new words. The program should also detect if the user has added new words and propose them automatically to be added to the 'official' word lists. Ofcourse new words would have to be verified first.
The main problem probably is there should be a place to host the server-side program, maybe here at lokanova??? It should also be Linux

So imagine having 2 client programs, one ordinary dictionary-program as we know it but where you have a button 'UPDATE' which uploads new words you have added and downloads new words in the official word list.
Another client program is for people who subscribe as being 'verificators'. They regulary should connect to the server-program to see if there are new words pending to be added to the official wordlist and answer with something like 'ok' 'not ok' or 'not sure'. For example if 3 people say the translation is good we can assume it really is good and add it to the wordlist. In a matter of seconds the updated wordlist would then be available for other users to update. Can you see the wordlists growing??

Personally I would also like it if the wordlists included type of word (like verb, noun, ...) and gender. There could be programs written for updating this too, so slowly but surely the entire community can update these databases so that in a (far?) future freelang could grow to be a very good replacement of ordinary dictionnaries.

Please share your thoughts, I'd be happy to serve the language community in this way. All the software would be GNU licensed BTW.
Hans

kokoyaya · Post by **kokoyaya** » 31 Aug 2003 15:22

Hiya

Thanks a really good idea.
Geache (a Frelang user) is thinking about making the Freelang dictionary in Java (this way, the program would be automatically compatible with Windows, Mac and Linux). I don't know if he would like to work in team but you may be interested...

hansman wrote:I don't know how you currently update the word lists with new words

In a bad way, with a batch file

but I think it would be great if there would be a program that itself can connect to the internet and download new words. The program should also detect if the user has added new words and propose them automatically to be added to the 'official' word lists. Of course new words would have to be verified first.

Great

The main problem probably is there should be a place to host the server-side program, maybe here at lokanova???

Why not on Freelang?

It should also be Linux

Do you mean a Linux server?

So imagine having 2 client programs, one ordinary dictionary-program as we know it but where you have a button 'UPDATE' which uploads new words you have added and downloads new words in the official word list.

Good idea.

Another client program is for people who subscribe as being 'verificators'. They regulary should connect to the server-program to see if there are new words pending to be added to the official wordlist and answer with something like 'ok' 'not ok' or 'not sure'. For example if 3 people say the translation is good we can assume it really is good and add it to the wordlist. In a matter of seconds the updated wordlist would then be available for other users to update. Can you see the wordlists growing??

Waouh!

As a summary, if you really can do what you've just said, that would be really really fantastic. Thanks

hansman · Post by **hansman** » 02 Sep 2003 17:26

Well if it really would become my thesis-project I could surely do this. Currently we have mainly been working in Java but personally I think especially the server program should be written as a linux daemon mainly for speed-reasons

I guess it could also be Java server or something but we haven't seen that yet

Why not on Freelang? Do you mean a Linux server?

Do you think it's possible to run your own daemon on the freelang-server? Where is it hosted?

I do want to say I'm totally not an expert in this matter, but since I have to do something for a thesis anyway, why not make it this? Other thesises are about the same difficulty so it shouldn't be a real problem

Ofcourse I could do something completely else like something with DSP-chips or HF transmission but if there's enough interest and my teachers allow it.....

Geoffrey · Post by **Geoffrey** » 08 Sep 2003 10:32

Your ideas are indeed very nice. But if you want to run a daemon, besides the fact that the server should be Linux, I guess you should use C. Another idea would be to use a database (if not already the case) that could be accessed via, say, PHP, as it is the case, I think, for this forum. This, a small PHP script could be used together with a form to post a word-list, updating the database (new file, and possibly a 'date' field, that gives the last update time). A simple HTTP request to a PHP script would be able to check the update time for a given word-list, and decide if the local list is up-to-date or not.
This could even be done automatically at Freelang program startup.

The dictionary program is currently in Delphi, if I'm not mistaken. A Java version would actually run on several platforms, but a well-written C version would do that too (with the advantage that one does not need to install Java).

Geoffrey

hansman · Post by **hansman** » 10 Sep 2003 08:51

I haven't got much experience with PHP but I guess it could be useful to check the status of the 'database' (wether that will be really stored in SQL or just in some text-files remains to be seen).
I would like this 'UPDATE' button in the dictionary-program though, I think it is the best way to ensure many people will contribute words - the main reason for me to write this program would be to have bigger wordlists

Keep sharing ideas please...

Greetinx,
Hans

P.S. I still don't have all the details about on what hardware/OS's the freelang server runs. Obviously PHP wouldn't be a problem here on lokanova

Geoffrey · Post by **Geoffrey** » 10 Sep 2003 12:12

Sorry, Hans, I hand't seen you wanted to upload new words too, via the 'UPDATE' button. Then, of course, such a button would be a good idea, with an inconvenience, though: you need to manage potential 'conflicts', if some translations are not compatible with others (well, this may have to be made more clear).
Another problem that might occur is if someone does upload translations that may be incorrect, and keeps those locally, even if checkers ('verificators') drop it. In such a case, the same words would be added for verification next time the user updates his wordlist. There should then be a way to ask (automatically) the user to remove this translation.

You can inspire yourself from CVS or the like version-management systems.

As for the server-side data, I'd rather see them in a real database, which would be easier to update and extend (for, say, phonetic transcriptions, or additional information). The storage is eventually in files, but a language like SQL provides you a handy way to access data.

Regards,
Geoffrey

hansman · Post by **hansman** » 12 Sep 2003 00:15

Indeed all problems that have to be thought through before starting.
I've always wanted to become familiar with the CVS system. Now may be the time to do so, thanks for the tip!
About additions to the database: I've allready expressed my wish to include especially gender therein.

Geoffrey · Post by **Geoffrey** » 12 Sep 2003 09:19

You're welcome :-)

If I can help, then it might be my way of participating to the development of Freelang... since I'm not good enough at languages to write a dictionary myself, and I cannot help so much in the program development itself, as I am not a Delphi programmer.

Regards,
Geoffrey

hansman · Post by **hansman** » 15 Sep 2003 01:01

The current software is written in Visual C++ actually but since I'm not familiar with the Windows API or MFC I will have to spend time to learn it (the concepts are the same as in Java though).
In the meantime it is becoming less likely that this can be my thesis, but maybe still another project we have to do this year.
Or else if there is time (there most likely will be) I will just do this in my spare-time.