root upload powers for rosetta on my own project

Asked by Jeff Fortin Tam

I keep bumping into a problem that is so counterintuitive and prevents my progress that I am considering dumping rosetta and forcing translators to just submit .po files through a mailing list.

I would like however to explain my problem before doing such a drastic change and see if it's just me that misunderstand the way rosetta works. Basically, here are the problems I found while using Rosetta officially for my project (Specto):

1- uploading single files is impossible (https://answers.launchpad.net/rosetta/+question/2824)
2- there is no page to track the files you have uploaded that are pending review. You never know when it's done or not.
3- the fact that it is *obligatory* to have the files reviewed each time is a terrible waste of time. I own my own project, I know enough to override my own stuff if need be. Waiting for someone else to review files that I uploaded does not seem to make sense to me. I have files there that have been waiting for 12 days or more (those which were older, I deleted them. Some were months old)
4- from what I would guess, the current system sucks valuable ressources out of Canonical for pretty much nothing (looking at the speed of the uploads, I must assume they are reviewed by humans)

Therefore, as to point #3, I would strongly beg of you to add some "root mode": that select members can override any (or specific) translation files, without needing external review.

The current Rosetta upload system is counter productive. I am sitting here looking at outdated translation templates that will likely induce translation errors from those who would contribute valuable translations through rosetta (wasted work).

Did I miss anything? Is there a way to make this work fast?

By the way, if the tone of my post is aggressive in some way, forgive me. I don't intend to rant/flame, I tried to make it as objective as possible to highlight a huge problem I see in the way Rosetta currently works. If it cannot be done, I will simply withdraw my translations and do it the "manual way", but at least I wanted to submit this to you guys to have your thoughts on the issue.

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
Данило Шеган Edit question
Solved by:
Jeff Fortin Tam
Solved:
Last query:
Last reply:
Revision history for this message
Данило Шеган (danilo) said :
#1

This area is one demanding attention from us, but it's not as bad as it seems to you (which is our fault again, for not documenting it better).

The way it all works is like follows:
 a) You upload a POT file for your project
 b) We approve it manually (this step is needed because people tend to register *others* projects, and provide templates for them, then translators start translating, thus wasting their work which is not going anywhere); we can change this to something like marking your project as "approved project", but it's not easy to find balance between privilege problems, wasted work and maintainer satisfaction. You often need to remind us to approve your specific POT if you don't want to wait for us to get to it (which may sometimes take a few weeks)
 c) With every other upload, be it a PO file or a POT file, it's automatically approved, so no need to worry about it. However, it needs to follow certain policy, so it can be automatically recognised as what it is (i.e. a French translation, or a PO template). Also, it requires next run of our import process to start (which does the automatic approvals), so it make take a bit for it to get approved (between 5 minutes and a couple of hours, depending on how busy our servers are)

So, if you are talking about Specto project, there are no templates uploaded yet:
  https://translations.launchpad.net/specto/0.2/

And, I don't see them in the import queue:
  https://translations.launchpad.net/translations/imports/+index?target=products&status=NEEDS_REVIEW&type=pot

So, you are probably doing something wrong, so let me outline the steps you need to take to start using Rosetta:

 1) Go to https://translations.launchpad.net/specto/0.2/+translations-upload
 2) Upload a *POT* file (eg. named "specto.pot") which is untranslated file; if you are having problems with this step, please send us the POT file and we'll look into helping you
 3) Ping "carlos" or "danilos" on irc.freenode.org/#launchpad (or file a Rosetta question), and we'll set it up, and it'll be imported in a couple of hours
 4) Upload all the translations (you *can* do that one-by-one by going to each language page once steps 1-3 are done, but I strongly suggest doing it with a tarball since you'll be able to upload them all at once)
 5) Let translators know that they can do their work

Also, to answer your points:
1. It's possible, it's just that your translation is not yet set up
2. The page to track files you uploaded is up on https://translations.launchpad.net/translations/imports/+index?target=products -- we are going to add a per-project page very soon, so it'd be even easier for you to track it
3. It's not obligatory to have files reviewed every time, just the first time for a new template
4. Uploads are slow for a number of reasons, and it's rarely people who are the bottlenecks; we are working on improving the speed, though

Revision history for this message
Jeff Fortin Tam (kiddo) said :
#2

Hmm. I'm always working with translations in the trunk, not the 0.2 series:
https://translations.launchpad.net/specto/trunk

Also, I upload .pot files AND .po files. For example, here is my usual search for my .po files in the queue:
https://translations.launchpad.net/translations/imports/+index?target=products&status=NEEDS_REVIEW&type=po&start=0&batch=200
(use ctrl-F to find specto)

If the .pot template upload is indeed instantaneous, there is a big factor of uncertainty on my side (mostly because of the not-uploaded .po files I would guess). That leads me to *think* the .pot file might not have uploaded as well. Especially when I look at the template details and I see:
"166 messages
6 Languages
Owner: Jeff Fortin
Created: on 2006-10-29"

...created on 29th of november? I just uploaded it 3 minutes ago! And if I look at the file properties (in my file manager), I see this modification time: 2007-05-11 23:38:22 EDT
It does seem like it does not upload. I would be very, very, very surprised that my template file had not changed since november. Or maybe I misunderstand what the creation date is for :| ?

Oh, and I was suggested to upload everything in a tarball (in answer 2824). It works, but as you can see, my problem is that those .po files are stuck in the import queue for eternity. That's pretty much why I was asking if there could be an "instant upload" like the .pot files. I take care to not destroy the rosetta contributor's work by merging their changes into my trunk.

In any case, thanks for the quick reply :)

Revision history for this message
Данило Шеган (danilo) said :
#3

Oh, sorry, I looked at the answer 2824, where you are linking to 0.2.

So, the problem is with the layout of your tarball.

We are not yet supporting this kind of layout (LANGUAGE/TEMPLATE.po), since most every other project using GNU gettext files uses (po/LANGUAGE.po) or similar (eg. there would be 'po/specto.pot', 'po/ro.po', 'po/fr.po', etc.—all files should be in the same directory, not necessarily "po", but you shouldn't change the path from then on if you want automatic approvals to work). Carlos is a bit more familiar with these bits of code, so I suspected we also supported things like "ro/specto.po" from his answer in 2824, but I guess not.

It will still work this way if we manually approve each PO file for the first time, but that's PITA, and will require action for each new language you add, so until we add support for layout like you are using, I guess using the general layout is the quick way out.

Basically, before uploading tarball just do a "cd tarball-contents; mkdir po; for i in *; do mv $i/specto.po po/$i.po; done" and add po/specto.pot as needed, and tar that back up :)

I can see you have now uploaded a POT file as well. It should have been auto-approved by now, but if it isn't auto-approved soon, I'll have to look at it in the morning (it might be a bug).

Revision history for this message
Jeff Fortin Tam (kiddo) said :
#4

As currently there is still fr/sv/de/it/ro .po files waiting in the queue since may 10th, I will delete them from the queue, and give your bash magic a shot at uploading a "compliant" tarball. I'll see if that fixes it or not

my current benchmark is the French translation for Specto (it is supposed to be 100% translated; it is my mother tongue).

As I see the translation template on launchpad contains 166 messages, BUT, I have no confirmation whatsoever that it is the same as the one I am using (because all I have is a "creation" date. What does it mean? Is it possible to see the *upload* date?)

Revision history for this message
Данило Шеган (danilo) said :
#5

Your POT file has indeed been automatically approved and imported, as you can see here:
  https://translations.launchpad.net/translations/imports/?target=products&status=IMPORTED&type=pot
(and another one has been imported on May 22nd: that one was probably from the tarball)

So, the remaining bits is having translations automatically approved, so let me know how does uploading a tarball with different layout go.

Thanks for the patience.

Revision history for this message
Best Jeff Fortin Tam (kiddo) said :
#6

It seems it worked :)

for your curiosity, the reason why my folders are set the way they are is that it simplifies greatly the setup.py file we have (actually, could not find another way). Thanks for your help, hopefully this particular tarball layout could be supported in the future (is there any way I can be notified of that?)

Revision history for this message
Jordi Mallach (jordi) said :
#7

Jeff, if you open a wishlist bug against Rosetta requesting your old layout be supported, you'll be notified whenever the bug state changes, until it gets resolved (or in Launchpad-speak, 'Fix released').