Why I Don’t Think Supporting Machine Translation Systems Is A Good Idea
I’ve been contemplating about machine translation (MT) for a while, but finally the recent TriKonf in Germany accelerated my thoughts on the topic. Machine translation was discussed there, and here are my comments.
(Please note: with “supporting” in the headline I don’t mean “using”.)
First I would like to say I’m no big fan of MT. I don’t care or mind too much about MT as such and I can live happily without it as a translator, but on the other hand I think when it crosses certain borders it could have a negative impact on our work. It is how and for what purpose it is used what makes the difference in my eyes. As long as MT systems are used to support pro-bono efforts or share medical knowledge to save lives where otherwise this wouldn’t materialize for budget restrictions, I have no problem.
What makes me concerned (also given how easily some colleagues might be willing to have a help in form of semi-artificial pre-translations from MT systems and don’t realize the potential risks of the whole chain) is the following:
1) Some data from translators help improve MT mechanisms used for commercial purposes, i.e. to partially or fully replace the paid translators
While some suggest “all is fine, they predicted miracles already 50 years ago and still nothing”, there are on the other hand statements from experts in the TM/MT processes like Emmanuel Planas or Philipp Koehn indicating that the situation is changing (maybe slowly, but surely). P. Koehn said at the TriKonf that high quality translations are “currently almost exclusively done by human translators” – this looks great at the first sight, but there is “currently” and “almost exclusively”. I think the recent boom of extensive data sharing, cloud systems etc. changes the situation because translators (or their work) are no longer isolated from each other.
Some users of MT are (depending on their settings and agreements) submitting their own translations back into MT systems, and these can learn/improve based on this input. This fact might be diminished by a counter-argument that input from one translator has no real impact. Fine. But imagine you would have input from 10% of all translators to improve the MT algorithms. (As it was said at the TriKonf, more is better.) In my opinion that changes the perspective.
Saying “your own input makes no effect” (read “harm” in my eyes) sounds somewhat utilitarianistically to me, i.e. “it is only the large scale that makes the difference, and you can profit from what 99,99% of translators have been or will be doing, so don’t care about how your 0,01% input can affect you or the profession as a whole”. Well, that large scale had to start somewhere, right? Honestly, I don’t like this way of justification.
(I’m aware that MT systems are loaded with plenty of data from existing translations anyway and I know that the pool of publicly accessible translations is huge, but I see a difference between accepting/ignoring a status quo on one hand and getting directly involved in the system on the other.)
The problem is that any data provided to the MT system is/might be used to teach the MT system to create better constructions, evaluate prevalence etc., eventually leading (especially with a large pool of segments) to a better “conglomerate” than if using the same words/grammar on random basis. If these conglomerates – based on a translator’s earlier input – are then used to replace or reduce his/her own services, it means that providing the input is “feeding the enemy” (even if on a very small scale).
2) MT is to some extent supported through post-editing machine translation (PEMT) jobs
The other potential issue is post-editing of machine translation. While the benefit might be faster work, the risks are:
- Lower price per a word (this is quite certain, unlike the benefit). Of course this would ideally be compensated by higher output, resulting in the same hourly rate, but that is not granted – there are quality issues and certain language pairs are more “demanding” than others because of their grammar, flexibility etc. Also, in case there is no significant excess demand that would make a translator busy – in terms of unchanged amount of working hours – this potential compensation is questionable as the monthly income would decrease.
- Supporting the idea of price-focused clients that MT is something to make the professional translation cheaper, or even unnecessary. (A kind of a vicious circle.) Imagine you come to a restaurant and instead of ordering a steak, you give the waitress a ready-to-cook frozen meat of unknown quality and origin and ask them to warm it up. Do you like this idea? In my eyes, clients requiring PEMT act just like this.
- As above, potentially helping improve the MT algorithms (that depends on how does the client treat the final translation).
With this post, I have tried to outline some facts and risks I see in relation to MT. I decided earlier to act based on what they mean (or might mean) to me (or my colleagues). I say openly I won’t be supporting this system – I won’t wilfully participate in projects that explicitly expect me to “feed” the MT databases to improve the learning curve of MT, and I won’t do post-editing of machine translations. Surely I’m a single drop in the ocean, but I’m open to the idea of forming a group with other drops – MT-resistant translators.
PS I do understand MT won’t succeed for a long time to come in certain fields or language pairs that are more sensitive to “human touch”. I just prefer early caution to late unpleasant surprise.
Did you enjoy this post? Feel free to share it or join the list of subscribers in the right menu.
You making valid points and I mostly agree, but much of the same would apply to CAT tools, yet they are here to stay. MT is going to make inroads into the world of translation and eliminate real people’s jobs – I have no doubts about it. I used to dread it, but not anymore. Why? Because MT is going to take away the most boring and least creative of jobs, but it will leave the more complicated, interesting work to humans. If you want to continue to be a translator you must lift the quality of your work and your specializations to a higher plane. Let the machines do the tedious, uninspiring work and let me do the more colourful stuff.
Try to stay upbeat – it’s better for you.
Hi Marinus, thank you for your comment. Just to clarify: I’m not concerned about my own workload or specializations at this point (and I hope I don’t produce garbage translations), and I personally don’t feel downbeat. I only tried to outline a few risks some translators are perhaps not fully aware of, consequently supporting something (perhaps because it is trendy and promoted by certain developers or agencies) that can take more from them than give to them (when looking at the final result).
I agree with every word in your post
Thank you for saying it so well!
Pingback: Why I Don’t Think Supporting Machine Tran...
I love the cooking metaphor! Great article!
Interesting post, Tomas. You say “Supporting the idea of price-focused clients that MT is something to make the professional translation cheaper, or even unnecessary.” There are a range of emerging use-cases for translation where human translation is not warranted, or even possible. These typically involve the translation of user-generated content, where online on-demand translation is required in ‘Google time’. Much of this data is very disposable content; pretty much as soon as it is published, it becomes obsolete.
@Andy: I don’t object anything against the requirements for “Google time” translations, but my post was about something slightly different. If there were no real-time translations in the past, then indeed no harm is done when MT takes charge of that, as it is an addition to the market offer (I’d say some of these texts were/are translated by humans though, even if with a slight delay, so I think it is not a mere addition, but let’s ignore this subset now).
My point is: MT tries to step into areas that were earlier exclusive to human translators – I believe we both can agree on that. And in my opinion it does so not (only) to help them, but to learn from them (via PEMT or publicly accessible resources) and, to some extent, to replace them (even if only by creating pre-translated texts). Is there any reason why a professional translator shouldn’t be cautious about this? (And now I mean it on a general level, all translators, not only those working in very special – or difficult for MT – fields etc.)
Sorry, posted too soon … I meant to continue:
Some examples include hotel or product reviews, online chat, social media posts in the form of tweets, blogs, etc. While this type of data is becoming more and more prevalent given the ever-broadening access to the Web to more and more users speaking a wide range of languages, it has next to no shelf-life at all; one might even say that as soon as online chat between speakers of mutually unintelligible languages has been facilitated by MT, the data has no further purpose and may immediately be deleted. For all these use-cases – at least in the initial stages – state-of-the-art SMT engines are already capable of producing ‘good enough’ results, which are fit for purpose. For many (such as multilingual chat), real-time translation can only be facilitated by MT; there simply isn’t time for a professional translator to be part of the loop.
I attended Andy Way’s presentation at the British Academy event on “Translation in a Digital Age” on Monday and I know from my own project management experience that much content goes untranslated because it can’t be done fast enough. The amount translated daily is more than the content of one million books – ie more than available translators can handle. There is a bigger demand for than translators can hope to meet. Often a gist translation is sufficient.
To quote JFK: “Change is the law of life. And those who look only to the past or present are certain to miss the future”. No-one anticipated the death of the translator on Monday. Translators have an opportunity to cooperate in the development of customised engines and make themselves more productive. Like Marinus, I would prefer to concentrate on the higher end, more creative and interesting marketing texts where cultural knowledge is important.
@Karen: In case MT would focus only on texts that would be otherwise left untranslated, I think everything would be fine. But this doesn’t seem to be the case here, when PEMT jobs from agencies or end clients serve not only as a way of getting something – what wouldn’t normally be translated at all – polished (so actually giving translators more work), but sometimes also as a way to reduce translators’ input/work and make them just editors instead of translators (I understand not everyone is or will be affected, but both affected and non-affected are the same professionals; and not everyone can do marketing texts – or any other “sensitive” field – just like there are not only teachers of mathematics at schools).
I have no problem with changes and I definitely don’t intend to dwell in the past (or present). But there are different “alternative” futures. It is up to us (everybody, indivudually) which way we want to progess. As was said at the mentioned Trikonf event, “the future is what we make of it”. What are the benefits in (eventually) making MT part of my/your/our future that ultimately outweigh the negatives?
You mention that MT engines can increase productivity. So far so good (I will omit the aspect of different fields or language pairs, as in certain cases there is no benefit here), but:
1) How do you know that the very same MT engines won’t take over (also thanks to linguistic improvements submitted by humans) that or that share of translations in years to come, leaving the human translators (originally working in that or that field) and their customised engines out of the loop?
2) How do you know that there is so high demand for well-paid translations (or post-editing jobs) that the existing human translators cannot handle it, i.e. that there would actually be room for everybody to translate (or post-edit) more words within one month, without actually decreasing – or even only keeping it the same – everyone’s monthly income (compared to a scenario without MT)?
I mean, to translate x words within y hours per n cents each is the same as to post-edit (just an example) x words within y hours for n/2 cents. If PEMT (now I mean “internal use” by a translator, not jobs distributed for post-editing) should be beneficial, it would have to be so that an average – I don’t mean of average quality, but of average workload, methods of work, computer skills, fields of specialization etc. – translator A (not using MT at the moment) is continuously forced to refuse (well-paid) jobs because he/she is too busy at their current n rate, or that an average translator B (already having MT engine/plugin) has some free time he/she could dedicate to more work (without decreasing the hourly income) at n/2 rate. How common are these scenarios with average translators in your opinion?
These questions are not meant to belittle your thoughts, I just would be really interested to know the details/background. Thank you.
This issue is not whether to adjust to new technologies or not. The issue is: is this new technology going to produce good translations? My impression of CAT tools and MT is that they have been created in the sole and only goal to rob translators, so that intermediaries could make a fatter profit margin!
It all started with SDL, who sold CAT tools to intermediaries with the solid argument that they might go under if they did not buy it, because all (!) of their competitors would buy it, thus offer lower prices to end-customers. At the same time, SDL was using the same software internally, thus producing more repetitions, thus offering even lower prices to end-customers… As to translators, agencies then forced them to buy this expensive tool so as to earn less in a translation market that was going downwards…
A race to the bottom, as some call it rightfully!…
To me, the next idiotic move is to make the end-customer believe that translation can be available for free or for not much and that a machine can do the job!
People who buy CAT tools and who participate in machine translation post-editing are collaborating to kill the translation business, while producing erroneous translations, or working at a loss. Post editing machine translation will be less paid than translation (that’s the whole purpose of course!!) but will take long hours of work (remember, translators are not paid by hour but by word!!!) of rewriting…
Let’s stop having crazy computer programmers and silly agencies tell us how to translate!
There is no miracle : a good translation takes time, including proofreading by the translator himself (before it is mutilated by some silly “revisor”) and it takes expert and experienced, thus expensive, translators.
This trying to make machines translate human thoughts is LEADING US NOWHERE. IT WILL NEVER WORK.
The meaning of a word ENTIRELY depends on its CONTEXT. A machine cannot foresee all the possible contexts in which a word will appear. This attempt is GUARANTEED TO FAIL and translators will find themselves working at a loss, night and day, to compensate for too low rates. You’ll die early, guys, that’s what’s awaiting you…
Tomas, you may have examples of agencies who apply your formula ‘as is’, namely:
“I mean, to translate x words within y hours per n cents each is the same as to post-edit (just an example) x words within y hours for n/2 cents.”
They will rapidly go out of business. What we are seeing in the post-editing scenario is that rather than x words being translated within y hours, we are seeing kx words being translated in that time, where k can be anything up to 5 times or more. By the same token, the denominator in your formula is not 2, but is less than 2 (say, 1.5). That way the translator — admittedly now working as a post-editor — achieves a huge productivity improvement albeit at a lower rate, but at a rate that allows him or her to make more money. That way everyone wins: the agency, the client, _and_ the translator/post-editor …
@Andy: I intentionally wrote “example”, understanding the denominators may vary – my intention was simply to indicate the increase of output for calculation purposes.
In regards to super-high productivity, as I already wrote to Karen above: How common are in your opinion scenarios with average translators which are either (without MT plugin) continuously forced to refuse (well-paid) jobs because they are too busy at their current rate, or which have (with MT plugin) some free time they could dedicate to more work (without decreasing the hourly income), and at the same time there actually is significant demand for this extra work? I mean, there should supposedly be a huge overhang of PEMT jobs (sufficiently paid to match the usual hourly rate) right now if it is so beneficial for clients that they enquiry about projects that wouldn’t otherwise materialize (hence the boost in demand), but somehow I don’t see it. Are you aware of any case study about this?
If the demand is about the same, then the result (with MT involved) is lower costs – and lower income. With similar logic, there should have been a significant increase in demand with the arrival of TMs years ago, but did that happen? As far as I can see the only thing that happened on a large scale was for many agencies to demand a “Trados scheme”, but barely to provide more work. I understand you are busy, but I’d appreciate if you can elaborate at least a little bit how MT is to be any different, i.e. not just to make translators earn less (per hour/month).
Another aspect which I don’t see to be reflected too much by MT promoters in general is productivity vs. subject and language pair. Please can you advise which MT system in which language pair and subject field increases productivity “5 times or more” (and what is the qualification of the “tested” translators)? Even if I set out into “pro-MT” waters, I read e.g. about increase from 20% to 134%. (C.f.
P. Koehn said that current MT is good at weather forecasts which sounds plausible (then I don’t know any human translator working on forecasts). But honestly 5-times increase sounds to me rather like a wish than a reality or something achievable thanks to MT (now or in the near future) in projects of all types, not just in projects “tailored” to skills of MT systems.
But even if PEMT jobs would lead to higher monthly income without increase of working hours, there are still the “non-monetary” negative aspects of this type of work (MT learning + quality/psychological factor). “One fear is that too much post-editing will distort an individual’s
perception of the language.” (Nigel Bevan 1981/2 – http://www.mt-archive.info/Aslib-1981-Bevan-1.pdf) – I can clearly see how when someone is continously reading unnatural structures, their linguistic “knowledge” or preference gets (negatively) affected over time; just like listening to children (babble) songs likely won’t make someone an acclaimed poet.
To close my post, I’d like to mention this line from Nataly Kelly “The tools of the future depend on translators” which introduces an interesting paragraph here – http://www.huffingtonpost.com/nataly-kelly/why-translators-are-the-n_b_3150209.html?utm_hp_ref=tw
I totally agree and have flown this flag for years now — to the chagrin of a few of my long-time agencies trying to insist that I employ their favorite CAT tools. (Yes, in my book MT includes using CAT tools.) If I bought and had to maintain every single CAT tool that every one of my agencies used, I’d go bankrupt — and insane.
As I note in cover letters and my CV:
As I’ve posted in another group here, my experiences with CAT tools has, to put it as tactfully as possible, been less than positive. Their increasing prevalence here (in European agencies) has taken away translators’ professional sovereignty, reduced our income (i.e. forced prices into the cellar), while costing us precious time and money. (Don’t forget, one must purchase & then babysit these high-priced, fussy programs, learning to use them (and their 350-page –?!?!?– manuals) while simultaneously maintaining one’s regular workload to pay for them!)
Then, to add insult to these injuries, it is my experience that CAT tools lower the quality of translations overall, e.g. insisting on plugging in fallible terminology simply because some other (lazier) translator’s version got into the TM first. And now I – a conscientious/experienced/traditional translator – must take the time to circumvent the CAT tool’s robotic insistence that such erroneous TM entries be used, when normally I need only to research and WRITE THE CORRECT TRANSLATION. That time spent fighting with the CAT tool to get it right (at a much lower per-word rate) makes my translation & their TM better — but as a freelancer, my next translation is almost always for a different client on a different topic, meaning same battle, different TM.
Meanwhile that very same battle — my underpaid, conscientious labors and “rage against the machine” — supply it with the very “weaponry” it is using to debilitate and ultimately kill off my profession.
For example, it has been my experience that e.g. TRADOS & WordFast use GoogleTranslate to plug in an initial machine translation for my (IN!)convenience. But — over the years I’ve seen its relevancy improve by leaps and bounds. This, in my opinion, is the double-edged sword that is killing off the translating profession. Because every entry that human translators add to the CAT tool/MT machinery is
(true), a word or phrase that one need not type ever again (FOR THIS PARTICULAR CLIENT), but
it is another brick in the wall forcing good writers out of the profession, while keeping legions of less literary-minded drones at work churning out ever faster, less nuanced, grammatically inferior translations.
Now, am I saying that all translators who acquiesce to using CAT tools are “lazy, illiterate drones”? No. I agree that a CAT tool used for in-house translations for one company to maintain and homogenize its corporate terminology is very useful. Extremely so. No question there.
But for many freelance translators, who will write thousands of translations for hundreds of different clients over the course of their careers, the mandatory use of CAT tools can be an albatross around the neck that inexorably leads to lower income and even unemployment.
I’ve been extremely fortunate to occupy a small enough professional niche that I’ve been able to stick to my high-quality, low-tech guns and voila — agencies CAN and DO accommodate traditionalists like me, because they don’t want to lose what counts: being able to deliver on-time translations of high literary quality. But for many freelancers, who don’t have as much experience or long-term agency affiliations, that low-tech, better-paid, creative avenue is frequently blocked — made inaccessible by their own unwitting professional predecessors.
Oops — I note that using << omitted that text. Okay …
Here (If anyone cares) is what it was supposed to say:
"As I note in cover letters and my CV:
"I am a traditional translator, i.e. I'm an experienced and conscientious professional writer who researches and incorporates cultural and linguistic nuances into my work – and who has thereby learned the hard way to avoid invasive CAT tool programs. Such fallible, nuance-hostile, technical 'nannies' impose unwelcome hindrances and semantic limitations that only add to and distract translators from their work loads, while depressing the prices they are paid for their increased labors.
The flexible and creative human mind is infinitely better-suited to evaluating and translating human communications than any robotic, bean-counting processor.” (…)
Thank you for your post, Beth.
In regards to TM quality and “quality” (depending on the source and purpose), you might be interested in this doctoral thesis (in case you don’t know it) by Ana Grueberof Arenas, specifically the chapter “5.5. Error classification” and the parts that follow:
Thank you, Tomas, for inviting me to comment on your blog. There are many interesting questions that I have personally pondered about for quite some time. From a researcher perspective, “most”, not all, studies that measure the impact of MT in productivity and quality of post-edited material in very particular scenarios and in experimental environments (as close to reality as possible) that I have seen to date show that there is an increase of productivity without decreasing the quality of final text (and in some cases, increasing it). Also, there is a very high subject dependency, as you pointed out, but this is also common when using TMs or even when translators translate on their own. Regarding the actual percentage of increase, as you mentioned, there is a high dependency on the subjects but also on the language combination and quality of the output. It is difficult to draw a unique percentage for all cases, but I think the formula “x words within y hours per n cents each is the same as to post-edit (just an example) x words within y hours for n/2 cents” is a good start. I applied a similar formula in my thesis to try and find out an estimation of the discount amount applicable with high productivity increases.
Having said this, it should be interesting to see a study that shows that MT does not increase productivity (for example, if the quality of the output is low) or that MT will cause lower final quality. In fact, one of the issues when talking to the translators in experiments was that they often receive low quality output to post-edit, and this causes a certain amount of frustration, to put it mildly.
From a practitioner point of view, I see that MT is more or less integrated in the localization workflow with a certain degree of success, some outputs reach a very high level of quality (even by human evaluation standards not only automatic scoring) and translators agree to discounts because of this. I also see, however, very poor output with unreasonable discounts. I think that Kirti Vashee’s post was trying to offer advice on how to work with different MT outputs and realities. This is certainly needed.
From a personal point of view, I do agree that the way MT is applied in a commercial scenario is, mostly, to lower prices and this might result in translators losing a percentage of work. There is much to say about the current economic system, but let’s remember that rates have not increased for freelance translators or even LSPs in about two decades, and that is really shocking. Lowering rates is not something brought by MT, but by financial interests. However, I see, as I said before, discounts that are not realistic for the quality of the output provided. That is not to say that MT is not useful. This also happens with low quality TMs or with TMs that do not have their terminology updated, for example, and translators also complain about the rate of fuzzy matches (very little studies on TM and productivity and quality, by the way).
Also, I think there is an “idea” of the dedicated translator that produces “perfect” or nearly perfect translations, always checks terminology, does not make grammar or spelling mistakes, and researches on to the actual meaning of the source text. But this “ideal” translator is not every translator, in the same way that not all MT output is good. Sometimes, MT helps certain translators a lot, more than they will avail.
Another point that you mentioned is how the work done from translators is leveraged by MT and this is certainly true, but not only translators, but humans. MT is very human, not only because of translators but also because “human” data is used. You mentioned that you won’t use MT or participate in MT, and that is fine, but for example, there are plenty of users of “public” and free MT that find it very useful and they do not mind “feeding” the system with data because they are also benefiting from it. I do think, and Andy Way is probably much more qualified to say this, that computational linguists have made incredible (truly incredible) progress in how to approach MT and the results are astonishing. I also see that if we (translators) participate in research and collaborate with technology, the result will be better for all. Sharing information and knowledge is a key aspect of MT.
Finally, in a way, the future is not ours, this sounds very philosophical but, let me clarify, younger generations will have a different approach to MT, its application and quality, and especially on how they want to deal with textual realities. So, it is difficult to judge MT from our perspective. We are seeing younger generations of translators that do not even engage in this type of debates because they take MT as a “natural” tool.
CAT tools can be a bad idea as they impose a sentence for sentence approach to translation where paragraphs work better but for most large technical translation accounts where products are updated incrementally they do save money for translation buyers. @Isabelle SDL and Trados were quite right to tell other LSPs that did technical translation they would go out of business if they did not use CAT tools. Any that didn’t did go out of business or were acquired for a song in the 90’s. I was there. I saw this.
You may feel that CAT tools contribute to a race to the bottom but I think you have to be careful about overgeneralising based on your own client profile. Note, as a translator you will know what is best for your own direct clients so I am not suggesting you change your habits.
Ana’s summary completely tallies with our own findings on post-editing. Many millennials seem to have no problem with MT. Perhaps this is because it saves them terminology research time or because they were not taught to touch type but I suspect it is because like iPads and Facebook they just take it for granted.
My advice to other freelance translators on post-editing is:
1) Always keep an open mind about MT for each new project. If the MT is bad you can always stop working on on the project. Sooner or later you may see good MT. It is out there. I have seen it. It doesn’t bite. Look at it as an opportunity to practise learning a new skill. You might enjoy it. Many including myself do, so long as it is not all day every day.
2) If you do see good MT give yourself a bit of time to get used to the new mode or working and keep a good record of what you are earning per hour. It should always be more than if you were not using MT as you are now working in a niche. Many (and I suspect most) translators are not productive post-editors. Everyone has their own HT/MT ratio depending on quality expectations so find yours.
3) Unless you really like post-editing or are really fast at post-editing, avoid UGC if you are an experienced translator. You will be competing with translators who are not as good as you. The same applies for any work on the croudsourcing spectrum, e.g. working for clients with a long tail disposable client base. A good indicator is heavy use of Google ad words. I am thinking of an agency with 27k clients with a PM staff of 10.
4) I suspect, one strategy to preserve sanity is to mix high volume account work with more interesting material that has to be translated manually.
Reputable agencies will normally have agreed with a client that content will be post-edited. For certain content types like User Generated Content or internal engineering manuals this style concession is often acceptable. For example, I spent a number of years working for Siemens and much of our documentation was written in English by my German colleagues. I could still use it so style is not always important. Quality is a slippery concept.
Finally, be aware of other technologies like autocomplete and Automatic Speech Recognition. MT is a technology and post-editing, like touch typing, is a skill. Boycotting technology is rarely a good strategy and it is not something I would recommend to any young translator. My only caveat here is that I accept there are risks that we have yet to quantify in terms of how over-exposure to MT could impact on how a young translator develops a personal style of writing. However, I don’t know of any young translator that only post-edits so I think we need to be pragmatic about this.
I would say for technical translation invest time learning how to make technology work for you. Learn how to leverage auto-complete, buy a good microphone for Dragon Dictate, know where to get your own MT (e.g. MS Translator Hub). Develop a network of subject matter experts on LinkedIn.
The trick is to mix good productivity with good self-marketing and sales (which you can learn from people like Chris Durban).
I am quite certain that for regular high volume account work for normal quality technical and commercial translation the average per day throughput will be at least 4000 words per day by 2020. After leverage, the current average is 2,500 for companies that use in-house translators.