Pages in topic:   [1 2] >
Suggestions for changing Blue Board rating system
Thread poster: Robert Forstag
Robert Forstag  Identity Verified
United States
Local time: 20:58
Member (2003)
Spanish to English
+ ...
Nov 16, 2008

The Blue Board as it stands is an indispensible resource for freelance translators. It is the first place that I turn to when I am offered work by an outsourcer that I am unfamiliar with. The more positive comments and the higher LWA ratings I see, the less squeamish I feel about the prospective collaborative relationship: especially when the outsourcer is based several thousand miles away and a major project is involved.

Yet there are, as I see it, two drawbacks to the Blue Board as it is now constituted:

1. It primarily addresses the issue of whether the rating freelancer has been paid on time (of critical importance, to be sure) and, secondarily, whether the outsourcer is "pleasant to work with" (not irrelevant, but not of the highest importance--as long as there is not pervasive evidence of downright *unpleasant* treatment).

This leaves unaddressed at least a couple of other relevant factors, such as the freelancer's satisfaction with the rate of pay offered and the level of support he or she receives from the agency/company/individual outsourcer (e.g., proofreading, feedback, assistance with terms, interest and appreciation by the outsourcer for feedback offered by freelancer regarding problems with the source document, etc.).

2. The fact that the Blue Board is not anonymous seems to have resulted in many translators posting very high ratings as part of what they seem to hope is a *quid pro quo* for more work. Thus, ratings are typically either very high (very often, it seems, submitted by freelancers hoping for more work) or very low (submitted by freelancers who have absolutely no interest in working with the party they are reviewing). There is very little middle ground, and I'm quite confident that a statistical analysis of Blue Board ratings would bear out this impression.

My suggestions for remedying the above deficiencies are as follows:

1.)
Enable freelancers to rate agencies along three dimensions: timeliness of payment, satisfaction with rates, and level of support/cooperation. This last variable could be divided into "pleasantness" and "substantial support offered."

2.)
Make the rating process anonymous, showing only the country of the freelancer.

3.)
Do not allow the posting of comments in addition to the ratings. I would submit that the comments tend to fall into two categories, neither of which are particularly helpful:

i. "Bill is a great guy." / "Diana is wonderful to work with." (nice to know, but irrelevant if they offer an appallingly substandard rate of pay).
ii. "It took me ten months to get paid for work I did, and even then they didn't get the check right." (often accompanied by a reply from the outsourcer alleging that the translator did inferior work, didn't meet the deadline, didn't submit an invoice, etc.; and who really knows the truth of what happened).

Obviously there would be challenges involved in modifying the system. What I am interested in right now is launching a discussion about the merits of what I've suggested here. I feel strongly that implementing my suggestions would vastly increase the utility of the Blue Board for both freelancers and outsourcers by differentiating companies that pay well and/or provide substantial support from those that simply pay on time (at least with respect to a given market and language pair).

I'd like to hear what others think.


Direct link Reply with quote
 

Ralf Lemster  Identity Verified
Germany
Local time: 02:58
English to German
+ ...
The Blue Board is *not* a payment practices list Nov 16, 2008

Hi Robert,
Thanks for your suggestions. One correction, if I may:

1. It primarily addresses the issue of whether the rating freelancer has been paid on time (of critical importance, to be sure) and, secondarily, whether the outsourcer is "pleasant to work with" (not irrelevant, but not of the highest importance--as long as there is not pervasive evidence of downright *unpleasant* treatment).

That's a common misconception - please refer to the BB FAQ:

What is the Blue Board?

We'll explain in a roundabout way. First of all, any ProZ.com user can enter a number from 1 to 5 corresponding to his or her likelihood of working again (LWA) with a given outsourcer (ie. client or translation company/agency). By way of explanation, the user can also enter a line of text along with the number. In response, the outsourcer may enter a line of text of his/her/its own.

Best regards,
Ralf


Direct link Reply with quote
 

Yamila Sosa  Identity Verified
Argentina
Local time: 22:58
English to Spanish
+ ...
I prefer knowing who made a Blue Board entry Nov 16, 2008

Making the Blue Board rating system anonymous would pose some disadvantages, such as the possibility of making false entries against outsourcers. In this case, how the agency would defend against false entries? I prefer knowing who makes a Blue Board entry.

Regards,
Yamila


Direct link Reply with quote
 

Tina Vonhof  Identity Verified
Canada
Local time: 18:58
Member (2006)
Dutch to English
+ ...
Some good suggestions Nov 16, 2008

Keeping in mind that the Blue Board rating, as Ralph points out, is an indication of how willing you are to work with this outsourcer again, I think the addition of a few ratings as Robert suggests under point 1) would be helpful. Currently, if someone provides a WWA rating of 3 or 4 without making any written comments, you have no idea what, if any, problems they may have had. I sometimes give a high WWA rating because the outsourcer was great to work with, paid promptly etc., even though I may think their rates are a bit low. Being able to qualify your WWA rating would provide a more complete picture - at least as long as people are honest but that is what the whole system depends on anyway.

I don't like Robert's suggestion 2). When I look on the Blue Board, I want to know who made the entry - do I know/trust this person? What language pair?

Nor do I like suggestion 3). I think people's comments are helpful - do they emphasize the good relationship or do they emphasize prompt payment? That at least gives you some idea what their WWA rating is based on. All these things play a role in my impression of the outsourcer.


Direct link Reply with quote
 
Amy Duncan  Identity Verified
Brazil
Local time: 23:58
Portuguese to English
+ ...
The support factor Nov 16, 2008

I had a situation not long ago where I received a job offer from a translation company that had high Blue Board ratings. My experience with them was a nightmare, because of the PM who was assigned to my job. this person was confused, sloppy, inconsistent, non-communicative and dishonest. I had to fight for one of my payments because of errors this person made. So, I agree with Bob's suggestion about support. A company may be very good for the most part, but if there's one rotten apple, it spoils the whole bunch.

Direct link Reply with quote
 

Paul Kozelka  Identity Verified
Local time: 02:58
Member (2004)
French to English
+ ...
Work a bit more analysis Nov 16, 2008

I tend to agree with Robert's general principles, though doing away with text lines for either/both parties and making entries anonymous would likely lead to more abuse/exaggeration rather than less. Given that the BB is clearly a critical tool to most of us for checking bona fides of whatever sort, it might be worthwhile for someone at ProZ (or a volunteer translator??) to dig a bit deeper than maybe Robert has gone, in terms of determining if the BB Rules and actual entries are indeed producing the sorts of useful info expected. I do agree that there seem to be few mid-level ratings, and admit to the general cowardice of not wanting to offend good clients with anything less than a 5, so yes, maybe a few breakouts such as Robert's 3 categories would be in order.

Direct link Reply with quote
 

Marek Buchtel  Identity Verified
Czech Republic
Local time: 02:58
Member (2005)
English to Czech
+ ...
Satisfaction with rates is irrelevant Nov 16, 2008

If the translator has negotiated rates he or she is not satisfied with, it's the fault of the translator, not of the client/agency. It merely says "the agency's negotiating skills are better than mine".
Not much relevant.

I know translators, who charge rates I wouldn't be satisfied with. And on the other hand, I know people who charge much more than I do, and would never be satisfied with my rates.

Marek


Direct link Reply with quote
 

Ralf Lemster  Identity Verified
Germany
Local time: 02:58
English to German
+ ...
Pricing may be a factor determining LWA Nov 16, 2008

Hi Marek,

If the translator has negotiated rates he or she is not satisfied with, it's the fault of the translator, not of the client/agency. It merely says "the agency's negotiating skills are better than mine".
Not much relevant.

I beg to differ - the Blue Board is based on the likelihood of working again for a given outsourcer. The pricing agreed upon may (or may not) be a factor determining LWA, but it is certainly relevant.

Best regards,
Ralf


Direct link Reply with quote
 

Marek Buchtel  Identity Verified
Czech Republic
Local time: 02:58
Member (2005)
English to Czech
+ ...
Agreed price Nov 16, 2008

Hello Ralf,


I beg to differ - the Blue Board is based on the likelihood of working again for a given outsourcer. The pricing agreed upon may (or may not) be a factor determining LWA, but it is certainly relevant.


Yes, you are right, I did not think about it in this way. Pricing is important for LWA.

But I don't think "satisfaction with rates" should be one of the categories, as Robert suggests. For the reasons I described before.

Maybe it could be modified into a question like:
"In my opinion, the agency's prices are:
- below market average
- equal to market average
- above market average"

But that's probably outside the concept of "likelihood of working again"

Regards

Marek

[Upraveno: 2008-11-16 23:48 GMT]


Direct link Reply with quote
 

Ralf Lemster  Identity Verified
Germany
Local time: 02:58
English to German
+ ...
Part of LWA Nov 17, 2008

Hi again, Marek,

But that's probably outside the concept of "likelihood of working again"

Personally, I don't think it is.

I like the concept of incorporating a limited number of 'sub-scores' into the overall LWA level: besides enhanced transparency, this would also reflect the various factors involved.

Best,
Ralf


Direct link Reply with quote
 

Marie-Hélène Hayles  Identity Verified
Local time: 02:58
Italian to English
+ ...
In favour of a more detailed system, totally against anonymity Nov 17, 2008

This and other issues were also discussed last week, in this thread:

http://www.proz.com/forum/prozcom_job_systems/120176-we_need_a_better_or_more_honest_blue_board_system.html


Direct link Reply with quote
 

José Henrique Lamensdorf  Identity Verified
Brazil
Local time: 23:58
English to Portuguese
+ ...
I second Marie-Hélène's vote Nov 17, 2008

I gave some more specific ideas recently at http://www.proz.com/forum/prozcom_job_systems/120176-we_need_a_better_or_more_honest_blue_board_system-page2.html#988427

Quite frankly, I think there is too much anonymity there already. Click on some free users' links there and look at their profiles. All too often you'll see no more than a nick. It would be quite easy for a less-than-ethical outsourcer to get one or two dozen friends who would build them a solid-looking façade of a great company, in no time at all.

Nothing against free users, I was one of them for several years, but my profile always had enough data that would inequivocally point to me, and not apply to virtually anyone on the planet.


Direct link Reply with quote
 

Roman Bulkiewicz  Identity Verified
Ukraine
Local time: 03:58
Member (2004)
English to Ukrainian
+ ...
I agree with Marek Nov 17, 2008

Ralf Lemster wrote:
I beg to differ - the Blue Board is based on the likelihood of working again for a given outsourcer. The pricing agreed upon may (or may not) be a factor determining LWA, but it is certainly relevant.


Indeed, it may be the crucial - even the blocking - factor determining this likelihood, in the proper sense. But I agree with Marek that it should not be relevant for the purpose of scoring outsourcers in the BB. Suppose you worked for an agency at a rate you both had agreed upon, and you had great experience in terms of communication, payment etc. -- but since then your rate has gone up so much that the agency cannot possibly afford it any more. You are sure that the likelihood of you working again for that agency is, strictly speaking, zero. Would it be just if you put the lowest LWA score for that agency? More importantly, would it be useful for your fellow freelancers who may have different rates, and who are checking the BB to decide whether it is worthwhile even to start talking to this agency about rates?

Leaving alone rates, suppose you've been happy with a client (rates, payment etc.) but something happened that makes your further collaboration objectively impossible -- they decided to specialize in a subject area you don't work in, or you moved to another country, and international transactions are too costly, whatever -- again, the likelihood is zero, but would it be proper to grade this client with anything lower than 5 because of that?

I would say the term LWA itself is not precise, and may be confusing. Maybe it ought to be clarified in more detail in the manual/help. But, if we take it literally, it could lead to absurd results and make the BB useless.


Direct link Reply with quote
 

Roman Bulkiewicz  Identity Verified
Ukraine
Local time: 03:58
Member (2004)
English to Ukrainian
+ ...
some comments to the original suggestions Nov 17, 2008

Robert Forstag wrote:
This leaves unaddressed at least a couple of other relevant factors, such as the freelancer's satisfaction with the rate of pay offered


Well, I did see comments like "great to work with, but rates are low" - and LWA=4 or 3 (presumably lowered for that very reason). Personally, I find it... to say the least, strange -- agreeing on a rate and then leaving comments like this (or, worse, lowering LWA for this reason without mentioning it). Why did you take the job if it was too cheap for you?


and the level of support he or she receives from the agency/company/individual outsourcer (e.g., proofreading, feedback, assistance with terms, interest and appreciation by the outsourcer for feedback offered by freelancer regarding problems with the source document, etc.).


I don't think so. I take these factors into account when I score, and I've seen these aspects commented alongside other people's ratings.


2. The fact that the Blue Board is not anonymous seems to have resulted in many translators posting very high ratings as part of what they seem to hope is a *quid pro quo* for more work.


Well, assuming you are right -- then, if one posts a very high LWA for an outsourcer because one is so eager to get more work from that outsourcer, it means that one's likelihood (or desire) to work again with that outsourcer is indeed very high, doesn't it?


1.)
Enable freelancers to rate agencies along three dimensions: timeliness of payment, satisfaction with rates, and level of support/cooperation. This last variable could be divided into "pleasantness" and "substantial support offered."


It would not hurt, but would not make a big difference, IMHO. There is a comment field, and you can inquire the graders personally if you need more details.
The second item (satisfaction with rates), for one, would tell more about the grader than about the graded.


2.)
Make the rating process anonymous, showing only the country of the freelancer.


Absolutely NOT.


3.)
Do not allow the posting of comments in addition to the ratings.


What will you ahieve by doing this?


I would submit that the comments tend to fall into two categories, neither of which are particularly helpful:


I would say: 1. Leave helpful comments. 2. Use your judgement when reading other people's comments.


Direct link Reply with quote
 
Robert Forstag  Identity Verified
United States
Local time: 20:58
Member (2003)
Spanish to English
+ ...
TOPIC STARTER
The Blue Board as it stands now produces positively skewed ratings Nov 17, 2008

The reason that this is so is, as Paul Kozelka points out, the only time freelancers tend to give a company low ratings is when they have either made up their minds that they do not want to work with the entity they are rating, or when they simply don't care if they work with them again.

Right now, the main usefulness of the Blue Board lies in its enabling those who consult it to determine the risk factor of non-payment or slow payment for a prospective client. (Given the limitations I noted above, if a client gets low ratings and negative comments, then this is something to pay very close attention to indeed!) Other than serving this admittedly highly important purpose, the directory seems mainly to provide a venue for a love-in among the listed entities and the freelance raters who are eager to work with them again (and I am well aware that there are exceptions to this rule, as Roman has pointed out; but the general tendency--and the structural problems enabling this tendency--should be clear to all).

I personally find comments such as "so-and-so is nice and/or professional" to be of limited value. I would expect collaborators to be "nice and professional", and would only find it noteworthy when they are not. Far more relevant are issues such as timeliness of payment, equitability of rates offered with respect to the market I function in, and whether substantial support is offered.

In response to a number of comments regarding why rates are an issue if the freelancer and agency have agreed to terms, I would say that there are many occasions when I have taken jobs for rates that I wasn't happy with. After all, I have to put food on the table and pay my bills. I'm sure many other freelancers have found themselves in similar predicaments. It is part of the real world in which we live.

I would say that a more meaningful rating system would not only provide more useful information for freelancers, but would keep outsourcers more accountable in terms of their own professional practices. I see anonymity as vital to this process: without it, much negative and potentially helpful criticism will continue to be suppressed, for reasons I cited at the beginning of this post. I challenge those who do not support anonymity to show how the system can really work without it.

Finally, a reformed Blue Board would enable those outsourcers that do offer better rates to stand out from those that do not, and therefore should help them secure the services of the most able and accomplished translators.



[Edited at 2008-11-17 16:00 GMT]


Direct link Reply with quote
 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Suggestions for changing Blue Board rating system

Advanced search






PerfectIt consistency checker
Faster Checking, Greater Accuracy

PerfectIt helps deliver error-free documents. It improves consistency, ensures quality and helps to enforce style guides. It’s a powerful tool for pro users, and comes with the assurance of a 30-day money back guarantee.

More info »
CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use SDL Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search