Please Read: About Editors' Ratings

88 posts (showing 1-20)

FGL AdminCommunity Level 13
3719 posts

It's an open secret that we grade almost every game when we approve it for sponsor viewing -- in fact, we've always given games an Editors' Rating since the day we opened our doors. As developers ourselves, this is a heavy responsibility, and not one we really enjoy -- stones and glass houses and all that. But it's become a crucial part of making sure that games find the right sponsors.

This is why the process of approving games takes a while. We have to play your game for a while to get a feeling of its sales-worthiness before we approve it. Our rating scale is from 1 to 10, but the internal mechanisms lump them into a few groups: unsellable, poor, mediocre, good, great, amazing. While we're grading the game, we also take note of particular characteristics that may be interesting to certain sponsors. (We hand-write letters to many of our sponsors each week, telling them about games we think they'll like.) In addition, these ratings affect how your games show up in some of the FGL sponsor features.

The editorial rating system is not intended to judge the value of one game over another. It is often the case that an "8"-rated game sells for much more than a "9" or vice versa... it's really just a way to help cluster games into a few quality areas. We let the sponsors decide how much each game is actually worth to them. Single-point rating differences don't matter much.

It's also not a perfect system, and we recognize that. We often change the score of a game after we notice a developer has made dramatic changes to the game. The Developer Ratings and First Impressions Ratings system also help detect incongruities, and give sponsors a second or third opinion about the quality of the game. (These ratings are shown to sponsors in the same way that Editors' ratings are.)

We've mentioned these ratings numerous times in the past, but we've never made a big deal out of the editors' rating system until now because it's just one of many determining factors in getting your game sold. We've had very-low-rated games sell for thousands of dollars, and occasionally our top-rated games never sell. It's not a magic bullet. If your game thumbnail is awesome and your text description is alluring, you will definitely get more sponsor views regardless of your game rating.

However, as we've grown, we've found that sponsors rely on our ratings more and more. While there are a great deal more active sponsors now than ever before, they are also extremely busy people and just don't have the time to view every game in FGL's ever-growing list.

This creates an interesting statistic: the average number of sponsor-views per game has gone way up over the past year, but the views are not at all evenly distributed across all games. Sponsors use every shortcut they can to try to cut through the noise to find the best games. Our editors' rating system is one of the shortcuts they use. (For instance, some sponsors customize their RSS feeds to list all games of a certain quality or higher. Others rely on FGL's automated daily emails, which prioritize higher-rated games first, so they always click on the first few games each day. Others routinely check the Editors' Choice section of the site, where top-rated games are shown.)

Since our rating system has slowly become very important to your sales potential, we want to make it clearer what to do to get a high score in our rating system.

(If you're skimming this post, HERE'S WHAT YOU NEED TO KNOW!)

  • Only submit completed games for sponsor bidding approval. We rate them when we approve your game, so if it's not finished, it's obviously not going to get a high score. (This is why we've recently been turning down games that aren't completed, telling you to finish your game first.) It's not just about our score -- most sponsors only look at a game one time, so you only get one chance to impress them. You've got to make it count!
  • Invest in graphics. The quality of Flash games has gone up too high to allow "programmer art" to earn top dollar anymore, and our ratings reflect this. Having sub-standard art is the fastest way to turn a "Great" game into a "Mediocre" game.
  • The first 90 seconds of gameplay are key. We play for longer than 90 seconds before grading your game, of course, but we weigh those first 90 seconds most heavily. This is because sponsors do, too, and so do most players. If it doesn't wow them right off the bat, they won't stick around to see how amazing it gets on level 12. Don't forget sound, music, instructions, and an impressive introductory screen and menus.
  • Get feedback BEFORE you submit it for bidding approval. Upload your game and get developer feedback first, and take that feedback seriously. Invest in a few dozen First Impressions to really make sure you are giving off the right vibe during the first few minutes of gameplay. You may have heard the old adage that "the last 10% of the game takes 90% of the time" and that's close to true. Expect to spend at least as much time polishing the game as you did making the engine itself.

Please also keep in mind that we want you to sell your game. That's how we get commissions. And we hate to have to give out low scores. If you submit your game for bidding and a reviewer contacts you and suggests that you to change some things first, you can read between the lines: the details they're pointing out are probably the difference between a "Good" game and a "Great" game. Please take their advice seriously.

We are always experimenting with other ways to help sponsors find games, including:
  • The Games Spotlight system remains a key way to get games noticed. This system is completely independent of any other rating system -- if you impress one of the reviewers, they will flag it for a run in the spotlight. Think of it like the "my favorite books" section at a Borders.
  • Recently we've been experimenting with using gameplay videos when we shop your games around to sponsors. Here's the prototype version, which has already had good success in getting new bids. The beta version is open to anyone who is Market Level 1 or higher; feel free to contact FGL_Steve to include your video in the next run. This is a free service right now, though it may incur a modest fee (based on your market level) when it is completed.
  • We will also be inviting some developers to participate in a brand new experimental display system soon -- one that has a chance to dramatically boost sponsor views. This system will not use editors' ratings at all, but rather screen shots from your games.
  • We also have improved keyword-based search tools coming soon, to help sponsors find seasonal and specialty games.
  • One of the most important ways sponsors find your games is when we manually poke them and give them a heads-up about games that especially fit their particular portal. This has less to do with ratings than it does with fitting a portal's particular niche.

And several other systems are on the horizon. Editors' Ratings are hardly the sole determinant of your games' success -- in time, we want the editor rating to become less and less important. However, if you want to sell your game today, then the editor rating is important.

To repeat: this isn't a new policy; it's something we've done since we started FlashGameLicense. But it's very slowly become an important determiner in how many sponsor views you get, especially during the first few weeks the game is for sale. Please help us to help you -- only submit games of the highest quality you can muster, and it will almost always pay off in spades.

posted 2009-07-29T00:20:12-07:00 | edited 2009-07-29T12:01:54-07:00

FGL AdminCommunity Level 13
3719 posts

How to see your game's rating: you can see your game's Editors' Rating by viewing the game. This information will be near the bottom of the page. If your game has at least 10 Developer Ratings, you will see that, too. This is a new feature for developers; previously this information was not clearly shown on the site. Note that developers can only see their own games' ratings, not the ratings of other games.

What to do if you think your game is incorrectly rated: If you have solid evidence that your game has been incorrectly rated (or that your game has improved dramatically), please let us know. The easiest way to show us discrepancies is with your Developer Rating or First Impressions rating.

If your game's Developer Rating is two points or more higher than your Editor Rating, please contact FGL_Steve and he will arrange a re-review. This may or may not result in a new grade, but we will seriously consider it.

Alternatively, if your public First Impression average is two points or more higher than your Editor Rating, you can also contact FGL_Steve for a re-review. You can find your average First Impression score by looking at the numbers at the top of your public First Impression folder. You will need at least 10 First Impressions in a folder in order to make it public. (And you need to make the First Impressions folder public before it counts.) The average score is the average of all the sub-scores listed at the top of the folder-view page.

If you have made substantial changes to your game (such as a graphics upgrade) and it has been at least one month since the game started being for sale, you can also contact us for a re-review.

posted 2009-07-29T00:25:54-07:00 | edited 2009-07-29T00:50:57-07:00

FGL AdminCommunity Level 13
3719 posts

How Editors Ratings Are Determined: Games are rated by our review team. Currently the review team consists of seven people. All games have at least one reviewer, and most have two or three. Each reviewer's score is averaged into the final value. All reviewers' opinions have equal weight. We have a checklist of features that we look for (which exactly match the five categories checked for by First Impressions). But beyond the checklist, reviewers are also encouraged to use their own intuition when they see a game that has lots of hidden potential.

How Developer Ratings Are Determined: Once a game is up for bid (or for sale in the Game Shop), developers can also grade your game. (Assuming you make the game visible to developers. Otherwise, they can't.) You need at least ten different developers to rate your game before the Developer Rating is displayed. Each developer's vote is weighted heavily by their Community Rating and Market Rating. Developers can also change their vote at any time. Votes are auto-tallied every few hours.

Developers are on the honor system to vote fairly and to use the full spectrum of available scores. However, we do monitor scoring patterns and occasionally we delete developers who abuse the system by rating all games "10" or "1" or something similar.

How First Impressions Ratings Are Determined: First Impressions reviewers are never asked to determine an overall score for your game. Instead, they are asked to rate your game on five different axes: Graphics, Sound, Intuitiveness, Fun-Factor, and Polish Level. These five factors are averaged together to determine the "overall average" score.

About Sponsor Ratings: It is also possible for sponsors to rate your game. If you receive 10 or more sponsor ratings, this will also be displayed to you at the bottom of your game-view page. Sponsor ratings are calculated using the sponsors' market level and community level, just as developers' ratings are. Sponsor ratings are not used for very much at the moment, because very few sponsors have the time to rate games. At this time, it is very rare for a game to have 10 sponsor votes! This is a feature we are encouraging sponsors to take more advantage of.

posted 2009-07-29T00:40:23-07:00 | edited 2009-07-29T00:54:04-07:00

Market Level 1Community Level 16
3696 posts

Nice idea Eric, though I'm kinda think most of these should be for market leaders only ;) .. Such as ability to request re-review and score fix.


EDIT: Just checked my old games. I'm blushed of the scores given :) .

posted 2009-07-29T01:14:24-07:00 | edited 2009-07-29T01:16:38-07:00

FGL AdminCommunity Level 10
1457 posts

Awesome post Eric, I hope everyone reads this post! Remember if you have any questions about anything don't hesitate to PM us and ask.

posted 2009-07-29T01:22:15-07:00 | edited 2009-07-29T01:24:08-07:00

FGL AdminCommunity Level 6
782 posts

Be sure to check out the trailer page.  You might have missed the link while reading through all the other great information.

If you are >= market level 1 and have made a trailer be sure to send a PM to Steve about it.

posted 2009-07-29T01:34:46-07:00 | edited 2009-07-29T01:37:11-07:00

Market Level 4Community Level 0
3406 posts

Very nice read, I definitely see the thought and effort put into all of this. I think a summarized version of this should be a mandatory reading for all who sign up to FGL, the completed game idea really needs to be put into peoples heads more.

posted 2009-07-29T01:40:20-07:00

FGL AdminCommunity Level 6
782 posts

Re: RoseDragon - Market Level

We have plenty in mind to enhance the value of the market level.  The new experimental system Eric alludes to will be heavily influenced by the users market level.

posted 2009-07-29T01:47:53-07:00

FGL AdminCommunity Level 10
1457 posts

Re: Porter

Yep we've planned to include a summarized version of this for newcomers who are signing up to the site.

posted 2009-07-29T01:48:51-07:00

Market Level 1Community Level 16
3696 posts

Thank you FGL. :)

Hmm, appareantlly I found one of my game doesn't have editor rating... is it eaten by something?

posted 2009-07-29T01:53:18-07:00 | edited 2009-07-29T01:54:00-07:00

FGL AdminCommunity Level 13
3719 posts

rosedragon - the most common reason is that the game hasn't been up for bidding yet. We only rate them when they're submitted for bid. That way you can work on them and get feedback first. If that's not it, PM me the game's name and I'll take a look.

posted 2009-07-29T01:56:59-07:00 | edited 2009-07-29T01:57:15-07:00

Market Level 0Community Level 7
974 posts

Wow awesome post Eric. That definitely cleared things up and makes me think twice before submitting sub-par or unfinished games.

posted 2009-07-29T02:08:42-07:00

Market Level 0Community Level 3
233 posts


Our rating scale is from 1 to 10, but the internal mechanisms lump them into a few groups: unsellable, poor, mediocre, good, great, amazing. 

So assuming a rating of 10 is gonna’ drop a game into the “amazing” category and a 1 - 2 will probably equal unsellable … where do the other numerical grades drop your game? I would assume a grade of 5 - 6 would equal mediocre; 3 - 4 being poor, 7 being good, 8 – 9 being great?

Am I correct … or waaaaaaay off … or have I just completely misunderstood this whole system? :p


posted 2009-07-29T02:26:46-07:00 | edited 2009-07-29T02:28:28-07:00

Market Level 3Community Level 13
3515 posts

That must be the longest post in the history of forums....

Quite amazing... you guys always amaze me with these features... I am quite surprised with my ratings. Only one bad rating, and that game is long gone now. Now that FGL is so amazing and makes selling games so much easier, all my games will be on here, just need to make more of them.

posted 2009-07-29T02:33:07-07:00

Market Level 2Community Level 3
61 posts

Thank you for this new game is almost 2 points higher on developer ratings than sponsor ratings, but not quite! 7 is quite respectable though ;)


Plus I just put it on last call, so a re-review would be pointless at this point!


Great feature, thanks!


EDIT: Well at least until Daniel Sidhion lowballed my game just now ;p

posted 2009-07-29T03:06:18-07:00 | edited 2009-07-29T03:19:25-07:00

Market Level 4Community Level 0
3406 posts

@Rocketman, second longest, I once wrote a review for a super hero based game on this site (don't remember the name, it was removed) that was about 1.5x that post, I don't know what I was on that night.

Secondly, you may want to rewrite part of the above article to specify that the editor rating is only appointed upon submitting your game to be put up for bid, I recall it being a bit unlcrear during my read through. This bit here,

"Only submit completed games for approval."

Perhaps change it to,

"Only submit completed games for bidding approval".

Just something to specify that it's not putting your game on the site in general, but for putting your game up for bid.

posted 2009-07-29T04:14:39-07:00

Market Level 3Community Level 1
40 posts

I have mixed feelings about the editor ratings...  Since there are only a handful of reviewers, they do not represent the vast taste of flash gamers.  E.g. If you have a hardcore flash games sponsor play a really easy game rated a 10, they might sponsor a game they thought was good because they trust the editor rating.  I feel that you guys need to put the editor rating more into context.  Make it clear to the sponsors what the rating means besides just words like poor, mediocre, good, and etc.  E.g.  It's a poor game from a puzzle-loving casual flash gamer's perspective.  This is not a good example, but you guys get the idea.

I'm also concerned about how this will affect my (and others) contributions to FGL via commissions.  If you get an editor rating of 2, you'd definitely think twice about whether to pay the commission after the bid goes through...

posted 2009-07-29T04:53:25-07:00 | edited 2009-07-29T04:53:54-07:00

Market Level 1Community Level 16
3696 posts

I'm also concerned about how this will affect my (and others) contributions to FGL via commissions. If you get an editor rating of 2, you'd definitely think twice about whether to pay the commission after the bid goes through...

haha... agree... we can be that meany :P .


@Eric: It was at bid for a month earlier. Now it just stay at gameshop and I don't think it need to be fixed since I'm done with that one. It was fgl contest's game so probably your reviewers forgot to assign that rating. With the current rating shown, next time I will know if my editor's rating is missing :) .

posted 2009-07-29T08:25:24-07:00

Market Level 6Community Level 10
1449 posts

I was going for my big "bla bla bla" posts, but there's no need.

Basically, this is what I want to read about and addresses so many issues that I really have nothing to add (believe it or not). So thanks to FGL for all the hard work.

posted 2009-07-29T08:43:00-07:00

Market Level 8Community Level 12
3263 posts

Also, unless I mis read something, the new agreement when placing a game up for bidding is no longer volunteer to pay the 10%, it's mandatory. Which is long over due if you ask me. :)

"There is no charge for this service until your game is sold, at which point you will be responsible for paying a 10% commision on the opportunities that this site provided you."

posted 2009-07-29T08:51:51-07:00 | edited 2009-07-29T08:52:45-07:00