If you're a Ruby developer you are probably familiar with the website rubygems.org. It's "the" repository for all information about Ruby gems.
Things i find important when looking for a gem:
Number of total downloads
Release date of the last versions
This will show you how frequently is the gem updated, which is useful to know how fast are fixes and new features introduced.
Sometimes it can happen that a really good gem hasn't had a release in a while. For example a couple of weeks ago i was looking for a gem that wrapped the Marvel Comics API. If you look for gems for that you find that most are really old. Doesn't mean they don't work properly, it just means that the API hasn't changed since and they are still relevant. When that's the case, i usually go for the one that was released last, just because the author had the gift of hindsight over the other ones and might have done a better, more polished job at it. Not always true, but... :)
Dependencies and License
To a lesser degree, i also take a look at these.
The less dependencies a gem has the lighter your code will be. Again, not always true, but it's a good rule of thumb. Also pay attention if the license is compatible with your own project.
If a particular gem feels like a good candidate, I then visit its...
There, i usually check for a couple of things:
This is how you learn how to use the gem, so it helps if the docs are really good (and maintained!).
If it's an opensource project (which most gems are), then i also look for:
Last commit date
A clear indicator of the health of the gem. Also, take into consideration what i wrote earlier about gems that don't update often: it might be because they just don't need to.
Number of contributors, commenters, discussions
They show how active the community is around this gem. If big, it helps later when trying to find information how to do things if you get stuck, for example.
Number of open issues (and resolved)
If you take a glance at the last issues, when they were created, when they were replied to or resolved, this will also give you a good idea of how active the developers are, and how fast can you expect a response if you report something.
Looking at the source code, though not as important as the rest, can also provide some insight to how performant, optimized the gem is at the job it does. It might also help you figure out if will be easy to monkey patch something to fit your needs later. And who knows, you mighty also learn something new from it :)
Though not (by any means) a complete guide for choosing a gem, this might be a good starting point. Also, some of these "metrics" might apply for choosing other software and code libraries.
Like most things in life, your mileage may vary.