If you are using ruby with bundler and Gemfiles properly, you probably know about running commands with bundle exec. However, sometimes this does not get you quite the right results, in particular if your Gemfile is not quite precise enough.
For example, I had an issue with cucumber + autotest + rails where I had both rails 3.1 and rails 3.2 apps using the same RVM. Since I was confused, and in a hurry, I figured one brute force option to unconfuse me would be to simply remove all old versions of all gems from the environment. I did just that, and thought I’d explain the process of incrementally coming up with the right shell script one liner.
First things first, let’s figure out what we have installed:
$ gem ... Usage: ... gem command [arguments...] [options...] ... Examples: ... gem list --local ...
Ok, I guess we want gem list:
$ gem list *** LOCAL GEMS *** actionmailer (3.2.2, 3.1.1) ... ZenTest (4.7.0)
actionmailer is part of rails, and we can see there are two versions installed. Let’s figure out how to remove one of them…
$ gem help commands
GEM commands are:
...
uninstall Uninstall gems from the local repository
...
$ gem uninstall --help
Usage: gem uninstall GEMNAME [GEMNAME ...] [options]
Options:
...
-I, --[no-]ignore-dependencies Ignore dependency requirements while
uninstalling
...
-v, --version VERSION Specify version of gem to uninstall
...
Great. Let’s try it:
$ gem uninstall actionmailer -v 3.1.1 You have requested to uninstall the gem: actionmailer-3.1.1 rails-3.1.1 depends on [actionmailer (= 3.1.1)] If you remove this gems, one or more dependencies will not be met. Continue with Uninstall? [Yn] y Successfully uninstalled actionmailer-3.1.1
Ok, so we need to have it not ask us that question. From studying the command line options, the magic switch is to add -I.
So once we have pinpointed a version to uninstall, our command becomes something like gem uninstall -I $gem_name -v $gem_version. Now we need the list of gems to do this on, so we can run that command a bunch of times.
We’ll now start building our big fancy one-line script. I tend to do this by typing the command, executing it, and then pressing the up arrow to go back in the bash history to re-edit the same command.
Looking at the gem list output again, we can see that any gem with multiple installed versions has a comma in the output, and gems with just one installed version do not. We can use grep to filter the list:
$ gem list | grep ',' actionmailer (3.2.2, 3.1.1) ... sprockets (2.1.2, 2.0.3)
Great. Now we need to extract out of this just the name of the gem and the problematic version. One way of looking at the listing is as a space-seperated set of fields: gemname SPACE (version1, SPACE version2), so we can use cut to pick fields one and three:
$ gem list | grep ',' | cut -d ' ' -f 1,3 ... gherkin 2.5.4) jquery-rails 2.0.1, ...
Wait, why does the jquery-rails line look different?
$ gem list | grep ',' | grep jquery-rails jquery-rails (2.0.2, 2.0.1, 1.0.16)
Ok, so it has 3 versions. Really in this instance we need to pick out fields 3,4,5,… and loop over them, uninstalling all the old versions. But that’s a bit hard to do. The alternative is to just pick out field 3 anyway, and run the same command a few times. The first time will remove jquery-rails 2.0.1, and then the second time the output will become something like
jquery-rails (2.0.2, 1.0.16)
and we will remove jquery-=rails 1.0.16.
We’re almost there, but we still need to get rid of the ( and , in our output.
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' ... childprocess 0.2.2 ... rack 1.3.5
Looking nice and clean.
To run our gem uninstall command, we know we need to prefix the version with -v
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' ... childprocess -v 0.2.2 ...
Ok, so now at the start of the list we want to put gem uninstall -I . We can use the regular expression ‘^’ to match the beginning of the line. We’ll need sed to evaluate our regular expressions…
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -r 's/^/gem uninstall -I/'
sed: illegal option -- r
usage: sed script [-Ealn] [-i extension] [file ...]
sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]
Ugh. -r is the switch used in the GNU version of sed. I’m on Mac OS X which comes with BSD sed, which uses -E.
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /' ... gem uninstall -I childprocess -v 0.2.2 ...
Ok. That looks like its the list of commands that we want to run. Since the next step will be the big one, before we actually run all the commands, let’s check that we can do so safely. A nice trick is to echo out the commands.
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/echo gem uninstall -I /' | sh gem uninstall -I childprocess -v 0.2.2
Ok, so evaluating through sh works. Let’s remove the echo:
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /' | sh ... Successfully uninstalled childprocess-0.2.2 Removing rails Successfully uninstalled rails-3.1.1 ...
I have no idea why that rails gem gets the extra line of output. But it looks like it all went ok. Let’s remove the ‘sh’ again and check:
$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall /' gem uninstall jquery-rails -v 1.0.16
Oh, that’s right, the jquery-rails gem had two versions. Let’s uninstall that, then. We press the up arrow twice to get back the command line that ends with | sh, and run that. Great: we’re all done!
Let’s look at the final command again:
gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /' | sh
gem listshows us the locally installed gems| grep ','limits that output to lines with a comma in it, that is, gems with multiple versions installed| cut -d ' ' -f 1,3splits the remaining lines by spaces, then picks fields one and three| sed 's/,//'removes all,from the output| sed 's/)//'removes all)from the output| sed 's/ / -v /'replaces the (one remaining) space with-v| sed -E 's/^/gem uninstall -I /'putsgem uninstall -Iat the start of every line| shevaluates all the lines in the output as commands
Note that, as a reusable program or installable shell script, this command really isn’t good enough. For example:
- It does not check the exit codes / statuses of the commands ran, instead it just assumes they run successfully
- It assumes the output of
gem listwill always match our expectations (for example, that the output header does not have a comma in it, or that gem names or versions cannot contain space, comma, or-- this may be true but I wouldn't know for sure) - It assumes that
-Iis the only switch needed to preventgem listfrom ever asking us questions
The best approach for a reusable command would probably be to write a script in ruby that used the RubyGems API. However, that would be much more work than writing our one-liner, which is “safe enough” for this use-once approach.
(For the record, this didn’t solve my rails + cucumber woes. Instead, it turns out I had run bundle install --without test at some point, and bundler remembers the --without. So I needed rm Gemfile.lock; bundle install --without ''.)
LSD::RELOAD 