March 7, 2016 by Daniel P. Clark

Make the Impossible Possible

[Don’t Listen to the Naysayers.] How many times have you heard that you can’t do this and you can’t do that and it’s never been done before? I love it when someone says that no one has ever done this before, because then when I do it that means that I’m the first one that has done it. So pay no attention to the people that say it can’t be done.

– Arnold Schwarzenegger

Recently I was digging around through online documentation trying to figure out how to add private repository gems to my Rails site and have it work in production.  I found bits and pieces of information here and there and the process took a lot more work than it should have.  In the process I happened to overlook one of the steps in the procedure and I ended up with a problem.  The problem was using special characters in an environment variable name which works on Linux but not through the Heroku environment setting tools.  Let me repeat that I overlooked a step in the process which is why this ended up being an issue for me.

Basically I needed to use a URL as part of the environment variable name to permit credentials to a Github private repository.  I could get it to work on my development environment but not on Heroku without publish sensitive data to my repository.

I opened an issue on the Bundler features Github repository and discussed the issue with a bunch of other developers.  I also opened an issue with Heroku to ask for support of special characters in environment variables.  The issue seemed to be moving forward and an update somewhere looked to be what was going to be needed.

Now what is considered appropriate characters for an environment variable name are only alphanumeric characters and an underscore.  So the “problem” was to figure out how in the world we can change any valid URL into an environment friendly variable name; both to encode and decode it.  So I thought of this as a really fun challenge and so I wrote a gem to do it.

The Gem env_compat

Now in my thought process on how to do this I came to a conclusion that the underscore should be used as a marker to show when “code” blocks begin and end.  In other words, as you progress through the string and come upon an underscore you know to toggle on the flag that the following is special character encodings and when you come across the next underscore you know to turn the flag off for them and read the following characters as normal characters.  With this toggle code block feature you can reuse all alphanumeric characters to represent special symbols.  Excluding the mapping for the underscore character you have 35 characters to map to special characters.

So I’ve written the gem in under 80 lines of Ruby code and shared it with the people on the issue thread.  I gave an example of it’s output “GITHUB_D_COM_F_COMPANY_F_” and then I was told the following:

How do you now that _F_ isn’t a part of the URL though? You can’t.

Here I’m being told that it’s impossible to do.  I was told this without the person giving my code any consideration and seeing how it was implemented.  In fact the code already works just fine in this scenario.  With the underscore mapped to the character U that part of a URL would be encoded to _U_F_U_ .  The first underscore toggles on the code block flag and the character U gets decoded into an underscore, the next character being an underscore turns off the code block and the F gets read as a normal character.  Same decoding occurs on the next _U_ code block.

Bundler uses a special tag at the beginning of their environment variables BUNDLE_ .  Using a tag like this it would be very simple to know whether to encode or decode.  For example, if the tag BUNDLE_ is present than you can assume to decode the string, if it is not present then you encode it.  Here are the provided methods in the gem:



The gem allows you to define whichever character mappings you would like.  You are not restricted to the current implementation.

In Closing

After days of this issue some one pointed out to me something I had overlooked and I realized my mistake.  Bundler already has an implementation for private repository credentials as a dependency and I did not need to use any special characters to make it work.  So from my misunderstanding I ended up on a journey where I made something useful and did what was deemed impossible by some people.  When I had found out my oversight I felt quite foolish for it.  But in truth a lot of good has come from this.  The Bundler team’s agenda now includes adding official documentation on their site for private repository credential usage within projects.

The point of this is that there are many things we may think are impossible, or we are simply unaware of their possibility, but where problems arise – there lies opportunity.  Misunderstanding and mistakes can lead to ideas, inventions, and creativity.  You find jewels and gems by digging through dirt and earth.  Some of the best things you’ll come to, discover, are things you don’t search for.  Here’s an awesome talk which demonstrates this point: Kenneth Stanley: Why Greatness Cannot Be Planned: The Myth of the Objective

So I encourage you all to explore, experiment, be creative and stumble upon great things.  As always I hope you’ve enjoyed this!  Please comment, share, subscribe to my RSS Feed, and follow me on twitter @6ftdan!

God Bless!
-Daniel P. Clark

Image by Jeff Wallace via the Creative Commons Attribution-NonCommercial 2.0 Generic License.