Why you should write your own plugins.

Synopsis: Write it yourself if you can, you’ll probably learn something, and that’s more beneficial for you (you as a developer) in the long-run.

This is going to be a controversial post (for anyone that actually reads it). Gems and plugins are at the heart of Agile dev, specifically Ruby on Rails. There are some great open source Gems on the web that can simplify processes, save you time, and lead to a more stable production application; however, there are many more that you’d be better off doing without. I’m going to argue that you might consider doing without any third-party plugins or Gems. Why?

These are simply my opinions and preferences, YMMV or YMMLV - likely vary.

0. It might actually take you more time, more resources, and sweat to use a plugin, especially when something goes wrong.

I numbered this as 0. because this is probably the number one reason why you might use a Gem: you’re lazy. I don’t mean that in the negative sense. Most likely you’re pressed for time, you have more things to worry about, or your current development schedule might not allow you to delve into the inner workings of a particular task.

Gems have the possibility of getting you up and running right away, but that’s not always the case. There have been too many times when I’ve decided to use a gem and spent a few hours deciphering the documentation, handling compatibility issues, or even deciding which gem of many similar ones to use. There are also multiple branches of the exact same gem maintained by different parties you need to sort through as well.

After it’s all hooked up, you still have other issues to deal with…

1.You learn the plugin, not the underlying technology.

Technology moves fast. New paradigms are introduced every few months and they often improve on existing technology or use existing technology as a springboard for new concepts. You don’t want to be left behind. Plus, if you’re like me, you might want to learn about the technologies you work with so you can make your own contributions down the road or, even more likely, be able to adapt your current knowledge to new scenarios.

When you learn a plugin, you’ve pigeonholed your learning to that particular Gem. You allocate your time to understanding the Gem, its syntax, its gotchas, and you forgo the the knowledge you would have learned doing it yourself. This is a huge opportunity cost.

Furthermore, when that Gem becomes obsolete, your knowledge expires with it.

A good example of this is SQL and ActiveRecord. I’m not saying you shouldn’t use ActiveRecord, you should. It will be maintained and it’s not a “plugin” in my book as I’ll later explain, but it’s a good example in the context of SQL and ActiveRecord’s syntax. When you use ActiveRecord, you learn the .find syntax and that you can pass in criteria with .where, etc. When you use it’s relationships, you know that you can access a user’s post with User.posts. What you don’t learn, necessarily, is SQL. You don’t have to write “SELECT * FROM posts, users WHERE posts.user_id = users.id AND user.id = x”.

What’s worse, is if you blindly use a plugin, you might be writing horrible code. Assuming ActiveRecord didn’t cache results, you might be making dozens upon dozens of database calls on each request.

2. You don’t know the weak points of the plugin. How do you recover when SHTF?

For a mission critical app or an app that gets a lot of hits, it’s likely that your code will at some point operate out of the bounds in which you or a plugin author initially expected. When that occurs, you may need to rescue it or fix it so that it doesn’t happen again. When you write your controllers and design your models, you’ll identify weak points. You might address them then or you might make a mental note of it for later. You might not even realize it at all when you first write it, but when something breaks, your chances of isolating the issue with code you’ve written is a lot higher than if the error occurs in a plugin you didn’t author.

When your plugin fails, you’re going to have to wait for the original developer to fix it or commit yourself to picking apart reams of code you didn’t write and probably don’t understand.

An example of where I personally got burned from this is when I used Mongoid. Mongoid is a great gem, and I’d really consider it more of an ORM than a “plugin”. I was passing it a simple datetime criteria, where I wanted records that had a date greater than x and less than y. It failed to return the correct records on occasion. I spent a few hours trying to figure it out on my end and eventually posted the bug on the github project. Thankfully the author got back to me in a few days, but it turns out that Mongoid rounded precision before passing the query to the MongoDB ruby driver. If he had not chimed in, I likely would have never found out what the issue was. The logs were exactly what I expected and there were no signs of faults or other errors.

3. Plugins often offer more than you need.

Part of the appeal of Gems is rich functionality. You can paste one line of code in your Gemfile and access an entire suite of tools, but do you really need all that bulk and extra complexity? Devise and Restful_Authentication are both (or once were) very popular authentication gems. They offer generators, nice forms, and handle password logins, and a lot more. But how much of it do you really need and how much of it are you going to find yourself customizing anyways later?

Password authentication is relatively simple, and if you’re going to store your users’ passwords you really should understand it. You can roll your own authentication, complete with current_user and logged_in? helpers in a matter of minutes.

Use Bcrypt to do simple encryption, store the password hash and salt, throw away the original password, maybe have an observer or a callback fire off a welcome email and you’re done. You’ll have less moving pieces, it’ll be tailored to your needs, and you can get coding instead of reading guides or sifting through documentation.

A lot of plugins are also written for accessing existing APIs, a lot of them are even written against REST APIs, which are already simple to access and manage. Make the calls yourself, learn the APIs, know for certain what is and what isn’t being cached.

4. Exceptions, non-plugins, cryptography.

At this point you might think I’m off my rocker. How can you not use any plugins? Some of them are tried and true and probably better than what you yourself could write. This is true.

What I’m trying to say is that there is a time and a place for plugins. Don’t write your own cryptography. Review all the plugins you use and make sure they’re going to be maintained, then ask yourself if you really need to use it. If you use plugins without ever peaking at the code or making any effort to understand the underlying technologies, you’re really doing yourself and your product a disservice. You’re not growing as a developer and you’re not acclimating yourself to the added responsibility of maintaining the addition of another mission critical service.

Upcoming Posts

When I set out to start my Tumblr blog my main goal was to log my work for my own reference. I’m going to continue doing that but I’m also going to try to post more than just code from now on - if you’re scared of long winded people, I suggest you leave now.

I’ve been watching YouTube videos by “nutnfancy” who reviews guns and knives and also talks about his opinion on “Rule of Law” and “Without Rule Of Law”. The videos are pretty fascinating. Yes, as a guy I love guns, but the videos are entertaining. You could make a strong argument that he is a bit paranoid, but he makes a case for being prepared. Boy is he passionate about being prepared.

His passion drives his research, his reviews, and ultimately his subscribers hip. His passion brings in millions of hits. He is a bit narcissistic - he’s not the first, otherwise why would the word even exist - but he believes in what he’s doing and ultimately I think what he gets off on is making an impact in the lives of his viewers in the areas where he has the strongest opinions on and is more passionate about. I admire that.

I’m a supremely passionate person, and I’m dedicated to the causes I see value in. Knowledge is what drives me and what I pursue. I want to be wowed by something but more than that I want to know how it works and then to be able to build it, or something better.

My preferred medium for that is software. It’s cheap in terms of the resources I have (time), it can be incredibly effective, and I have a knack for it.

Before I go on, I want to say that I’m not the most popular person. Sometimes I think it’s best to keep my views to myself, often I feel for good reason. Enough of a reason to sit there and hold in whatever I want to say. I’m not, however, ashamed of my opinions, and I’m open to all forms of self improvement - so long as they improve. Make mistakes, recognize them, apologize when necessary, try not to make them in the future.

Anyways, if you can’t tell I’m passionate about software. At this point in time that mainly encompasses: iOS, MongoDB (NoSQL), and Ruby on Rails - in that order. IOS is my new mistress, and I’m desperately trying to unlock her secrets (with some success so far). MongoDB is what I explored last month, and RoR is the staple of all my projects.

I’ll continue to write about those technologies, focusing more and more on iOS. I’m also going to start writing more about my philosophies on programming, entrepreneurship, and maybe life.

I’d like to cover (some of it is reference material for myself):
“Cheap server to server communication in RoR”
“Existing user JS check”
“Responding to JSONP cross domain”
“Daemons custom app names”
“$exist vs $ne => nil”
“Alias method chains”
“Ruby open classes”
“Programming pitfalls”
“How Wall Street Game works”
“AsyncLoader class for iOS”
“UI as frames and nesting, common to all languages”
“Web Dev to Flash to iOS”
“What drives me”
“The importance of respect”
“The importance of hard work”

Cocoa Blocks

No, not the chocolate cereal you used to eat as a kid (or adult). Blocks in Apple’s Cocoa serve the function of lambdas or anonymous functions in other languages. They’re most useful for implementing callbacks. Their only downfall is their funny syntax, which I’ll outline here:

Block as Property

@property (copy, nonatomic) void (^completeCallback)(NSMutableData *);
self.completeCallback = ^(NSMutableData *data) { ... };

Block as Variable

void (^completeCallback)(NSMutableData *);
completeCallback = ^(NSMutableData *data) { ... };

Block as Method Argument

- (void)loadWithCallback:(void (^)(BOOL)) loadCallback;
[obj loadWithCallback:^(BOOL success) {}];

Strange MongoDB Date Behavior

I’m about to post this in the MongoDB support forums, but I thought I’d keep track of it here as well.

Be careful of your date comparisons in your queries. When comparing the end of the day to the start of the next day, $lte seems to yield the wrong results.

1.9.3-p125 :011 > h = MarketHour.on_date(Time.zone.now).first
MONGODB wall_street_game_production['market_hours'].find({:_id=>{"$gte"=>2012-04-05 04:00:00 UTC, "$lte"=>2012-04-06 03:59:59 UTC}}).limit(-1).sort([[:_id, :asc]])
 => #<MarketHour _id: 2012-04-06 04:00:00 UTC, _type: nil, is_open: false, open_time: 2012-04-06 04:00:00 UTC, close_time: 2012-04-06 04:00:00 UTC>  

1.9.3-p125 :006 > h = MarketHour.where(:_id.gte => date.beginning_of_day.utc).where(:_id.lte => date.end_of_day.utc).first
MONGODB wall_street_game_production['market_hours'].find({"$and"=>[{:_id=>{"$gte"=>2012-04-05 04:00:00 UTC}}, {:_id=>{"$lte"=>2012-04-06 03:59:59 UTC}}]}).limit(-1).sort([[:_id, :asc]])
 => #<MarketHour _id: 2012-04-06 04:00:00 UTC, _type: nil, is_open: false, open_time: 2012-04-06 04:00:00 UTC, close_time: 2012-04-06 04:00:00 UTC> 

I was able to fix this by temporarily changing $lte to $lte.

Introduction to iOS Dev Part1: Cocoa and Objective-C

We’re finally starting dev on our iPhone app at Ender’s Fund, and I’m pretty excited because it means I get to dig into something new and hot. Officially, we won’t start deving for a few weeks, but I’m taking the time to get caught up (plus, I really just can’t wait!).

Apple makes individuals and companies sign up as developers (and pay a fee) to dev for their products, including iOS. Once you sign up, they give you access to their library of tutorials and reference material, which has been excellent and very comprehensive.

I’ve never written Objective-C, so I started there. Here are my big takeaways (the link to the original material is: https://developer.apple.com/library/ios/#referencelibrary/GettingStarted/RoadMapiOS/Languages/Languages.html, you’ll need to log-in first as it’s a dynamic js url):

Method Declaration and Calling (aka Object Messaging)

Methods (functions) have a funny declaration syntax in Objective-C. Instead of your typical function sumThisWithThat(arg1:typeArg1, arg2:typeArg2):returnType{…}, you have:

- (returnType)sumThis:(typeArg1)arg1 withThat:(typeArg2)arg2

…which looks very odd and is hard to type, but yield “sentence” like method calls:

[myObj sumThis:arg1 withThat:arg2];

Objective-C uses [,] to call functions. One benefit I can see from this structure is never getting confused about argument order as the method name itself will likely hint at the argument type.

- denotes an instance variable and + denotes a class variable.

File Types and Extensions

.h is for your headers or interface declarations of methods (class and instance)

.m is for your implementation of methods

A Typical .h File

@interface SomeClass : NSObject
{
  int       x;
  id        y;
  NSString* z;
}
+ (SomeClass*)initAndAlloc;
- (int)setX:(int)newX;
@end

@interface and @end are just syntactical sugar for defining the interface. : is used to denote inheritance. A class can only inherit from one parent. NSObject is the root class from which all classes extend. id is the type given to untyped objects. int and NSString* are the respective types of the primitive integer and object reference to a NSString object.

A Typical .m File

#import "SomeClass.h"

@implementation SomeClass
+ (SomeClass*)initAndAlloc
{
  //  allocate and initialize
}

- (int)setX:(int)newX
{
  // set x to newX
}
@end

The header file gets imported with the #import declaration. The #import declaration is special in that it will only include the header file once.

Just like in C, memory must be allocated via alloc. Once the memory is allocated, the base object can be initialized. Cocoa calls methods that both alloc and initialize factory methods.

No Public, Private, Protected, Internal

Objective-C doesn’t seem to allow a programmer to limit access to particular methods. However, http://stackoverflow.com/questions/172598/best-way-to-define-private-methods-for-a-class-in-objective-c outlines a way to handle this in newer versions of iOS. Also, since instance variables should be set with setters and getters, it is prudent to not define setters or getters when not necessary.

Auto-magically Define Setters/Getters with @property and @synthesize

Creating setters and getters for your instance variables is tedious so Cocoa provides a quick way to do so with @property, which does in your header files, and @synthesize, which goes in your implementation file.

The @property declaration outlines all the details of that particular setter and getter, and @synthesize basically says, “yes, go about making them here.” An example:

# .h
@property int x;

# .m
@synthesize x;

Lambdas and Anonymous Functions as Blocks

I don’t want to go into blocks here, but from my initial readings they behave just like lambdas and anonymous functions. They’re useful for callbacks.

BOOL = {YES, NO}

I found this strange. A (BOOL)var takes on the values YES and NO, not true and false.

Protocols

For interfacing different bits of your code base it’s very often necessary to define interfaces (a set of common methods that have agreed-upon return and argument types. Protocols satisfy this need. They’re used like this:

# .h
@interface SomeClass : NSObject 
...
@end

# other .h
@protocol SomeProtocol
...
@end

They can’t define instance variables of course, but can define @property. This just means it expects SomeClass to implement the setters and getters.

Mutability

A lot of Cocoa’s framework relies on objects that are prefixed with “NS…”. Mutability specifies the ability for an object to change after it has been defined, like host .freeze and const work in Ruby and AS respectively. Cocoa provides both mutable and non-mutable version of their objects. For example:

• NSString and NSMutableString
• NSData and NSMutableData
• NSArray and NSMutableArray
• NSDictionary and NSMutableDictionary
• NSSet and NSMutableSet

Data Structures

Out-of-the-box data structures are listed above. They are arrays, dictionaries, and sets, both mutable and immutable and are self explanatory

End Part 1

I’ll try to update the blog as I learn more, but I will likely get ahead of my blog and become too lazy to update the block or too occupied with learning more. I will try to keep the blog up to date however, and I will post about the things I find weird in iOS dev as I have for my other endeavors.

Resolved: Function Closures in jQuery vs JavaScript

Edit: DUH. I didn’t realize this when I was explaining it to my friend but this is actually really simple. The function closure does happen in jQuery, the function we pass in acts as the function closure essentially. The function in each’s argument $.each(function(i,e){…}); gets called with i,e and creates the function closure we need. I was so used to simply typing function(i,e){…} after my .each method calls that I totally overlooked it.

I was explaining function closures to my buddy when I realized that jQuery handles function closures somewhat “auto-magically” with the each iterator.

The normal behavior of JavaScript without a function closure is:

for(i=0;i<as.length;i++){
  as[i].onclick = function(){
    console.log(i);
    return false;
  };
}

4
4
4
4

With a function closure, you get the intended results:

for(i=0;i<as.length;i++){
  as[i].onclick = (function(z){
    return function (){
      console.log(z);
      return false;
    }
  })(i);
}

0
1
2
3
4

However, function closures aren’t needed with jQuery’s each iterator:

as.each(function(i,e){
  $(e).click(function(){
     console.log(i);
     return false;
  })
});

Here’s the full source:

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js" type="text/javascript"></script>
    <script type="text/javascript">
      $(document).ready(function(){
        //rotating banner
        var as = $('a');

        // incorrect
        for(i=0;i<as.length;i++){
          as[i].onclick = function(){
            console.log(i);
            return false;
          };
        }

        // correct
        for(i=0;i<as.length;i++){
          as[i].onclick = (function(z){
            return function (){
              console.log(z);
              return false;
            }
          })(i);
        }

        // correct
        for(i=0;i<as.length;i++){
          as[i].onclick = callback(i);
        }

        function callback(z){
          return function(){
            console.log(z);
            return false;
          }
        }

        // correct in jQuery
        as.each(function(i,e){
          $(e).click(function(){
            console.log(i);
            return false;
          })
        });

      });
    </script>
  </head>
  <body>
    <a href="#">0</a>
    <a href="#">1</a>
    <a href="#">2</a>
    <a href="#">3</a>
    <a href="#">4</a>
  </body>
</html>

Fixing Time Drift with NTP

I found a curious bug in Wall Street Game’s logs today.

RuntimeError (Start date must be before end date.)

We do a number of validations to make sure data is groovy throughout the game. On a particular call to our utility that calculates the number of market opens between two datetime objects, that error popped up.

What’s happening is a record is being created on one of our server nodes that sets the start date to the current time. Then, on a subsequent request, the start date is compared again against the current time to find the number of market days between the two dates. However, because we load balance our traffic the second request was hitting a server where the current time was actually before the start date. This only occurs when the object is created and then accessed immediately afterwards from a different node.

The solution is to sync the clocks across all nodes and minimize the difference in time from one node to another. RHEL and other linux distros use NTP to sync time from public hosts. To install, configure and run NTP you just have to issue the following commands:

sudo yum install ntp
sudo chkconfig ntpd on
sudo /etc/init.d/ntpd start

The first line installs ntp from your configured repositories using yum, the second line makes the service start on boot, and the third line actually runs the service.

There are additional configurations to NTP where you can sync servers down to the millisecond, but for our setup syncing the date to the second is enough.

Rails int (in seconds) to Time

I always forget how to convert seconds since epoch to Time:

1.9.2-p0 :006 > Time.now.to_i
 => 1331234348 
1.9.2-p0 :007 > Time.at(1331234348)
 => 2012-03-08 11:19:08 -0800

Setting the Time Zone to UTC on RHEL

For whatever reason the default time-zone on my EC2 instances running RHEL was set to “America/New_York”. This was problematic because our cron tasks and applications depended on the system clock reading in UTC. To set the time-zone to “UTC”, I did the following:

sudo vi /etc/sysconfig/clock
ZONE="UTC"
UTC=True

sudo cp /usr/share/zoneinfo/UTC /etc/localtime

rvm uninstall ruby completely

To make it like it never happened:

rvm remove 1.9.3 --gems --archive