Programming

You know when you see code like this:

  class CompulsionsController < ApplicationController

      # ... standard actions above here

      def update

        if params[:obsessions].include?(ObsessionsTypes[:murdering_small_animals])

          handle_sociopathic_obsessions 

          redirect_to socio_path and return

        elsif params[:obsessions]

          handle_normal_obsessions

          redirect_to standard_obsessions_path and return

        end

        # normal update for compulsions 

        @compulsion = Compulsions.find(params[:id])

        if(@compulsion.update_attributes(params[:compulsion]))

        # ... remainder of the standard actions below here 

    end

and the phrase "WTF were they thinking?" runs through your mind? More...

I have a theory about that little "pass a flag in the url to skip over the real action and perform a different one" trick I see so often (and have been guilty of using myself).

It's because you've got this omniscient file that knows everything about where to route requests that's not part of your editing routine, so finding and opening it breaks your train of thought.

It's a pain to open routes.rb when you suddenly realize you need a new route.

That got me thinking: Should controllers route themselves? Would it make more sense for a controller to tell the router how each of it's actions should be reached?

In the second edition of Code Complete (that's an affiliate link), Steve McConnell writes about using the Principle of Proximity (page 352) as a way to think about organizing code.

Keep related actions together.

From that point of view, it certainly would be easier to follow along when you're questioning "how do I get to this action?" Further, I think it would help solve the "pass a flag to an action to perform a different one" problem I illustrated in the code snippet above.

It was on my mind over the weekend, so I put together this little experiment to see what controllers routing themselves in Rails would look like. In that repository is a one-controller Rails project which specifies routes to itself using a gem you'll find in vendor/gems/route.

One major drawback to doing routing in this style has to do with nested routes: should a controller that's part of a nested route know who it's parents are? Should a higher-in-the-nest controller know about its child routes? And if you choose one or the other, how would you specify it? What if there are conflicting routes -- who wins out?

It leads to a lot of questions for which I have no immediate answers.

Anyway, what do you think? Would this help solve the problem of recycled routes? Is that even a problem?

What are the drawbacks of such an approach? Do you see any merits?

Hey! Why don't you make your life easier and subscribe to the full post or short blurb RSS feed? I'm so confident you'll love my smelly pasta plate wisdom that I'm offering a no-strings-attached, lifetime money back guarantee!

My vocabulary is failing me right now. What do you call it when a piece of code checks the type of an object before doing something to it?

Type Casing is the act of using case statements in a program to determine what to do with an object based on what type of object it is. It's an OO fail, often hoping to implement Multiple Dispatch. (See also Case Statements Considered Harmful)

Here are three passive-aggressive ways to feel like you're getting back at typecasers.

More...

The first tactic turns your object into an everything, so it's whatever the typecaser was looking for. I've called it OmniObject.

module OmniObject

  def is_a?(*)

    true

  end

  def kind_of?(*)

    true

  end

  def nil?

    true

  end

end

foo = "hello, world!"

foo.extend OmniObject

puts "Is foo a Fixnum? #{foo.is_a?(Fixnum) ? 'yes' : 'no'}"

puts "Is foo a Kernel? #{foo.is_a?(Kernel) ? 'yes' : 'no'}"

puts "Is foo a NilClass? #{foo.kind_of?(NilClass) ? 'yes' : 'no'}"

puts "foo.nil? => #{foo.nil?}"

The next one makes your object unable to decide what it is, turning it into a FickleTeenager. If he has to check more than once, the typecaser is going to have a tough time with a kid who can't make up his mind.

module FickleTeenager

  def is_a?(*)

    sorta

  end

  def kind_of?(*)

    sorta

  end

  def nil?

    sorta

  end

  def sorta

    truish

  end

  def truish

    rand < 0.5

  end

end

foo = "hello, world!"

foo.extend FickleTeenager

3.times{ puts "Is foo a String? #{foo.kind_of?(String) ? 'yes' : 'no'}"}

Finally, we have the AntisocialPrivacyAdvocate. When the typecaser asks him what he is, he tells them like it is: It's none of your damn business!

class WhatBusinessOfItIsYoursError < StandardError; end;

module AntisocialPrivacyAdvocate

  def is_a?(*)

    raise  WhatBusinessOfItIsYoursError

  end

  def kind_of?(*)

    is_a?

  end

  def nil?

    is_a?(NilClass)

  end

end

foo = "hello, world!"

foo.extend AntisocialPrivacyAdvocate

result = foo.kind_of?(String) rescue "#{$!} OMG, How Rude!"

puts result

With a name like each_cons, I thought you were going to iterate through all the permutations of how I could construct a list you operated upon. For example, I thought

[1,2,3,4].each_cons do |x| # I did not notice the required argument

    puts x.inspect

  end

would output: More...

  [[1,2,3,4], []]

  [[1,2,3], [4]]

  [[1,2], [3,4]]

  [[1], [2,3,4]]

  [[], [1,2,3,4]]

So when I needed to find the local maxima in an image projection to algorithmically find the staves in sheet music, I found myself wanting a C-style for loop.

I didn't know you'd provide me with a wonderful sliding window!

[1,2,3,4].each_cons(2) do |x|

    puts x.inspect

  end

  [1, 2]

  [2, 3]

  [3, 4]

From now on, I'll turn to you when I need that functionality. Thanks for waiting on me, each_cons. Not everyone would be as patient as you.

Warm Regards,
Sam

PS: In case you're interested, the "cons" in "each_cons" is short for "consecutive," not "construct," as Matz informed me:

Matz says 'I am not the one who come up with the name, but I suppose cons is the short form of consecutive.'

It's a small step, but emcee-3PO can now identify the staves in an image of sheet music for my single test case of "My Darling Clementine." I need to include hundreds more test cases, and I plan to when I implement code to make the tests mark the sheet music with what emcee3po detected so I can visually inspect the accuracy.

Ichiro Fujinaga's "Optical Music Recognition using Projections" (PDF) explains the process in detail, but it turns out to be relatively simple.

To locate the staves:

Do a y-projection on the image.
A projection just reduces the number of dimensions in an image. In this case, we just take the number of dark-colored pixels in a row of the image. It's similar in theory to 3D projection, but instead of projecting three dimensions onto a plane, we're projecting a plane onto a line.

I used a threshold of 50% to determine if a pixel was dark enough to include in the projection. So, if R+G+B < (FF+FF+FF) / 2, I count the pixel as dark.
Find the local maxima.
We want to find the places where the number of dark pixels in a row is highest - those will indicate the horizontal lines on the staff. To do that, we find all the places where the number of pixels stops growing and starts getting smaller -- or where the slope changes from positive to negative. To ignore noise, we set a threshold as Fujinaga suggests at the average of each row, so we don't include anything less than that in our collection of local maxima.
Find the tightest groups of 5.
We want to find all the places where 5 local maxima are the smallest distance apart, which should indicate the 5 lines in a staff. This part is accomplished by examining each 5-element window in the array of local maxima, and finding the one with the smallest distance between its points. Then you can remove all the windows that include any of those points, and continue until there are no more windows.
Expand those indexes to collect the places where the notes fall outside the staff lines.
I don't remember Fujinaga mentioning this in the paper I linked to above, but I'm thinking it must be in there. Essentially, since the local maxima get us only what's in between the 5 lines of the staff, we need to expand it a bit so we can get the notes that don't fall directly between the 5 lines. Right now, I've used 1/4 of the average of the rows in the projection, but I think it will need to be an even smaller threshold because I'm still not reliably getting all of the notes.

Up next: reading the notes on the staves. That's going to be cool.

Frequent changes and deprecations to technology you rely upon cause dependencies to break if you want to upgrade. In many cases, you'll find yourself hacking through someone else's code to try to fix it, because a new version has yet to be release (and probably never will). That can be a chore.

I get embarrassed when something I've recommended falls prey to this cycle. Backwards compatibility is one way to deal with the issue, but in the Rails world, few seem worried about it. It doesn't bother me that code and interface quality is a high priority, but it does cause extra work.

There's a trick you can use to help isolate the pain, decrease the work involved in keeping your app up to date, and improve your code in general. You've probably heard of it, but you might not be using it to help you out in this area: encapsulation.

More...

What I'm about to describe talks heavily about Ruby on Rails, but that's only because it was the focus of what got me thinking about this. I think it really applies in general.

Problem

Rails changes frequently. Some people think it changes too frequently. I'll leave it for you to decide whether or not that's the case.

One consequence of change is that plugins and gems designed to work with Rails break. Perhaps as a result of frequent changes (in Rails or just the whims of the community about which project in a similar area does the job best), the authors of those packages become disillusioned and abandon the project. They could just be lacking time, of course.

Now you get to fix their code (which doesn't have to be a bad thing, if you contribute it back and someone else is sparred the trouble), use a new project that does the same thing, roll your own, or sit on an old version of Rails that time forgot, and everyone else forgot how to use.

Don't you get at least a little embarrassed that you have to recommend large changes to your customer or product owner or manager as a result of upgrading to the latest version of a technology you recommended using?

It reminds me of a quote from Harry Browne I heard as part of his year 2000 United States Presidential election campaign:

Government is good at only one thing. It knows how to break your legs, hand you a crutch, and say, "See if it weren't for the government, you couldn't walk."

I'm likening programmers to the government of that quote, except we don't pretend to give the crutch away. We tell them "you can't walk without our crutch, so pay up." We sell people on a technology which they build their business around, and then tell them they have to choose between keeping a substandard version of it, or spending a lot of money to upgrade all the components.

(Understand that I'm talking about how it feels overall as a result of what happens in the community of programmers, not a particular instance of particular decisions by any particular well-defined group of programmers.)

I just got done migrating a Rails 2.3 app to Rails 3.1 that was heavily dependent on several plugins and gems. After writing a ton of missing tests, I made the switch and ran them. As expected, there were loads of errors. More than half of them were due to the application's own code, but those failures were fixed with very little effort. By far the most excruciating and time consuming task (even longer than writing all of the tests) was spent getting the plugins to work (or in some cases, using new plugins and changing the application to use them instead).

I acknowledge that I'm doing something wrong, because surely not everyone has this problem. So tell me, how can I improve my experience?

A Solution?

Something I'd like to see more of in this area is to encapsulate your plugin usage. Rather than include NoneSuch, why don't you take the time to wrap it in your own code? In doing so, I see a couple of benefits:

You document which features you're using by only writing methods for those in use. That means you have something specific to test against in your app, and something specific to run those tests agains when you trade one dependency for another. It also means you know exactly what you need to implement when you swap dependencies.
You ensure a consistent interface for your app to use, as opposed to having to change it when you swap out which plugin you're using. Also, all your changes to that interface are localized, instead of spread throughout your source code.

That means you can change one external dependency for another with greater ease, which means you'll be a lot less likely to get off in the weeds trying to make something fundamentally unworkable work.

Thoughts?

emcee-3PO takes a text file of music, runs it through a Markov Model, and generates new music to be played with Archaeopteryx.

It's an idea I've had for a while, which _why day gave me a good excuse to start.

There's not much there yet -- the first sentence above tells you exactly what it does -- but I hope to add some features to it as time allows:

Read from sheet music
Use different instruments instead of simply playing the notes as written / make it symphonic
Choose probabilities in some smart way to provide to Arkx (right now it's just 100% play this note)
Mixing multiple songs

I just thought it'd be a fun exercise, but I bet there's some really cool practical stuff this could turn into.

Feel free to watch the repository if you're interested in seeing this develop.

More importantly, and use your imagination here: Do you have any thoughts about what something like this could do?

When you first introduce someone to source control, things often go smoothly until the first time they have to merge conflicting changes. Then they wonder,

What's the point of this? I thought it was supposed to be easy but it's a PITA.

Two responses on how to combat this immediately come to mind.

The first thing that can help in this situation is to ensure they aren't having to merge useless files. I'm thinking of files here like the CSS generated from SASS: they change frequently but the changes don't affect them one way or the other. (In this case, because the CSS will be regenerated). Another example is a user-specific project settings file.

Two strategies to avoid useless merging are to ignore files (do not have the repository track them) and to automatically use one version of a file or another. Each has it's place.

In the case of a file that needn't be in the repository to begin with -- things like Thumbs.db or .DS_Store -- you should obviously ignore them. In the cases where files should be in the repository, but where you know which version you want all the time, you should consider telling it to always choose one file over another.

If you're using git, .gitignore will help you ignore files, while .gitattributes will help you choose one file over another without specifying it every time. I only wanted to make you aware of this, and Pro Git explains it better than I could, so I'll let you read about how to do it over there.

Thanks to Markus Prinz who helped me find .gitattributes when I didn't know the term I was looking for.

So what's the second thing that helps a newcomer to source control overcome their hatred of merging conflicting changes?

Remind them the alternative is to consistently have their own work overwritten.

I was introduced to the concept of making estimates in story points instead of hours back in the Software Development Practices course when I was in grad school at UH (taught by professors Venkat and Jaspal Subhlok).

I became a fan of the practice, so when I started writing todoxy to manage my todo list, it was an easy decision to not assign units to the estimates part of the app. I didn't really think about it for a while, but recently a todoxy user asked me

What units does the "Estimate" field accept?

More...

My response was

The estimate field is purposefully unit-less. That's because the estimate field gets used in determining how much you can get done in a week, so you could think of it in hours, minutes, days, socks, difficulty, rainbows, or whatever -- just as long as in the same list you always think of it in the same terms.

A while back, Jeff Sutherland (one of the inventors of the Scrum development process) pointed out some of the reasons why story points are better than hours. It boiled down to four reasons:

We are bad at estimating hours, but more consistent with points
Hours tell us nothing since the best developer on the team may be multiple times faster than the worst
It takes less time to estimate in points than hours
"The management metric for project delivery needs to be a unit of production [because] production is the precondition to revenue ... [and] hours are expense and should be reduced or eliminated whenever possible"

But I noticed another benefit in my personal habits. Not only does it free us of the shackles of thinking in time and the poor estimates that come as a result, it corrects itself when you make mistakes.

I recognized this when I saw myself giving higher estimates for work I didn't really want to do. Like a contractor multiplying by a pain-in-the-ass factor for her worst customer, I was consistently going to fib(x+1) in my estimates for a project I wasn't enjoying.

But it doesn't matter. My velocity on that list has a higher number than on my other list, so if anything I hurt myself by committing to more work on it weekly for any items that weren't inflated.

What do you think about estimating projects in leprechauns?

Yesterday I got sick of typing rake test and rake db:migrate and being told

You have already activated rake 0.9.2, but your Gemfile requires rake 0.8.7. Consider using bundle exec.

I know you should always run bundle exec, but my unconscious memory has not caught up with my conscious one on that aspect, so I always forget to run rake under bundle exec.

So I wondered aloud on twitter if I could just alias rake to bundle exec rake, but confine that setting to specific directories (with bash being my shell).

Turns out, it is possible with the help of another tool that Calvin Spealman pointed me towards: capn.

More...

To successfully run the commands I've listed below, you need to have python and homebrew already installed. If you already have libyaml installed or have another way of getting it, there's no need for homebrew.

The section beginning with the line that starts out with echo and ending with -unalias rake"... creates the capn config file. It's just YAML, so if you'd rather create it with a text editor, you can surely do so. See the capn project for details on the config possibilities.

Either way, you'll want to change the paths I've used to the places you want to do the aliasing.

From the terminal, run the following commands:

curl -O http://python-distribute.org/distribute_setup.py

sudo python distribute_setup.py 

sudo easy_install pip

sudo brew install libyaml

sudo easy_install -U pyyaml

sudo pip install capn

echo "

settings:

  default_type: path

hooks:

  - path: ~/workspace #change this to the path where you want to alias rake

    type: tree # if you don't want the whole tree under the path above, remove this line

    enter: 

      - echo aliasing rake to 'bundle exec rake'

      - alias rake='bundle exec rake'

    exit: 

      - echo unaliasing rake from 'bundle exec rake'

      - unalias rake" > ~/.capnhooks

source capn # put this line in your .bash_profile if you want capn to work when you enter the shell

# to deactivate the hooks, use: unhook

Enjoy the silence now that you don't have to hear the whining.

I have a job where in any given week I might be working on any one of 30 projects spread across a half dozen product lines. I freelance, sometimes with a single company, but I also work a lot through another company for several different customers. I have my personal projects too, of course, and then there's non-work type things like getting a haircut, building a garden, or changing the air filters around the house.

Problem

My list of things to do is too complex for me to keep in my head. It doesn't fit in a typical list because I might want to see what needs to be done, not just at work, or not just for a client, but also for different customers or projects, or both.

Furthermore, it doesn't quite fit into a project management tool either. I need something more flexible that lets me keep my professional and personal lists in the same place, and that gives me just a list when I need it, or some predictions and statistics and data when those things are appropriate.

Goals

So when I started on this project, I wanted something more than a todo list, but not as involved as a project management suite. More...

I don't remember what I thought the first time saw the title of this chapter ("Learn to Love Maintenance") from My Job Went To India. I felt sick though. The thought process probably went something like this:

Oh no. You mean I'm going to be stuck here so long I better learn to love it?

I've got it really bad - I have to maintain a bunch of the code I wrote. Mere mortals cannot comprehend how bad that is. When do I get to toss my refuse off to some other sorry excuse for a programmer?

Apparently that's a common response programmers have, given the prospect of enjoying an illustrious career as a an elephant dung-slinger.

But Chad Fowler (in the book) turns it around, saying that not much is expected of maintenance programmers. You just need to fix the occasional bug or add a small feature here or there. It really boils down to this:

[In a greenfield project,] when we don't have the constraints of bad legacy code and lack of funding to deal with, our managers and customers rightfully expect more from us. And, in project work, there is an expected business improvement. If we don't deliver it, we have failed. Since our companies are counting on these business improvements, they will often put tight reigns on what gets created, how, and by when. Suddenly, our creative playground starts to feel more like a military operation - our every move dictated from above.

But in maintenance mode, all we're expected to do is keep the software running smoothly and for as little money as possible. Nobody expects anything from the maintenance crew. Typically, if all is going well, customers will stay pretty hands-off with the daily management of the maintainers and their work. Fix bugs, implement small feature requests, and keep it running. That's all you have to do.

Moreover, after enough code is written, that new project isn't much different than your maintenance work. It's just missing the benefits.

Consequently, you've got a lot more freedom in maintenance to do as you will. Get creative. Spruce up the UI a little bit. Since you get to interact with your "customer" more often, "more people will know who you are, and you'll have the chance to build a larger base of advocates in your business."

On top of that, being responsible for the entire application, it's likely that "even without much effort, you will come to understand what the application actually does." In doing so, you're well on your way to becoming a domain expert.

As I've mentioned before in several of the "Save Your Job" series' posts, as of this writing, I'm working with a small company. So, not only am I a maintenance programmer, I'm a greenfield project programmer too. I've been known to wear more than one hat (As I'm sure many of you can say).

Because of that and the push to drive maintenance costs down - I don't get as many opportunities to get creative in maintenance as Chad suggests. That's a bit of a downer for me.

But he ends it on a motivational high note in the "Act on it!" section: Pick the most important aspect of your maintenance work and find a way to measure it. Make it a point to improve your performance in that area, and measure again. Continuously improve. It's more motivating than having the mindset laid out in the introduction to this post, and you'll likely raise a few eyebrows.

Suppose you want to write an algorithm that, when given a set of data points, will find an appropriate number of clusters within the data. In other words, you want to find the k for input to the k-means algorithm without having any a priori knowledge about the data. (Here is my own failed attempt at finding the k in k-means.)

def test_find_k_for_k_means_given_data_points()
  data_points = [1,2,3,9,10,11,20,21,22]
  k = find_k_for_k_means(data_points)
  assert(k==3, "find_k_for_k_means found the wrong k.")
end

The test above is a reasonable specification for what the algorithm should do. But take it further: can you actually design the algorithm by writing unit tests and making them pass?

I've previously expressed my doubt that TDD makes an effective approach to algorithm design. More recently, I alluded to a some optimism towards the same idea. Then, in a comment to the same post, Dat Chu asked about using unit tests in algorithm design, also referencing something that Ben Nadel had said about asserting in code-comments how the state of the algorithm should look at certain points.

That all led to this post, and me wanting to lay my thoughts out a little further.

In the general case, I agree with Dat that it would be better to have the executable tests/specs. But, what Ben has described sounds like a stronger version of what Steve McConnell called the pseudocode programming process in Code Complete 2, which can be useful in working your way through an algorithm.

Taking it to the next step, with executable asserts - the "Iterative Approach To Algorithm Design" post came out of a case similar to the one described at the top. Imagine you're coming up with something completely new to you (in fact, in our case, we think it is new to anyone), and you know what you want the results to be, but you're not quite sure how to transform the input to get them.

What good does it do me to have that test if I don't know how to make it pass? The unit test is useful for testing the entire unit (algorithm), but not as helpful for testing the bits in between.

Now, you could potentially break the algorithm into pieces - but if you're working through it for the first time, it's unlikely you'll see those breaking points up front. When you do see them, you can write a test if you like. However, if it's not really a complete unit, then you'll probably end up throwing the test away.

Because of that, and the inability to notice the units until after you've created them, I like the simple assert statements as opposed to the tests, at least in this case.

When we tried solving Sudoku using TDD during a couple of meetings of the UH Code Dojo, we introduced a lot of methods I felt were artificially there, just to be able to test them. We also created an object where one might not have existed had we known a way to solve Sudoku through code to begin with.

Now, we could easily clean up the interface when we're done, but I don't really feel a compulsion to practice TDD when working on algorithms like I've outlined above. I will write several tests for them to make sure they work, but (at least today) I prefer to work through them without the hassle of writing tests for the subatomic particles that make up the unit.

When I try out a product, I like it to work. Sometimes, I like to tinker with things to gain a better understanding of how they work. Occasionally, I can manipulate them with skill. At other times, I'm tinkering in the true sense of the word.

I'm going to point out a few problems I've had with some products I've been using. I won't name names, but some of you who also use these products (or similar ones with the same problems) might understand what I'm talking about. Hopefully, you all can draw a good conclusion from it.

I'm not a mechanic, but sometimes I might want to look into my radiator. Is there a reason I need to disassemble the front of my car to unscrew the radiator cap? Likewise, diagnosing a problem with, and subsequently changing a starter or alternator isn't a task that only automobile doctors should perform. These are relatively high-failure parts that shouldn't require taking apart the exhaust system to replace. That's the tinkerer in me talking.

Picture of an engine

Image courtesy Daimler-Chrysler via How Stuff Works

I am a software developer, but sometimes I don't want to act like one when I work with software written by someone else. Don't make me build your product from source. I'm not asking you to build it for me, and then email it to me. I'm just asking if it's possible for you to set up an automated build on all of the platforms where you intend your software to work.

I enjoy the ability to tinker with open source software, not the requirement to do so.

The situation is even worse for proprietary programs. If you're lucky, there might be a way to mess with the settings that satisfies the tinkerer in you. But if you're not lucky, you could do things like upgrade your operating system and have it break something in 3rd party code. Then you'll be at the mercy of that vendor to make their software run properly with your upgrade. In the mean time, you're stuck manually starting the software, or writing your own scripts to do it.

In the first release of an accounting package, I might expect to edit a text file to set my tax rate. But I better not have to do that in the ninth version. Moreover, if your software makes certain assumptions about its running environment and those assumptions are parts that are likely to fail, you better make sure I can change them in your program without tearing it apart.

In the end, I understand that for early versions of products, you might not have worked out all the kinks or even found its purpose or audience. I expect that as an early adopter, and when I take that plunge, I enjoy it. But once you're several years old and multiple versions in, I don't think it's too much to have certain expectations. At that point, your software should just work and make it easy to make small adjustments.

When I work in Windows, I don't get as much done as when I'm in MacOS X.

It's not because MacOS is inherently better than Windows productivity-wise. It's because my calendar and time-boxing mechanism resides on MacOS. So when I've got an entire day of work to do in Windows, I don't have anything telling me "it's time to switch tasks."

Why is that a problem? That's the focus of this week's chapter in MJWTI. (Last week, I took a mini-vacation / early bachelor party to go fishing at Lake Calcasieu in Southwest Louisiana, so I didn't get around to posting then in the Save Your Job series.)

The "Eight-Hour Burn" is another of the chapters in Chad's book that really stuck with me after I first read it. The premise is that instead of working overtime, you should limit each day's work to an 8 hour period of intense activity. In doing so, you'll get more done than you otherwise would. Our brains don't function at as high a level as possible when we're tired. And when we're working on our fifth 60-hour week, we're probably tired.

We may love to brag about how productive we are with our all-nighters [paraphrasing Chad], but the reality is we can't be effective working too many hours over a long period of time.

A Brain Scan

And it's more than just our brains being tired that should prevent us from working too long. It's the fact that when we know we've got so much extra time to do something, we end up goofing off anyway:

Think about day 4 of the last 70-hour week you worked. No doubt, you were putting in a valiant effort. But, by day 4, you start to get lax with your time. It's 10:30 AM, and I know I'm going to be here for hours after everyone else goes home. I think I'll check out the latest technology news for a while. When you have too much time to work, your work time reduces significantly in value. If you have 70 hours available, each hour is less precious to you than when you have 40 hours available.

That's why I'm in trouble when I work in Windows all day. I do work 12-16 hours most days between job, school, and personal activity (like this blog). I get burnt out every few weeks and have to take a couple of days off, but when I'm in MacOS X, at least my working days are very productive: I've got each task time-boxed and any time I spend reading blogs or news or just getting lost on the web is always scheduled.

When I'm in Windows with nothing to remind me but myself, I drift off quite a bit more easily. After all, it's only 6:30 in the morning. I've still got eight hours to get everything done (I'm leaving early to go check the progress on the house I'm having built).

The good news is that I've got an item on my longer-term to-do list to remedy the situation. Hopefully after reading this chapter again, I'll be more motivated to get it done. The bad news is, since it means working in Windows all day to get it done, I'll probably be off doing my best to read all of Wikipedia instead.

Anyway, how much do you work? Do you have any tricks for staying fresh and motivated?

An interesting discussion going on over at programming reddit got me thinking about what features I'd like my ideal job to have. Aside from "an easy, overpaid job that will get bring me fame and fortune," here's what would make up my list:

Interesting Work: You can only write so many CRUD applications before the tedium drives you insane, and you think to yourself, "there's got to be a better way." Even if you don't find a framework to assist you, you'll end up writing your own and copying ideas from the others. And even that only stays interesting until you've relieved yourself of the burden of repetition.

Having interesting work is so important to me that I would contemplate a large reduction in income at the chance to have it (assuming my budget stays in tact or can be reworked to continue having a comfortable-enough lifestyle, and I can still put money away for retirement). Of course,the more interesting the work is, the more of an income reduction I could stand.

Fortunately, interesting work often means harder work, so many times you needn't trade down in salary - or if you do, it may only be temporary.

The opportunity to present at conferences and publish papers amplifies this attribute.
Competent Coworkers: One thing I can't stand is incompetence. I don't expect everyone to be an expert, and I don't expect them to know about all the frameworks and languages and tricks of the trade.

I do expect programmers to have basic problem solving skills, and the ability to learn how to fish.
Sane Management: Don't make me delighted to have a broken watch. I shouldn't have to turn back the dial on it to meet deadlines. Or put more strongly, management/customers shouldn't be given the latitude to control all three sides of the iron triangle of software development.

Scope, Schedule, and Resources. Choose two. We, the development team, get to control the third.
Trust: One comment in the reddit discussion mentioned root access to my workstation and it made me think of trust. If you cannot trust me to administer my own computer, how can you trust me not to subvert the software I'm writing?

Additionally, I need you to trust my opinion. When I recommend some course of action, I need to know that you have a good technical reason for refusing it, or a good business reason. I want an argument as to why I am wrong, or why my advice is ignored. If you've got me consistently using a hammer to drive marshmallows through a bag of corn chips, I'm not likely to stay content.

I don't mind if you verify. In fact, I'd like that very much. It'll help keep me honest and vigilant.
Personal Time: I like the idea of Google's 20% time. I have other projects I'd like to work on, and I'd like the chance to do that on the job. One thing I'd like to see though, is that the 20% time is enforced: every Friday you're supposed to work on a project other than your current project. It'd be nice to know I'm not looked down upon by management as selfish because I chose to work on my project instead of theirs.

I wouldn't mind seeing something in writing about having a share of the profits it generates. I don't want to be working on this at home and have to worry about who owns the IP. And part of that should allow me to work on open source projects where the company won't retain any rights on the code I produce.
Telecommuting: Some days I'll have errands to run, and I dislike doing them. I really dislike doing them on the weekends or in the evenings after work. I'd like to be able to work from home on these days, do my errands, and work late to make up for it. It saves the drive time on those days to help ensure I can get everything done.

There are some other nice-to-have's that didn't quite make the list above for me. For example, it would be nice to be able to travel on occasion. It would also be nice to have conferences, books, and extended learning in general paid for by the company. I'd like to see a personal budget for buying tools I need, along with quality tools to start with.

But if you can meet some agreeable combination of the six qualities above, I'll gladly provide these things myself. If you won't let me use my own equipment, and you provide me with crap, we may not have a deal.

That's my list. What's on yours?

When I wrote about things I'd like to have in a job, I didn't expect that one of the items on my list would draw the kind of reaction it did. A couple of comments seemed to think I'm off my rocker about personal project time:

Why should I pay for the time you spend doing your own projects? You are free to have 20% of the time to yourself if you can take 20% pay cut (from Adedeji O.)

Did I understand you correct? You want to work on something that has no relation to your employer and still get paid by him?
...
I don't know the exact rates in the US, but, these 20% will easily sum up to more than 10,000 USD per developer per year. Would YOU really pay that for nothing? (from Christoph S.)

To be fair, Christoph did say that he's "not talking about time for personal and technical development and not about work related projects that MAY include benefit for your employer. I'm talking about working on 'my projects' and 'open source projects.'"

Here's what I had said that provoked the reactions:

I like the idea of Google's 20% time. I have other projects I'd like to work on, and I'd like the chance to do that on the job. One thing I'd like to see though, is that the 20% time is enforced: every Friday you're supposed to work on a project other than your current project. It'd be nice to know I'm not looked down upon by management as selfish because I chose to work on my project instead of theirs.

I wouldn't mind seeing something in writing about having a share of the profits it generates. I don't want to be working on this at home and have to worry about who owns the IP. And part of that should allow me to work on open source projects where the company won't retain any rights on the code I produce.

Calling it "my project," talking about who owns the intellectual property, and working on open source appear to be where I crossed the line. At Google, we see things like News and GMail coming from the personal time. What I stated could mean that a programmer for a bank's website that's done in ColdFusion could end up working on a GUI for Linux in C. It's rather hard to make the connection there.

First, I didn't mean to imply that the company would not own anything of any part of what I worked on. If I am willing to take a 20% cut in pay, then certainly I wouldn't expect them to own anything. What I was looking for is some equitable way to share what I create with my time. For example, I might take one day a week at work to build my new widget, but if I'm taking 2 days on the weekend to work on it, I ought to get more than a "thank you" if the project goes on to make hundreds of millions of dollars.

And, I was just saying there needs to be some way to allow me to work on open source software during that time. I don't know how the details would work, but there surely is an equitable way to deal with it. I certainly could have explained it better.

In any case, I don't expect that 20% of my time spent away from my main project translates to the project taking 20% longer, or 20% less profit or revenue or productivity for my company. The degradation may be linear if we are data entry clerks - if all we are doing is typing instructions given to us from on high (if the work is physical). But that's not typically what programming is about.

I'm not running a company though, and I've not done a study about such companies. If I were to do such a study, that would be my hypothesis, and what follows is what I would expect to see.

The great divide between average a good programmers is well-known. Let us suppose that an average programmer is being paid sixty thousand dollars a year. Let us also suppose that you have a team of good programmers at the lower end of the "good" scale - those who are 10 times more productive than their average counterparts.

Are you paying them six hundred thousand dollars per year? I'd be surprised if anyone said yes to that question. You may pay them more than average programmers, but not an order of magnitude more. Even if they are only twice as productive as the average ones, you aren't likely to be paying them enough to worry about 20 percent of their time compared to what you would be paying less-competent people and getting out of them.

But what does that have to do with anything? Suppose I don't have good programmers - what then? If all of your programmers are average or worse, 20% time is not justified in their salaries the same way it is for a good programmer. So what benefits would we expect to see for companies who provide some amount of unrestricted free time?

First, I think the cases where a developer does something unrelated with his free time will be rare. But even if what I'm choosing to work on in my time is the polar opposite of the company's direction, does the company still benefit from it?

I think so. Here are some ideas:

The company can list projects it sponsors by paying developers to work on them. We've seen examples of companies hiring developers to work on open source projects. This has been especially prevalent in the Ruby community lately, where Sun hired the JRuby guys to work on JRuby, Microsoft hired John Lam to work on IronRuby, and Engine Yard hired Rubinius developers. Relevance uses a lot of open source software in their solutions, so to give back, they have Open Source Fridays.

I would expect that allowing your developers that free time to work on anything would bring better developers your way. If I'm looking for a job and one company offers me unrestricted free time plus the same salary as you offer, who do you think will be my first choice?
If your work is generally not all that interesting, letting your programmers have a weekly break to work on things that are more interesting will help retain them and keep their minds fresh for the real work. As I've noted before, I almost quit the industry altogether when I got stuck in the boring-development cycle.
Working on open source software can make your existing developers better by interacting with other good developers and learning new things.
Your developers are learning things and keeping up with the newest technology, allowing you to react to changes in the market place quicker than your competitors. (You might not make iPhone applications at the moment, but if I spent a couple of Fridays digging into the iPhone SDK, you might already have an application ready to go.)
You might spin off new companies or departments based on products your employees are writing in their free time.

When I first read the reactions, I thought I did go overboard. But after some more consideration, I'm not so sure. Of course, we need a way to quantify and test these hypotheses to be sure what type of a positive or negative impact such a thing has on companies. But, I think it's very safe to say that going to a four day work week does not automatically mean we've lost 20% in one way or another. In the best of cases, it could mean gains in productivity if it allows you to attract better developers.

What do you think? I assume we can agree that a four day work week for thought workers would not necessarily translate into getting only 80% of the value they would normally output. I'm more interested in hearing what gains or drawbacks you can see in having employees take personal programming Fridays, or something like it.

For those who disagree with the idea entirely, what if we restrict it to open source projects the company utilizes in day-to-day work? What if we restrict it to projects the company can use as infrastructure or turn into products (as I understand Google's to work)? Would you feel better about it then?

Motivation

I recently decided to cut the cord on cable and downloaded PlayOn to do the job of serving video content from the internet to my Xbox 360 and Wii, so I could watch it on my TV. Naturally, this led me to figure out how it worked, and how I could play around with it.

One thing I immediately wanted to do was improve the organization functionality. I thought I'd start with a plugin that simply takes the media server's location of video output and provides it as input back to the same server, but in a different organization scheme. Unfortunately, that didn't work as PlayOn didn't know what to do with its own URIs as input for a VideoResource in the Lua plugin API.

I didn't want to get into the scraping, transcoding, and serving of files from the internet -- that's already done well and I didn't want to spend all that time creating a media server of my own. But I did want to see better fastforward and rewind capability. To solve that, I thought I'd create a DVR for PlayOn (or TVersity, or any other media server really) and knock out both the organization features I wanted, along with the seek functionality.

Launching a business

This will be my first attempt at launching a money-making venture online (aside from programming for other people's money-making ventures). I don't expect this will turn into a full-time job, nor do I expect I'll make retirement money off of it, but I think it can make a decent return that won't require a ton of ongoing work on my part, and it might make a fun experiment.

More...

Just as well, I thought it could make for some interesting blogging for me, and reading for you. Here's how I plan to begin:

I'm starting with a simple landing page that asks for your email address, telling potential customers we'll let them know when it's ready. Cost: $22.00 or so for the .net and .com domains. Email list is free to start out at MailChimp, and I'll only need to pay them if I generate more than 500 signups and want to email those folks (which of course, I will).

The reason behind the landing page and email list is to gauge interest before I start. Right now I've only tested the technological feasibility of my idea by trying out all the moving parts needed to make it work. It's not all in one code base (some of it isn't even code), and there's no UI to speak of.

If there's a decent amount of interest, then I'll put it all together with an easy to use interface, and I'll probably devote more time to it in the short run. If it looks like me and my tech savvy friends will be the only users, I'll probably still release it, but you might configure it with text files instead of WinForms until I get sick of doing that and build the UI for it later down the line. I'm pretty sure that would be a free version. =)

So by going the landing page + email list route, I only invest a little time and money up front, until I can see what the demand will be like. I seem to remember hearing that dropbox started similarly, and I'm sure I must have heard this tactic mentioned a hundred times on Mixergy, where Andrew Warner interviews entrepreneurs about how they've been successful (and sometimes not-so).

On to pricing: I'm thinking of initially starting it out at a one-time fee of $35 (US), but that may change. More than likely, I'll test different pricing schemes and see what works well. I've got $105 of credit in Google Adwords, and I thought I might use that to do some A/B testing on the signup page. Half of the users would see Test which A might mention the one-time $35 price point, while the other half see Test B, which may be $5 per month. Hopefully, this would inform us as to which one results in more signups to the email list.

However, I decided not to do that at this point. It's just more up front work that has to be put into the idea. Instead, I'll probably wait to see if anyone signs up for the list, then do something similar to test which pricing options are better than others (using the email list instead of the website).

As I mentioned above, the DVR is not ready for public consumption yet, but you can sign up for the email list on MediaServerX if you want to be notified when it's ready.

If this little business experiment generates any action on my part aside from "well that flopped, I give up," I'll keep you updated here on the blog.

Deciding on technology

At first I thought I might like to really dig deep and write something from scratch, so the world was open for me to be cross platform and there were flowers in the fields with unicorns and rainbows. However, after taking a look at the UPnP spec documents, I realized I didn't really care about that rainbow and what I really wanted was to just write some code.

So I moved on, thinking I'd use Developer Tools for UpNP Technologies to generate the beginnings of a control point. After messing around with it for a little while and reading that Casey Chesnut "would probably try to use the MS stack next time," I decided to go with the UPnP Control Point API that's been included on Windows since Millenium Edition was all the flop.

It seemed like the lowest barrier to entry, which is something I haven't heard in a while about programming and Microsoft.

Code

First I created a solution in Visual Studio and included a reference for the UPnP 1.0 Type Library (Control Point). It can be found by right-clicking on References, then clicking on Add Reference. It's under the COM tab.

To start out doing anything, you can find a UPnP device using the UPnPDeviceFinder class. The code below will find all devices listed as MediaServer:1 (there are also :2 and :3, but I have yet to look into the differences between them).

UPnPDeviceFinder deviceFinder = new UPnPDeviceFinder(); 
UPnPDevices mediaServerDevices = deviceFinder.FindByType("urn:schemas-upnp-org:device:MediaServer:1", 0); 
foreach (UPnPDevice mediaServerDevice in mediaServerDevices)
{ 
    // do something with the UPnPDevice 
}

The basic object model consists of a Device Finder (seen above) which can get you a collection of individual devices. Individual devices have a collection of services. A service is what you need to communicate with to control and get information from the device. (Here is a diagram).

In our case, we want to get the MediaServer's ContentDirectory service to give us a list of ContentDirectories and their children.

UPnPService cdService = mediaServerDevice.Services["urn:upnp-org:serviceId:ContentDirectory"];

Now that you have the content directrory service, you can invoke actions on it. For example, to see what it contains you can use the browse service:

string browseFlag = "BrowseDirectChildren"; // BrowseDirectChildren or BrowseMetadata as allowed values
string filter = "";  
int startingIndex = 0; 
int requestedCount = 1000;
string sortCriteria = "";

object[] inArgs = new object[6]; 
inArgs[0] = objectID; // use 0 for root
inArgs[1] = browseFlag; 
inArgs[2] = filter; 
inArgs[3] = startingIndex; 
inArgs[4] = requestedCount; 
inArgs[5] = sortCriteria; 

object outArgs = new object[4]; 
cdService.InvokeAction("Browse", inArgs, ref outArgs); 
object[] resultobj = (object[])outArgs; 

string result; 
int numberReturned; 
int totalMatches; 
int updateID; 

result = (string) resultobj[0];
numberReturned = (int)(UInt32) resultobj[1]; 
totalMatches = (int)(UInt32) resultobj[2]; 
updateID = (int)(UInt32) resultobj[3];

This where it really starts to get wieldy. I find it a complete PITA to be sending and receiving basic objects and casting them back and forth to their desired types. The result string ensures more fun searching through XML to get what you want. It's nothing we want to be doing throughout our source code, because it's a ton of noise to sort through. Side note: Setting the objectID to 0 will get you the root element. After that, the results report back their IDs so you can use what it returns. That took me forever to find out, so hopefully it will save you some time.

Obviously, this sort of hassle calls for some abstraction, so I began writing a wrapper library. So far it works through everything I've shown above, and includes a sample project showing how to use it.

Source code: upnplib-mediaserver-wrapper

It's a Visual Studio 2008 solution with 2 projects: the upnplib-mediaserver-wrapper is the library, and example-upnp-mediaserver-browser is an example of how to use the library. You'll need to set that one up as your startup project for it to run. I'd have guessed it would be in the .sln file but it's not, and I don't want to include my user-specific settings in the source control.

Let me know if you're interested in seeing it become more complete.

If you're a fan of PlayOn or TVersity and want a DVR service, click the preceding link to sign up and be notified when it's ready.

As always, I'd love to hear your thoughts or questions on the subject.

I'm writing a Client of FooServer which is reliant upon a rather clunky library whose functionality I'd like to encapsulate in a wrapper library to make the usage less clunky.

My first thought is to choose FooServer as the name of what will likely be the most used class in the library. That way, when I want to get some property of the server or tell it to perform some action I can do something like:

FooServer foo = FooServer.findById(12);

string property = foo.property;

foo.performAction();

That seems innocent enough. But my fear is that by calling the class FooServer, it may violate the principle of least surprise because it does not fully implement a Foo server, nor would it be of use in implementing a Foo server.

I also dislike tagging it with "Wrapper" or something of the sort, because reading that in the client code doesn't seem right either.

I know I'm probably over-thinking it, but names are important to me, because they are the start of clean, readable code that does what it says and says what it does. So that's why I come to you, dear reader.

What would you name the class?

Rails Rumble has nothing on this.

Of course, you could just click the edit button in your database management studio of choice and achieve the same functionality.

SELECT DISTINCT 'script/generate scaffold ' + t.name + ' ' + column_names
FROM sys.tables t
CROSS APPLY (
    SELECT c.name + 
           case when max_length > 255 then ':text' else ':string' end + ' '
    FROM sys.columns c
    WHERE c.object_id = t.object_id
    ORDER BY c.column_id
    FOR XML PATH('') ) dummy_identifier ( column_names )

A similar discovery was made in the 1930's. One important difference to note is that, since my program does not simulate the input on it's output program, I am able to achieve speeds that are logarithmically faster than what Turing could accomplish.

It seems easy to add an if statement the first time you need to deal with a difference between legacy data and new data models. It's still easy the second and third times.

Certainly it's easier than transforming the legacy data to fit the new schema.

Induction doesn't hold though. At some point, it becomes a tangled mess of code that has to deal with too many conditions and your mental model of it turns into total disarray.

This is one case where laziness and instant gratification can steer you in the wrong direction. Avoid the spaghetti; just transform the old data like you should have in the first place.

I was first introduced to the concept that choice could be a bad thing back in 2007 at NFJS in Austin, during Scott Davis's keynote.

It might have been obvious to me before then, had I paid attention to my own thoughts and actions. Instead, it took a little prodding and the knowledge that others experienced the same things I did to get me thinking about it.

I was reintroduced to the concept while listening to an episode of ~~my favorite podcast~~ what might be the best radio show on the planet, titled "Choice." Taking it in while traveling nowhere on my elliptical machine, I realized I had to share the podcast and its lessons with the rest of you.

The theme weaved by the lessons is that choice is not all it's cracked up to be. For programmers, I think this is especially so.

More...

When I was studying practice test questions for Exam 70-431 last week, the type of questions and answers I read led me to the thought that certifications attempt to commodify knowledge and use it in place of thought. More...

Through the last forty-plus weeks, we've explored looking at yourself as a product and service, different ways of positioning yourself in the programming market, ways to invest in yourself, how to execute decisions to be a better programmer, and several options you have for marketing yourself, guided by Chad Fowler's excellent book, My Job Went To India.

Today we'll start looking at the "Maintaining Your Edge" section of the book. More...

I put faith in web application development as an income source like I put faith in the United States Social Security system. That is to say, it's there now, but I don't expect to be able to rely on it in its current incarnation very far into the future.

James Maguire quotes Robert Dewar hitting the nail on the head: More...

I didn't expect many people to agree with me on the issue of web application development walking the plank.

Why? More...

Even though the practice of developing a specific piece of software is better enjoyed as a journey than as a goal, the same is not necessarily true when looking at your career as a whole.

More...

Automatic download of libraries and packages
I'd like to see automatic downloads of libraries/packages from IDEs.

The idea came to me a couple of weeks ago on twitter.

What if our programming environment automatically checked for libraries?

What if our programming environment automatically checked for libraries?

Suppose I decide I want to use ActiveRecord, but I don't have it installed on my system. Wouldn't it be nice to just type

More...

The Questions
Daniel Spiewak asks, "how do you apply polyglotism?"

Daniel mentions Google, "one of the most open-minded and developer friendly companies around," and points out that they have a strict limit in languages to use: Python, Java, C++, and JavaScript. He also says,

To my knowledge, this sort of policy is fairly common in the industry. Companies (particularly those employing consultants) seem to prefer to keep the technologies employed to a minimum, focusing on the least-common denominator so as to reduce the requirements for incoming developer skill sets.

We're afraid of being eaten by the poly-headed polyglot monster. More...

Some people call them fat pants. Some people call them stretch pants. Others might call them exercise pants or sweat pants. Whatever you call them, they're comfortable to wear. The problem with sweat pants is the reason they're comfortable is because they're big and expandable. And that expandability means they have a lot of room to accommodate growth as well. More...

If you want to trap a monkey, it's not very hard. Hollow out a hole (in a coconut, the ground, or whatever) just large enough for a monkey's hand to fit in when relaxed, but too small to pull out as a fist. Put some food in the hole, and wait. Soon enough, a monkey will come, fall in love with the food, grab at it and refuse to let go.

You see, monkeys value food higher than life or freedom, and their devotion to it will not allow them to let go. Or so the story of the south Indian monkey trap goes. (I am merely relating the parable, I have not actually tried to capture a monkey in this manner.) More...

Outsourcing is not going away. You can delude yourself with myths of poor quality and miscommunication all you want, but the fact remains that people are solving those problems and making outsourcing work.

As Chad Fowler points out in the intro to the section of MJWTI titled "If You Can't Beat 'Em", when a company decides to outsource - it's often a strategic decision after much deliberation. Few companies (at least responsible ones) would choose to outsource by the seat of their pants, and then change their minds later. (It may be the case that we'll see some reversal, and then more, and then less, ... , until an equilibrium is reached - this is still new territory for most people, I would think.)

Graph showing much more growth in IT outsourcing with statistics from 2006.

Graph showing much more growth in IT outsourcing with statistics from 2006.

Source

Chad explains the situation where he was sent to India to help improve the offshore team there:

If [the American team members] were so good, and the Indian team was so "green," why the hell couldn't they make the Indian team better? Why was it that, even with me in India helping, the U.S.-based software architects weren't making a dent in the collective skill level of the software developers in Bangalore?

The answer was obvious. They didn't want to. As much as they professed to want our software development practices to be sound, our code to be great, and our people to be stars, they didn't lift a finger to make it so.

These people's jobs weren't at risk. They were just resentful. They were holding out, waiting for the day they could say "I told you so," then come in and pick up after management's mess-making offshore excursions.

But that day didn't come. And it won't.

The world is becoming more "interconnected," and information and talent crosses borders easier than it has in the past. And it's not something unique to information technologists - though it may be the easiest to pull-off in that realm.

So while you lament that people are losing their jobs to cheap labor and then demand higher minimum wages, also keep in mind that you should be trying to do something about it. You're not going to reverse the outsourcing trend with any more success than record companies and movie studios are going to have stopping peer-to-peer file sharing.

That's right. In the fight over outsourcing, you, the high-paid programmer, are the big bad RIAA and those participating in the outsourcing are the Napsters. They may have succeeded in shutting down Napster, but in the fight against the idea of Napster, they've had as much strategic success as the War on Drugs (that is to say, very little, if any). Instead of fighting it, you need to find a way to accept it and profit from it - or at least work within the new parameters.

How can you work within the new parameters? One way is to "Manage 'Em." Chad describes several characteristics that you need to have to be successful with an offshore development team, which culminates in a "new kind" of PM:

What I've just described is a project manager. But it's a new kind of project manager with a new set of skills. It's a project manager who must act at a different level of intensity than the project managers of the past. This project manager needs to have strong organizational, functional, and technical skills to be successful. This project manager, unlike those on most onsite projects, is an absolutely essential role for the success of an offshore-developed project.

This project manager possesses a set of skills and innate abilities that are hard to come by and are in increasingly high demand.

It could be you.

Will it be?

Chad suggests learning to write "clear, complete functional and technical specifications," and knowing how to write use cases and use UML. These sorts of things aren't flavor-of-the-month with Agile Development, but in this context, Agile is going to be hard to implement "by the book."

Anyway, I'm interested in your thoughts on outsourcing, any insecurities you feel about it, and what you plan to do about them (if anything). (This is open invitation for outsourcers and outsourcees too!) You're of course welcome to say whatever else is on you mind.

So, what do you think?

Side Note: Posted Thursday instead of Friday because I'm off to Lone Star Ruby Conference in the morning. Hit me up on twitter or contact me if you're going to be there.

If we accept the notion that we need to figure out how to work with outsourcing because it's more likely to increase than decrease or stagnate, then it would be beneficial for us to become "Distributed Software Development Experts" (Fowler, pg 169).

To do that, you need to overcome challenges associated with non-colocated teams that exceed those experienced by teams who work in the same geographic location. Chad lists a few of them in this week's advice from My Job Went To India (I'm not quoting): More...

I was having trouble with dragging and dropping elements using Scriptaculous's dragdrop.js. Apparently, I'm not the only one. The problem was:

I have a div with overflow = auto, so when there is more content than the size of the div, scrollbars appear. My draggables and droppables are all elements inside of that div. Everything works fine when the scrollbar is scrolled all the way to the top, but when you scroll it down any amount, the draggables fail.

More...

There are plenty of potential solutions out there, but none of them worked for me. I would get the scrolling working, and then the draggable would move away from the cursor. I'd get it in sync with the mouse cursor and the scrolling would get crazy again. I'd fix that and then no matter where I dropped it, if the div had been scrolled, dropping would fail.

Here's how I fixed the problem:

In the Draggable#updateDrag function (~ line 356), on the first line, I changed the value of the pointer argument to take into account how much the container had scrolled:

pointer = new Array(pointer[0] + this.options.scroll.scrollLeft, pointer[1] + this.options.scroll.scrollTop);

At least one of the solutions I recall seeing mentioned this.
In the same function, I changed the first two elements in the p array before the last two elements get pushed onto it:

p = new Array(p[0] + this.options.scroll.scrollLeft, p[1] + this.options.scroll.scrollTop);

This also just takes into account how far the container has been scrolled.
To ensure my droppables were able to receive the draggables given the adjusted coordinates, we need to adjust the scroll position just as we did above. First, I adjusted the code in my webpage that produces the droppables and added a scroll parameter that should be the name of the element that scrolls (the same parameter the draggable accepts):

Droppables.add('slot_1_7_2', { scroll: 'weeklycalendar' });

Since Droppable elements don't generally take a scroll option, we'll need to modify that code in Scriptaculous's dragdrop.js file as well. In the Droppable#fire function (~ line 109) add the folling lines under Position.prepare();:

var point = [Event.pointerX(event), Event.pointerY(event)]; if(this.last_active.scroll){ point[0] += $(this.last_active.scroll).scrollLeft; point[1] += $(this.last_active.scroll).scrollTop; }
Finally, just underneath that where it calls this.isAffected, change the first parameter from [Event.pointerX(event), Event.pointerY(event)] to use the variable we created above, pointer.

That should be it. If you've tried the above and still get problems, feel free to leave a comment below, or contact me and I'll do my best to help out.

I haven't submitted a patch because I didn't check to see that this was a general solution. It seems like it should be, but without testing it outside my intended usage, I don't think it'd be accepted anyway. Quite frankly, I'm not thrilled about adding a new option to droppables, but it seemed like the simplest route to fix my problem at the time.

Yes, I tried setting includeScrollOffsets to true and using Position#withinIncludingScrolloffsets in Prototype, and that failed for me too.

You might think that "tech support" is a solved problem. You're probably right. Someone has solved it and written down The General Procedures For Troubleshooting and How To Give Good Tech Support. However, surprisingly enough, not everyone has learned these lessons. And if the manual exists, I can't seem to find it so I can RTFthing.

The titles of the two unheard of holy books I mentioned above might seem at first glance to be different tales. After all, troubleshooting is a broad topic applicable to any kind of problem-solving from chemistry to mechanical engineering to computer and biological science. Tech support is the lowliest of Lowly Worms for top-of-the-food-chain programmers.

More...

I've spent some time recently building a tool that makes my life a bit easier.

Finishing up a simple Rails log filter. Tired of tracking down sessions for support requests and it takes too long even for support kids.

Finishing up a simple Rails log filter. Tired of tracking down sessions for support requests and it takes too long even for support kids.

I've browsed plenty of Rails log analyzers that help me find performance bottlenecks and potential improvements. But what I really need is a faster way to filter my logs to trace user sessions for support purposes. Maybe it's just me, but I've got apps where users report problems that make no sense, where their data gets lost, and who can't tell me what they did. Add to that the fact that I've got the same app running on dozens of different sites, and you can see why performance analyzers aren't what I'm looking for to solve my problem.

Because of that, I need a solution that lets me filter down and search parameters to figure out what a particular user was doing on a certain date. Hence, Ties. More...

Shitty variable names, unnecessary usage of pointer arithmetic, and clever tricks are WTFs.

When I saw this WTF code that formats numbers to add commas, I had forgotten how low-level C really is. Even after using C++ fairly regularly for the last several months.

With 500+ upvotes, the people over at the reddit thread regarding the WTF apparently think the real WTF is the post, not the code.

nice_code, stupid_submitter - in which TheDailyWTF jumps the shark by ridiculing perfectly good code.

Let's forgive the misuse of the wornout phrase and get to whether or not looking at the code should result in utterance of WTFs. More...

It goes something like this:

Setup, in which we use globally shared memory
Negate the number if it is negative, for no reason I can think of, and set a reminder flag.
Set buffer to point to the final spot. Move it back one, dereference it, and insert the null string terminator there.
Move the buffer backwards. Dereference the pointer and set the value there to the character '0'. Add the remainder of the number divided by ten to that, since '1' is 1 away from '0', and so forth.
Divide the number by 10 so we can get to the next digit.
If we've done the loop 3 times, move the buffer pointer back one location and insert a comma.
Repeat starting at step 4 until the number is 0.
Cleanup - if the front character is a comma, remove it by moving the buffer pointer forward.
Cleanup - Move the buffer pointer backwards and insert our '-' if the negative flag has been set.

I felt like that required too much brain power to follow for doing something so simple. So I decided to make my own version. I thought I'd try the following:

Copy the number to a string.
Starting at the end, copy each character to another string, inserting a comma every 3rd time.

A thousand times simpler than the convoluted mess of nice_num. Here's that attempt, in C:

char* commadify(long num, char* result) {

  int input_len = num_digits(num);
  char* input_number = malloc( input_len * sizeof(char) );

  sprintf(input_number, "%ld", num);
  
  int number_of_commas = (input_len-1) / 3;

  
  int result_len = input_len + number_of_commas;
  
  int input_index = input_len-1;

  int result_index, count=0;
  
  for (result_index=result_len-1; result_index>=0; result_index--) {

    if( count == 3 ) {
      result[result_index] = ',';

      result_index --;
      count = 0;
    }
    
    result[result_index] = input_number[input_index];

    input_index --;
    count++;
  }
  
  free(input_number);

  return result;
}

I think it's clearer - but not by much. Certainly the variable names are better because you don't first have to first understand what's going on to know what they are for. I think moving the pointer arithmetic back into the more-familiar array notation helps understandability. And removing the trick of knowing that the ASCII codes for '1' is 1 more than for '0' ... for '9' is 9 more than 0 means less thinking.

Commadify is 4 times slower than nice_num

Commadify is 4 times slower than nice_num

On the negative side, the commadify code is slower than nice_num, most of which is caused by using malloc instead of having preallocated memory. Removing those two instances and using preallocated memory shaves a couple of tenths of a second off of the one-million-runs loop. But you have to compensate with more code that keeps track of the start position.

So what's the verdict? I don't think we're reacting to the WTFs I mentioned above when we see the nice_num code. I think we're reacting to C itself. We're so used to very high level languages that common tricks and things you'd know as a C programmer are the WTFs to us.

This kind of stuff isn't outside the realm of what a strong programmer should know. It isn't even close to the border. The truth is our low-level skills are out of practice and we should probably get some.

What do you think?

Code with main program and comparison with nice_num is available at my github repository, miscellany/commadify.

Update: Thanks to Dave Kirby's comment, I've fixed memory leaks and updated the code in this post and at the repository. Link to repo originally was to the specific commit - so I've changed that to link to master instead.

This post might be better titled, "How (and how not) to help yourself when Google doesn't have the answer: A whirlwind tour through Rails' source" if only I wasn't too lazy to change the max length of the database field for titles to my blog entries.

Google sometimes seems as if it has the sum of all human knowledge within the confines of its search index. It might even be the case that it does. Even if you prefer to think that's true, there may come a time when humanity does not yet have the knowledge you are seeking.

How often is that going to happen? Surely someone has run up against the problems I'm going to have, right? That hasn't been the case for me the last couple of months.

I may be the only developer writing Rails apps on MacOSX to be deployed to the world on Windows where SQL Server 2008 is the backend to a Sharepoint install used by internal staff to drive the data. I'm not so presumptious to think I'm a beautiful and unique snowflake, but I wasn't finding any answers. More...

Before I started this trek, I made a commitment to leave after an hour if I found my attention drifting toward something else. I never started checking email, reading blogs, or obsessively reloading twitter to see my tweeps latest tweets, so I thought I was in the clear.

However, even though I felt like I was focused, the fact that I had been sitting at the computer for so long contributed to poor decision making. The first of these was to keep searching Google even though every search was coming up useless. I always followed the path of least resistance - even if it wasn't going to get me to the goal quicker than an alternative path. If it was less challenging, it was for me.

What is a binary column's data type in Ruby?

What is a binary column's data type in Ruby?

After a while, I ran out of mentally casual paths and resigned myself to tracing through the source code (it is open source, after all, and this is one of the benefits everyone claims but so few practice). It was what I knew I should have been doing as I started out, and I had wasted several hours trying to tip-toe around it for the sake of my poor, tired brain.

Indeed, the binary column from Active Record does resolve to a Ruby String.

Indeed, the binary column from Active Record does resolve to a Ruby String.

Now that I was sure I had the right data type being returned, I needed to narrow down where the problem was occuring. I knew SQLServerAdapter was using DBI to connect to the database, so I figured I'd use a quick irb session to test DBI itself. The test came back negative - DBI was getting the correct data. I also ran a quick test in the Rails app, reaching through ActiveRecord::Base to use the connection manually. That worked, as expected.

Using DBI I got the correct data, checking SQLServerAdapter because Rails sucks on Windows

Using DBI I got the correct data, checking SQLServerAdapter because Rails sucks on Windows

I had thought, and now confirmed, that the best place to look would be SQLServerAdapter. If it were a Rails problem, certainly someone would have run into it by now. So it made sense the problem would be in the interface between Rails and Microsoft. Why? Because if Rails is a Ghetto, Rails with Microsoft products is a fucking concentration camp.

Excuse the profanity. I don't often use it here, so you know I must mean it when I do.

As I began to browse the source, I was first drawn to this innocent looking code from rails-sqlserver on github:

class << self

  def string_to_binary(value)
    "0x#{value.unpack("H*")[0]}"
  end

        
  def binary_to_string(value)
    value =~ /[^[:xdigit:]]/ ? value : [value].pack('H*')

  end
end

But it wasn't obvious how it was being used elsewhere. I even tried using the reverse operations in my objects - to no avail. And after searching in the source file, it certainly wasn't being called inside of SQLServerAdapter. So I went on a quest for the answer inside

/opt/local/lib/ruby/gems/1.8/gems/activerecord-2.1.1/

.

For quite some time I went back and forth inserting and removing debugging code between Active Record and SQLServerAdapter. select(sql, name=nil) is a protected method defined in the abstract connection adapter in Active Record. SQLServerAdapter implements it privately, and it was both getting and returning the correct data.

After ActiveRecord calls object.instance_variable_set("@attributes", record) when instantiating our object, object.attributes[binary].size becomes less than record["binary"].size. That was the problem. I thought for sure instance_variable_set was a monkeypatched method on Object, and that all I needed to do was issue a monkeyfix and all would be well.

Only I was wrong. It's there by default in Ruby, and Rails wasn't monkeypatching it (that I can tell).

All the sudden things started looking bleak. By this time I knew how I could fix it as a hack. I even had a nice little monkeypatch for my objects that I could issue and have it feel less hacky to be used. I had given up.

But for some reason I picked it back up after an hour and found that ActiveRecord was actually calling that string to binary method in SQL Server Adapter. It allows them to register calls that should happen before defining the read and write methods on the object. Excellent!

I opened up SQLServerAdapater, there it was: a different binary_to_string method that totally explained it. The pair in this version were encoding and decoding the data to/from base 64. That would work fine, if my data was going through the encoding part. But it wasn't - it was coming straight from Sharepoint.

And so the tour ends.

There's a comment in the code about some arbitrary 7k limit on data size from SQL Server being the reason for encoding as base 64 before inserting the data. I don't know about inserting, but retrieving works fine without it. If I could think of a way to distinguish, I'd submit a patch for the patchwork. Ideally, I'd rather find a way around the restriction, if it actually exists.

The original code I was looking at was on github. It (not surprisingly) differed from the code in use on my machine. Another side effect of the 16 hour monitor stare.

It's called the 8 hour burn for a reason. The only things burning after 16 hours are your brain from all the stimulants and your wife, wondering WTF you're really doing because there's no way you're working from before she wakes up until after she goes to bed.

What's the point?

There's two, if you ask me:

You have the source code. Look through it. You have no other choice when no one has had your problem, and you might benefit by doing so even if someone already has.
Even when you think you're focused working late, and resolve to leave when you lose the focus, you're still going to make stupid decisions that you won't notice until the morning. I turned a 5 hour journey into a 12 hour marathon. Sleep, FTW.

Oh, and Rails on Windows is a concentration camp.

This is the first in a series of my answers to Jurgen Appelo's list of 100 Interview Questions for Software Developers. Jurgen is a software development manager and CIO at ISM eCompany, according to his About page.

The list is not intended to be a "one-size-fits-all, every developer must know the correct answer to all questions" list. Instead, Jurgen notes:

The key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills.
...
This list covers most of the knowledge areas as defined by the Software Engineering Body of Knowledge. Of course, if you're just looking for brilliant programmers, you may want to limit the topics to [the] Construction, Algorithms, Data Structures and Testing [sections of the list]. And if you're looking for architects, you can just consider the questions under the headings Requirements, Functional Design and Technical Design.

But whatever you do, keep this in mind:
For most of the questions in this list there are no right and wrong answers!

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. The format will first list the question, my initial response (to start the discussion), followed by a place I might have looked for further information had I seen the questions and prepared before answering them.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy. More...

Requirements

Can you name a number of non-functional (or quality) requirements?
I'd first mention performance and security, from the user's perspective. I'd then mention meeting minimum requirements for metrics like code coverage in testing and dependencies in our design. I don't consider code quality out-of-order when it comes to requirements.

The non-functional requirements page at Wikipedia lists several examples. Notable exceptions from my quickie-response: accessibility, documentation, portability. There are several that are listed that I consider covered by what I've listed, but I missed some that caused me to say
What is your advice when a customer wants high performance, high usability and high security?
My "advice" starts with the questions: "What do you consider X to be?" where X belongs to the set {"high performance", "high usability", "high security"}. I might offer that I consider "high performance" to be a misnomer, as it's either acceptable or not, and that unless the customer defines it, I don't know how we'd even attempt to measure something as vague as "usability."

I'm not sure where I'd prepare for this question. Any suggestions are appreciated.
Can you name a number of different techniques for specifying requirements? What works best in which case?
I can name several: tell me in person, tell me over email, tell me over IM or over the phone. I know that's not what you're looking for. You're looking for answers like "use cases." It all boils down to the same. I might even mention "unit tests" here. That's part of the spec, as far as I'm concerned, and for almost any software I write for myself, it's the only way I specify requirements (except for maybe a very informal to-do list).

Face-to-face works best in most cases, I'd gather.

The answer after doing some research: ¡Ay, ay, ay, no me gusta! I didn't see this coming. There are a number of things that could qualify as answers (Prototyping, Storyboards, Modeling, ..., State transitions) that I knew about beforehand. I thought to include none of them.
What is requirements tracing? What is backward tracing vs. forward tracing?
My response? "I don't know anything about requirements tracing. I'm willing to learn."

Once again, Wikipedia tells us about requirements traceability, illuminating the issue for us.
Which tools do you like to use for keeping track of requirements?
I generally use a combination of a text file and emails, as far as the client is concerned. If it's a larger system, I'll use something like Sharepoint, Basecamp, or another system that performs a similar function. I have no preferences, because nothing I've ever used compares to a simple list. If it does, it's equally useful.

I don't know that I'd say I like any of them. In reality I prefer a simple to-do list that I encode in tests (insofar as I'm capable of writing the tests) and knock them out one-by-one.
How do you treat changing requirements? Are they good or bad? Why?
I don't give a value judgement on changing requirements: they are inevitable. They can be good or bad depending on how the client handles them.

I always try to let the client know: I can do X amount in Y timeframe. You asked for Z total. Here's an estimate for each item in Z, pick X from it for our current Y timeframe. I'll get back to you every Y timeframe to show a demo, and you can choose X from the remaining Z again (with changes based on circumstances if required). Feel free to fire me when you have enough out of Z that's functional. (Ok, I probably wouldn't say the last sentence in those terms, but I'd find a way to say it, if for no other reason than to sell them the rest of the process.)

As far as looking it up before the interview: I've review Agile literature. Searching any of the agile yahoo groups for the question at hand ought to be good enough.
How do you search and find requirements? What are possible sources?
"What do you mean?" would be my response. I really don't know. What does searching and finding requirements mean? Does it mean figuring out how to do requirements that I don't know how to accomplish?

If so, the first step is Google. In the rare event no one that I can find has had my problem, I'll go traipsing through the source code if it's available.

Clearly, I don't know where to start here.
How do you prioritize requirements? Do you know different techniques?
I rarely prioritize requirements. I let the customer decide. I give them a relative cost of implementing X requirement vs. implementing Y requirement, and let them decide. If Y requires X, then I tell them so.

I know of different techniques - take "random" for example. I don't know what they might be called. But I cannot think of anything better, even if it were decreed as a Top 10 Commandment for Prioritizing Requirements.

No web search for this comes to mind. I'd review a couple of process management books if I had no clue. This seems to be a decent discussion, if you must have one from me from a cursory browse.
Can you name the responsibilities of the user, the customer and the developer in the requirements process?
The user will be the person using the software, versus the customer being the one who pays for its development. I hate that distinction. The developer programs it. Responsibilities? In my ideal organization, I'd have:
- Developers working with the Customer to manage requirements.
- Developers working with the User to make the application work for them regardless of the Customer (I've seen too many projects where the User had to use whatever the Customer purchased, even if the purchase was ... little yellow bus-ish.)
- Customer/User having daily meetings with the developer
- Developer making the best software he/she can given the constraints.
Again, I don't know where I'd look this up before being asked. Suggestions (again, as always) are most welcome.
What do you do with requirements that are incomplete or incomprehensible?
I send an email saying "I don't understand what you mean. Please read the very small attached book and get back to me."

Just kidding, of course (at least in most cases. I've been recently tempted to send that exact email, as it happens.)

I just ask them to clarify. If I don't have contact with the customer, I ask the intermediary to clarify or get clarification.

Outside reading: Agile, and hopefully other processes for some compare and contrast.

I think these are decent answers to start a discussion with. If you're a hiring manager, what do you say? Would you show me the door, or keep me around for a while longer?

It's not quite as hard as Steve Yegge's list of things to know (I'll get to that eventually), but it's a good (and more well-rounded!) list nevertheless.

How would you answer the Requirements questions?

This is the second in a series of answers to 100 Interview Questions for Software Developers.

The list is not intended to be a "one-size-fits-all" list. Instead, "the key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills." Even still, "for most of the questions in this list there are no right and wrong answers!"

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. Here's that attempt.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy.

My lack of understanding the angle from which these questions come may influence other seemingly off-base answers. Help me out by explaining how you might answer these questions on functional design: More...

What are metaphors used for in functional design? Can you name some successful examples?
I hate to start the first one off with an IDK, but what else can you do? I feel like I can talk intelligently about some of the issues involved in functional design, but until now, I never realized it was a process unto itself, nor did I know what all it entailed until after I browsed the remaining questions.

Reading: The Wikipedia page on functional design is delightfully lacking any useful information about the topic. However, these functional design guidelines on Smashing Magazine seem to be a reasonable starting point.
How can you reduce the user's perception of waiting when some functions take a lot of time?
Show a progress bar. Distract them with something they need to read or stage data-collection/interaction with the user. Of course, the obvious yet overlooked answer here would be to make the offending functions faster. At worst, if the function doesn't require user interaction at all, you could spawn a new thread to do the work in the background and let the user continue working. Then they'd notice no waiting at all.

Reading: I picked this up from observation and deduction. I don't know where I'd go to read such information.
Which controls would you use when a user must select multiple items from a big list, in a minimal amount of space?
What controls do I have to pick from? Can the list be grouped by some recurring values? For instance, consider how the iPod uses the scroll wheel to let you navigate a long list of music. Now apply finding one song to finding several, and you have an idea of what I'm talking about.

Another option might be to have auto-complete, with a special key for "add to list" when you're satisfied, which puts you back at the beginning of the text box. Another key would end the sequence.

Can you visualize the list in some way? The user could then click pictures to drill down and roll up on items in the list, viewing it from different levels. For example, if the list was of colors, you could take the typical color-picker from every image program out there, and use it to choose your list.

Reading: I came up with these on the spot, and can't say I've seen them completely implemented yet, so again, when it comes time for advice on where to read up on this topic, I've got nothing.
Can you name different measures to guarantee correctness of data entry?
Aside from the pains of torture? You'll definitely want a fast-acting validation to let the user know as soon as possible that you think their input is incorrect. When talking about the web, you'll want to validate it server side as well. You'll want to show some examples if you need a certain format of input (or just if the customer expects to be forced into a format, you'll want to relieve their anxiety by providing an example). Finally, make the user type as little as possible, except if you're doing a progressive lookup / autocomplete on their typing.

Reading list is vacant. Please advise.
Can you name different techniques for prototyping an application?
At first, I didn't realize there were different techniques. Unless we were talking about storyboards vs. mark-up-only or a shell of an application. I don't imagine that was the point.

I did find something that I think was the point: for reading, Scott Ambler has a take from a different point of view: tips and techniques in user interface prototyping that includes another list of resources. A treasure trove!

Now that I know what we're talking about, I'd have given similar answers. I feel I could have been effective in a conversation about this, but the other party would have had to lead me that direction. I couldn't get there from the question alone.
Can you name examples of how an application can anticipate user behavior?
Analyze it and look for patterns. Use a Markov Model to predict next action based on previous N actions.

That's the only way (well, there are variation on the same theme) I can think of that an application would do it. We could of course observe the users and make changes ourselves. I don't think that would be a bad idea.

Reading is nonexistent. Those ideas came from elsewhere, but never as part of functional design.
Can you name different ways of designing access to a large and complex list of features?
There's the outmoded File/Edit/View tree menu controls (find link to discussion), ribbon interfaces, and progressive search, or autocomplete.

I'd also consider re-purposing my answers to #3.
How would you design editing twenty fields for a list of 10 items? And editing 3 fields for a list of 1000 items?
For the latter, I'd almost certainly put them all in one page. Or 100+ at a time. The fewer clicks the better. It's worth importing from a spreadsheet every time, if that's an option.

For the former- Can the 20 fields be broken down into more cohesive units? Twenty fields in one form is often too many. 10 forms is not absurd, on the other hand.
What is the problem of using different colors when highlighting pieces of a text?
The highlight color can make the text hard(er) or downright impossible to read. Anything else?

I don't see room for discussion, unless you're flashing the colors -- then you might get into causing epileptic seizures in those susceptible to them.
Can you name some limitations of a web environment vs. a Windows environment?
Access to the file system. Lag mainly - less responsive experience for most of web than on Windows.

As you can tell - while some of these questions are ones I've thought about in the past, or touch themes on others that I do - I haven't thought about functional design as a process by itself.

So in that regard I ask, Do you have any pointers?

This is the seventh in a series of answers to 100 Interview Questions for Software Developers.

The list is not intended to be a "one-size-fits-all" list. Instead, "the key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills." Even still, "for most of the questions in this list there are no right and wrong answers!"

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. Here's that attempt.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy (and help out where you can!).

This week's answers are about testing. More...

Do you know what a regression test is? How do you verify that new changes have not broken existing features?
You answered the second part of the question with the first: you run regression tests to ensure that new changes have not broken existing features. For me, regression tests come in the form of already written tests, especially unit tests that I've let turn into integration tests. However, you could write a regression test before making a new change, and it would work as well.

The point is that you want to have some tests in place so that when you inevitably make changes, you can ensure they didn't cascade throughout the system introducing bugs.
How can you implement unit testing when there are dependencies between a business layer and a data layer?
Generally I'd let that unit test become an integration test. But if the time to run the tests was becoming too long, I'd build a mock object that represented the data layer without hitting the database or file system, and that would be expected to decrease the running time significantly.
Which tools are essential to you for testing the quality of your code?
I don't know if anything is essential. If you've got asserts or throws, you can always implement testing yourself, and a good eye for bad code helps as well. That said, to reduce psychological barriers to testing, it would be nice to have tools already made for this purpose.

Luckily, we have such tool available: unit testing frameworks and static code analysis tools in your language of choice.
What types of problems have you encountered most often in your products after deployment?
Most recently I've encountered very specific integration errors, and written about some ideas on fixing the polysystemic testing nightmare.
Do you know what code coverage is? What types of code coverage are there?
Generally I'd thought it refers to the percentage of code covered by tests. I don't know what the second question here refers to, as I thought it referred exclusively to testing.
Do you know the difference between functional testing and exploratory testing? How would you test a web site?
I have to admit that before being asked this question, I wouldn't have thought about it. My guess is that functional testing refers to testing the expected functionality of an application, whereas exploratory testing involves testing without knowing any specific expectations.

As far as testing a web site, I'll have plenty of unit tests, some acceptance tests, perhaps some in Selenium or a similar offering, as well as load testing. These aren't specific to web apps, however, except for load testing in most cases.

I'm very interested in feedback here, given my misunderstanding of the question. If you can offer it, let me thank you in advance.
What is the difference between a test suite, a test case and a test plan? How would you organize testing?
A test suite is made up of test cases. I'm not sure what a test plan is, aside from the obvious which the name would suggest. As far as organizing testing: I tend to organize my unit tests by class, with the method they test in the same order they exist within that class.
What kind of tests would you include for a smoke test of an ecommerce web site?
Again, here's another where I didn't know the terminology, so having to ask would result in demerits, but knowing the answer of "what is a smoke test?" allows us to properly answer the question:
In software testing, a smoke test is a collection of written tests that are performed on a system prior to being accepted for further testing.

In that case, I'd click around (or more likely, write an application that could be run many times that does the same thing, or use that application to write Selenium tests) looking for problems. I'd fill out some forms, and leave others blank. Ideally, it would all be random, so as to find problems with the specs as often as possible without actually testing all the specs, since the point seems to be to give us a quick way to reject the release without doing full testing.
What can you do reduce the chance that a customer finds things that he doesn't like during acceptance testing?
The best thing to do is to use incremental and iterative development that keeps the customer in the loop providing feedback before you get down to acceptance testing. Have effective tests in place that cover his requirements and ensure you hit those tests. When you come across something you know won't pass muster, address it even though it might not be a formal requirement.

There are undoubtedly underhanded ways to achieve that goal as well, but I'm not in the habit of going that direction, so I won't address them here.
Can you tell me something that you have learned about testing and quality assurance in the last year?
Again I'm going to reference my polysystemic testing nightmare, because it taught me that testing is extremely hard when you don't have the right tools at your disposal, and that sometimes, you've got to create them on your own.

As far as reading goes, I'd start with literature on TDD, as it's the most important yet underused as far as I'm concerned.

What advice would you give?

I like to use descriptive variable names, and I try to err on the side of more-descriptive if I think there's any chance of confusion. contract_participants isn't terribly long, but if you're building up all of its members from different sources (i.e., you can't really loop over it), it can get cumbersome to type and worse, to read. Moreover, it's different from just "participants" and they certainly aren't "contracts," so shortening it in this case wasn't going to happen. More...

contract_participant.first_name = "Joe"
contract_participant.last_name = "Smith"

contract_participant.on_drugs? = params[:on_drugs?]
...

It doesn't really violate the spirit of DRY, but it's frustrating nevertheless.

Pulling your hair out.

http://www.flickr.com/photos/heatsink/110859301/ / CC BY-NC-SA 2.0

I wanted to just have a scope for contract participants and not have to type it every time:

with(contract_participant) do 
    first_name = "Joe"

    last_name = "Smith"
    on_drugs? = params[:on_drugs?]
end

Unfortunately, unlike some languages, Ruby doesn't have such a construct. Fortunately, it does have the facilities to create one. I thought there was already an implementation, but I couldn't find it at first. So I started to write my own.

For some reason I was looking through Utility Belt and noticed it had an implementation of with():

class Object
  def with(object, &block)

    object.instance_eval &block
  end
end

Unbelievable! My implementation was running into a WTF in its own right, and here was this one-liner.

Unfortunately, Utility Belt's with() didn't pass the test cases that demonstrate how I wanted to use it, so I had to move on.

With was created as a result of those efforts. It works how I want it to in the simple cases I've demonstrated. It still needs some work on left-hand-side variables that are not members of the aforementioned @foo. It needs some tests for more complex uses like nested blocks (and code to make them pass). But it works for what I imagine the majority usage would be.

Peaceful meditation

I opted for the syntax With.object(foo) do ... end so as to not force you to use a change to Object. However, you can require 'with_on_object' if you prefer to just use with(@foo). There's also a conditional patch on NilClass if empty? does not exist. It is added and removed in the same method.

It requires Ruby2Ruby and ParseTree to do its magic, though that will change soon due to compatibility problems with Ruby 1.9 and other implementations (for which another project exists, if I've read correctly).

To install, you can use:

sudo gem install codeodor-with -s http://gems.github.com

Let me know if you're using it and encounter any problems. I'd like to fix them quickly, if I can.

Being a programmer, when I see something repetitive that I can automate, I normally opt to do so. We know that one way to save your job is by automating, but another is to know when not to automate. It sounds obvious, but when you get into the habit of rooting out all duplication of effort, you can lose sight of the fact that sometimes, it costs more to automate something than to do it by hand. More...

What's with this nonsense about unit testing?

Giving You Context

Joel Spolsky and Jeff Atwood raised some controversy when discussing quality and unit testing on their Stack Overflow podcast (or, a transcript of the relevant part). Joel started off that part of the conversation:

But, I feel like if a team really did have 100% code coverage of their unit tests, there'd be a couple of problems. One, they would have spent an awful lot of time writing unit tests, and they wouldn't necessarily be able to pay for that time in improved quality. I mean, they'd have some improved quality, and they'd have the ability to change things in their code with the confidence that they don't break anything, but that's it.

But the real problem with unit tests as I've discovered is that the type of changes that you tend to make as code evolves tend to break a constant percentage of your unit tests. Sometimes you will make a change to your code that, somehow, breaks 10% of your unit tests. Intentionally. Because you've changed the design of something... you've moved a menu, and now everything that relied on that menu being there... the menu is now elsewhere. And so all those tests now break. And you have to be able to go in and recreate those tests to reflect the new reality of the code.

So the end result is that, as your project gets bigger and bigger, if you really have a lot of unit tests, the amount of investment you'll have to make in maintaining those unit tests, keeping them up-to-date and keeping them passing, starts to become disproportional to the amount of benefit that you get out of them.

More...

Apparently, that was enough for others to come out of the shadows and discuss frankly why they don't see value in unit tests.

Joel was talking about people who suggest having 100% code coverage, but he said a couple of things about unit testing in general, namely the second and third paragraphs I quoted above: that changes to code may cause a ripple effect where you need to update up to 10% of your tests, and that "as your project gets bigger ... [effort maintaining your tests] starts to become disproportional to the amount of benefit that you get out of them."

One poster at Hacker News mentioned that it's possible for your tests to have 100% code coverage without really testing anything, so they can be a false sense of security (don't trust them!). Bill Moorier said,

The metric I, and others I know, have used to judge unit testing is: does it find bugs? The answer has been no. For the majority of the code I write, it certainly doesn't find enough bugs to make it worth the investment of time.

He followed up by saying that user reports, automated monitoring systems, and logging do a much better job at finding bugs than unit tests do.

I don't really care if you write unit tests for your software (unless I also have to work on it or (sometimes) use it in some capacity). I don't write unit tests for everything. I don't practice TDD all the time. If you're new to it I'd recommend that you do it though -- until you have enough experience to determine which tests will bring you the value you want. (If you're not good at it, and haven't tried it on certain types of tests, how else would you know?)

Dude... Wait, what?

The Points

All of that was there to provide you context for this simple, short blog post:

If changing your code means broken tests cascading through the system to the tune of 10%, you haven't written unit tests, have you?

(Further, the sorts of changes that would needlessly break so many unit-cum-integration tests would be rare, unless you've somehow happened or tried very hard to design a tightly coupled spaghetti monster while writing unit tests too.)
I've not yet met a project where the unit tests are the maintenance nightmare. More often, it's the project itself, and it probably doesn't have unit tests to maintain. The larger the code base, with large numbers of dependencies and high coupling, the more resistant it is to change - with or without unit tests. The unit tests are there in part to give you confidence that your changes haven't broken the system when you do make a change.

If you're making changes where you expect the interface and/or behavior to change, I just don't see where the maintenance nightmare comes from regarding tests. In fact, you can run them and find out what else in your code base needs to change as a result of your modifications.

In short, these scenarios don't happen enough such that they would make testing worthless.
You may indeed write a bunch of tests that don't do anything to test your code, but why would you? You'd have to try pretty hard to get 100% code coverage with your tests while succesfully testing nothing.

Perhaps some percentage of your tests under normal development will provide a false sense of security. But without any tests whatsoever, what sense of security will you have?
If you measure the value of unit testing by the number of bugs it finds (with more being better), you're looking at it completely wrong. That's like measuring the value of a prophylactic by the number of diseases you get after using it. The value is in the number of bugs that never made it into production. As a 2008 study from Microsoft finds [PDF], at least with TDD, that number can be astonishingly high.
As for user reports, automated monitoring systems, and logging doing a better job at finding bugs than unit testing: I agree. It's just that I'd prefer my shipped software to have fewer bugs for them to find, and I certainly don't look at my users as tests for my software quality once it's in production.

What are your thoughts?

A while ago, I was working with a problem in C# where where our code would get deadlocked, and since someone must die or several must starve, I thought it would be nice to just toss a "try again if deadlocked" statement into the exception handler. I muttered this thought on twitter to see if there was any language with such a try-catch-try-again-if construct. More...

A couple of my tweeps responded with what we we're used to using: loops.

Isn't that WHILE?

Most people just figure a loop is sufficient.

Another two mentioned redo in Ruby.

There is redo in Ruby

redo is certainly a cool construct (and underused), but it doesn't do what I want it to:

begin
  raise from_the_dead
rescue
  redo
end

Ruby responds, "tryredo.rb:35: unexpected redo." As you might know, you need to use a loop:

class Zombie
  def initialize 
    @starving = true

    @last_feeding = DateTime.new
  end
  
  def eat(bodypart)

    @starving = (DateTime.now - @last_feeding) * 10_000_000 > 3 
    starving_enough_to_reconsider = @starving && rand > 0.5

    
    unless starving_enough_to_reconsider || bodypart == "braaaiiinzzz"
      raise "Zombies don't like #{bodypart}"  
    end
    
    puts "Mmmmm... #{bodypart}"

    @starving = false
    @last_feeding = DateTime.now
  end
end

zombie = Zombie.new
["feet", "stomach", "intestines", "braaaiiinzzz"].each do |bodypart|
  begin

    zombie.eat bodypart
    sleep 1
  rescue
    puts $!
    redo 
  end

end

So I'll ask to this larger audience:

Suppose I have a zombie who only really likes eating braaaiiinzzz. Most of the time, he gets exactly what he wants. But every so often, we try to feed him some other body part. Now, if he's really hungry, he eats it. We might only have fingers available at the time, so I want to try to feed him fingers again until brains are available or until he's hungry enough to eat the fingers.

I know that conceptually this is just a loop even if we don't explicitly code it. But does your language have a try again? What do you think the merits or demerits of such an approach would be? Is it just a harmful goto?

My zombie is hungry and waiting. Rather impatiently I might add.

This is the fourth in a series of answers to 100 Interview Questions for Software Developers.

The list is not intended to be a "one-size-fits-all" list. Instead, "the key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills." Even still, "for most of the questions in this list there are no right and wrong answers!"

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. Here's that attempt.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy (and help out where you can!).

More...

How do you make sure that your code can handle different kinds of error situations?
I write tests that describe the expected error situations and check to see that they are handled appropriately. If the software is legacy and prone to generating exceptions, I'll wrap it up to report the exceptions to get an idea of what needs attention first, and start attacking it there. Of course where required, we'll use try/catch.

Mostly I try to follow the rigorous WOMM development process.
Can you explain what Test-Driven Development is? Can you name some principles of Extreme Programming?
TDD: Red, Green, Refactor. You write a test that specifies some behavior the code should produce. Run the test to make sure it fails. Write code to make the test pass, then run it to make sure it passes. Change the code as needed to improve its design.

XP: Aside from testing, we want to continuously review code through pair programming, maintain sustainable work conditions with sane work-weeks, and continually integrate our code to reduce the time we have to spend working out kinks in that process. There's more, but those are a few.

Reading: Xprogramming.com is a good starting point. The associated lists (TDD and XP) on Yahoo Groups are great for discussion. It's probably worth reading a book or two on the subjects. I'd recommend Kent Beck or Ron Jeffries.
What do you care about most when reviewing somebody else's code?
Does it say what it's doing, and do it correctly? Is it readable?

Reading: As with most of these questions, Steve McConnell's Code Complete 2 is an outstanding resource on the subject. If you don't read anything else, read that.
When do you use an abstract class and when do you use an interface?
I'd use an abstract class when I want to provide some implementation for reuse, but where some also remains to be specified by the inheriting class. An interface is useful for multiple inheritance in languages that don't allow it, as well as a decoupling device - allowing you to depend on interfaces that don't change as often as implementations might.

Reading: Books on OO design are useful, especially those targeting static languages like Java.
Apart from the IDE, which other favorite tools do you use that you think are essential to you?
Continuous integration tools, testing frameworks, (some people might include dependency injection frameworks), scripting languages, the command line, source control, ... What else would you include?

Reading: McConnell's aforementioned book, The Pragmatic Programmer, Practices of an Agile Developer, tons of blogs that talk about the tools they use to make themselves more productive, and (although I've only seen the presentation and not read the book), Neal Ford's The Productive Programmer probably contains some useful items.
How do you make sure that your code is both safe and fast?
The question seems to imply these goals are normally at odds. I haven't felt that way. I'd program for security first, and then if it's slow, I'd try to identify the bottleneck and then find a way to improve its time complexity. If the algorithm is already at its lower bound for time complexity, I'd move on to micro improvements, like moving variable creation and function calls outside of loops.
When do you use polymorphism and when do you use delegates?
I don't have any hard and fast rules. I rarely need to use polymorphism since I primarily program in dynamic languages that make it unnecessary. (I guess it's still polymorphism, but you're not doing anything special to achieve it.) When I have been in static languages, I'll implement the methods that accept different types as needed to make client code more friendly to work in. If we're actually building an API for public consumption, then obviously we have to move from "as needed" to a more aggressive schedule.

I'm at a loss for a better answer to this question, because (surprisingly to me) I've not thought about it before now.
When would you use a class with static members and when would you use a Singleton class?
I don't know how to answer this except for "when the situation calls for it." I'd normally opt for the class with static members when it makes sense to do it. As far as a proper Singleton, I don't know that I've ever written one that enforces that property. More often, if I need just one, I only create one.
Can you name examples of anticipating changing requirements in your code?
I write unit tests, so that helps with changing requirements. How? It helps keep the design very modular to allow for extension and easy changes, and the tests themselves provide assurance I haven't broken anything when I do need to make a change.

I don't generally go leaving hooks and pre-implementing code that I think will be needed. YAGNI helps guide me in that regard.
Can you describe the process you use for writing a piece of code, from requirements to delivery?
Requirement -> unit test -> code -> run tests -> commit -> run tests -> deploy.

I think that explains it all, and explaining each step could be a blog post or more of its own.

My basic advice on how to prepare for this section of questions is: Read and internalize Code Complete 2. I should probably read it again, in fact.

What advice would you give?

This is the fifth in a series of answers to 100 Interview Questions for Software Developers.

The list is not intended to be a "one-size-fits-all" list. Instead, "the key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills." Even still, "for most of the questions in this list there are no right and wrong answers!"

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. Here's that attempt.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy (and help out where you can!). More...

This week's answers are on a topic I've been wanting to explore more in depth here lately: algorithms (though it doesn't go into as much detail). I'll wait until the end to give reference information because all of this post relies on experience, but there are two sources where I'd start learning the information for every question. Trying to keep the post DRY and all.

How do you find out if a number is a power of 2? And how do you know if it is an odd number?
To find out if a number is a power of two, you can divide by two until the number is 1 or odd. If the number is odd at any point before you reach one, the number is not a power of two. To find out if a number is odd, I'd normally take number mod 2 and see if the result is 1 or not (1 would mean the number is odd). If performance is a concern and the compiler or runtime doesn't optimize for mod 2, you could use a bit mask that checks if the smallest bit is set or not. If so, the number is odd. If not, the number is even.

Here's an example with the bit mask:
```
def even? n
  n == n & 0b11111111_11111111_11111111_11111110

end

n = 512
while even?(n) && n > 1 do

  n = n >> 1  
end

puts n==1
```
Note you'll need a large enough bit mask to cover the size of the number in bits.
How do you find the middle item in a linked list?
If it's important to be able to find the middle element, I'd keep a pointer to it. If it's implemented using an array and it's important, we can store the array length and divide by two. If it's implemented using pointers to elements, we can iterate over the list while counting its length, then iterate from the beginning until we get halfway there. We could also take the size of the structure in memory and divide by two, going straight to that element by adding 1/2 the size to the pointer, but that'd be a mighty WTF to most programmers when trying to understand it.
How would you change the format of all the phone numbers in 10,000 static html web pages?
Not by hand if I could avoid it. I'd write a regex that matches any of the known formats in the set of pages and use a language with a function that replaces based on a regular expression find.
Can you name an example of a recursive solution that you created?
I was creating a pattern enumeration algorithm in an effort to statistically identify potentially important subsequences in a given genome. The more immediate goal was to identify Rho sites in a set of bacterial genomes. Since we wanted to identify any potential pattern, the form needed to be general, so a variable depth was required and we used recursion to achieve this. (This is a job interview, so I tried to think of the most impressive sounding example from the last year I could think of.)
Which is faster: finding an item in a hashtable or in a sorted list?
Item retrieval is basically O(1) in a hash table, while O(log n) in a sorted list, so the hash table is faster on average.
What is the last thing you learned about algorithms from a book, magazine or web site?
I guess it depends on what you'd consider learning. For instance, I recently looked up merge sort to use as reference in writing a sorting algorithm for a custom data structure, but I wouldn't say I "learned" it there. If you take "learning" as being introduced to, it was in a course at school or via a book.
How would you write a function to reverse a string? And can you do that without a temporary string?
In most instances I'd be working with a language that already implements a reverse method for strings. If not working in such a language, and I'm using a temporary string, the problem boils down to iterating backwards over the given string, and assigning tempstring[realstring_length - i] = realstring[i]. If we restrict the usage of a temporary string, then we can just use a variable to store the current character for swapping:

for(i=0; i<len; i++) { lowerchar = realstring[i]; realstring[i] = realstring[len-i-1]; // -1 for 0 based arrays realstring[len-1] = lowerchar; }
What type of language do you prefer for writing complex algorithms?
I prefer very super extremely high level languages (to distinguish from VHLL) that tend to be dynamic. The reason is that, in using them, I don't have to worry about low level details that might otherwise get in the way of understanding the algorithm. After that, I'll normally have to implement the same algorithm in a lower level language and take care of the details I could otherwise ignore, because normally performance is going to matter when developing that complex algorithm.
In an array with integers between 1 and 1,000,000 one value is in the array twice. How do you determine which one?
I'd insert each value as a key in a hash and when the key already exists, I know we've hit the duplicate. This gives us O(n) time complexity, which I'm sure could be proven to be the lower bound.
Do you know about the Traveling Salesman Problem?
Yes, it's famous. The problem asks us to find the shortest path that visits every node in a graph.

I promised you reading material, so here it is:

The Art of Computer Programming series by Donald Knuth
Wikipedia's list of algorithms is another good place to start.

What advice would you give?

SOAP can be a huge PITA in Ruby if you're not dealing with a web service that falls under the defaults. In particular, if your web service falls under HTTPS where you need to change the default certificate acceptance, or if you need to authenticate before seeing the WSDL, you're SOL as far as I can tell as of writing this post. (If you know of a way that doesn't resort to this complexity, please speak up!)

I was using Ruby 1.8.7 and soap4r 1.5.8, but this may apply to other versions. Anyway, here are a couple of monkey patches to help get you there if you're having trouble. More...

If you need to change the SSL verify mode, for example, to accept a certificate unconditionally, you can use this monkeypatch:

def monkeypatch_httpclient_sslsocketwrap(ssl_verify_mode)
    return unless ssl_verify_mode 
    @sslsocket_monkeypatched = true

    require 'soap/nethttpclient'
    block = <<END 
      alias :original_initialize :initialize
      def initialize(socket, context, debug_dev = nil)

        unless SOAP::NetHttpClient::SSLEnabled
          raise ConfigurationError.new('Ruby/OpenSSL module is required')
        end

        @context = context
        @context.verify_mode = #{ssl_verify_mode}
        @socket = socket
        @ssl_socket = create_openssl_socket(@socket)

        @debug_dev = debug_dev
      end
END
      HTTPClient::SSLSocketWrap.class_eval block
end

If you need to authenticate before seeing the WSDL, you'll need this patch:

def monkeypatch_authentication(username, password)
    return unless username && password
    @auth_monkeypatched = true

    require 'wsdl/xmlSchema/importer'
    block = <<END
      alias :original_fetch :fetch
      def fetch(location)

        warn("importing: " + location) if $DEBUG
        content = nil

        normalizedlocation = location
        if location.scheme == 'file' or

          (location.relative? and FileTest.exist?(location.path))

        content = File.open(location.path).read
          normalizedlocation = URI.parse('file://' + File.expand_path(location.path))

        elsif location.scheme and location.scheme.size == 1 and

          FileTest.exist?(location.to_s)
          content = File.open(location.to_s).read
        else

          client = web_client.new(nil, "WSDL4R")
          client.proxy = ::SOAP::Env::HTTP_PROXY
          client.no_proxy = ::SOAP::Env::NO_PROXY
          client.set_auth(location, "#{username}", "#{password}") #added for auth

          if opt = ::SOAP::Property.loadproperty(::SOAP::PropertyName)
            http_opt = opt["client.protocol.http"]

            ::SOAP::HTTPConfigLoader.set_options(client, http_opt) if http_opt
          end
          content = client.get_content(location)

        end
        return content, normalizedlocation
      end
END
      WSDL::XMLSchema::Importer.class_eval block
end

Hope that helps someone else avoid days' long foray into piecing together blogs posts, message boards, and searching through source code.

And because you might get here via a search for related terms, normal access that only requires basic authentication could be done like this, without opening existing classes:

class SendBasicAuthFilter < SOAP::Filter::StreamHandler
  def initialize(loginname, password)

    @authorization = 'Basic ' + [ loginname + ':' + password ].pack('m').delete("\r\n")

  end

  def on_http_outbound(req)
    req.header.delete('Authorization')

    req.header['Authorization'] = @authorization
  end

  def on_http_inbound(req, res)

  end
end

list_service_url = 'https://somesite/path/?WSDL'
list_service_driver = SOAP::WSDLDriverFactory.new(list_service_url).create_rpc_driver

user = 'username'

pass = 'password'
list_service_driver.streamhandler.filterchain << SendBasicAuthFilter.new(user,pass)

I'm very welcoming of suggestions regarding how these things might be better accomplished. Resorting to this messy level of monkeypatching just sucks. Let me know in the comments.

We observe that organizational structure metrics are significantly better predictors for identifying failure-prone binaries in terms of precision, and recall compared to models built using code churn, code complexity, code coverage, code dependencies, and pre-release defect measures.

(Link is to abstract page, quote is from the PDF linked to from there, chart below is from the paper as well) More...

SOAP can be a huge PITA in Ruby if you're not dealing with a web service that falls under the defaults. In particular, if your web service falls under HTTPS where you need to change the default certificate acceptance, or if you need to authenticate before seeing the WSDL, you're SOL as far as I can tell as of writing this post. (If you know of a way that doesn't resort to this complexity, please speak up!)

I was using Ruby 1.8.7 and soap4r 1.5.8, but this may apply to other versions. Anyway, here are a couple of monkey patches to help get you there if you're having trouble.

If you need to change the SSL verify mode, for example, to accept a certificate unconditionally, you can use this monkeypatch:

def monkeypatch_httpclient_sslsocketwrap(ssl_verify_mode)
    return unless ssl_verify_mode 
    @sslsocket_monkeypatched = true

    require 'soap/nethttpclient'
    block = <<END 
      alias :original_initialize :initialize
      def initialize(socket, context, debug_dev = nil)

        unless SOAP::NetHttpClient::SSLEnabled
          raise ConfigurationError.new('Ruby/OpenSSL module is required')
        end

        @context = context
        @context.verify_mode = #{ssl_verify_mode}
        @socket = socket
        @ssl_socket = create_openssl_socket(@socket)

        @debug_dev = debug_dev
      end
END
      HTTPClient::SSLSocketWrap.class_eval block
end

If you need to authenticate before seeing the WSDL, you'll need this patch:

def monkeypatch_authentication(username, password)
    return unless username && password
    @auth_monkeypatched = true

    require 'wsdl/xmlSchema/importer'
    block = <<END
      alias :original_fetch :fetch
      def fetch(location)

        warn("importing: " + location) if $DEBUG
        content = nil

        normalizedlocation = location
        if location.scheme == 'file' or

          (location.relative? and FileTest.exist?(location.path))

        content = File.open(location.path).read
          normalizedlocation = URI.parse('file://' + File.expand_path(location.path))

        elsif location.scheme and location.scheme.size == 1 and

          FileTest.exist?(location.to_s)
          content = File.open(location.to_s).read
        else

          client = web_client.new(nil, "WSDL4R")
          client.proxy = ::SOAP::Env::HTTP_PROXY
          client.no_proxy = ::SOAP::Env::NO_PROXY
          client.set_auth(location, "#{username}", "#{password}") #added for auth

          if opt = ::SOAP::Property.loadproperty(::SOAP::PropertyName)
            http_opt = opt["client.protocol.http"]

            ::SOAP::HTTPConfigLoader.set_options(client, http_opt) if http_opt
          end
          content = client.get_content(location)

        end
        return content, normalizedlocation
      end
END
      WSDL::XMLSchema::Importer.class_eval block
end

Hope that helps someone else avoid days' long foray into piecing together blogs posts, message boards, and searching through source code.

And because you might get here via a search for related terms, normal access that only requires basic authentication could be done like this, without opening existing classes:

class SendBasicAuthFilter < SOAP::Filter::StreamHandler
  def initialize(loginname, password)

    @authorization = 'Basic ' + [ loginname + ':' + password ].pack('m').delete("\r\n")

  end

  def on_http_outbound(req)
    req.header.delete('Authorization')

    req.header['Authorization'] = @authorization
  end

  def on_http_inbound(req, res)

  end
end

list_service_url = 'https://somesite/path/?WSDL'
list_service_driver = SOAP::WSDLDriverFactory.new(list_service_url).create_rpc_driver

user = 'username'

pass = 'password'
list_service_driver.streamhandler.filterchain << SendBasicAuthFilter.new(user,pass)

I'm very welcoming of suggestions regarding how these things might be better accomplished. Resorting to this messy level of monkeypatching just sucks. Let me know in the comments.

This is the sixth in a series of answers to 100 Interview Questions for Software Developers.

The list is not intended to be a "one-size-fits-all" list. Instead, "the key is to ask challenging questions that enable you to distinguish the smart software developers from the moronic mandrills." Even still, "for most of the questions in this list there are no right and wrong answers!"

Keeping that in mind, I thought it would be fun for me to provide my off-the-top-of-my-head answers, as if I had not prepared for the interview at all. Here's that attempt.

Though I hope otherwise, I may fall flat on my face. Be nice, and enjoy (and help out where you can!).

More...

This week's answers about data structures are complementary (indeed very intertwined with) last week's questions about algorithms. Also like last week, I'll wait until the end to give reference information because all of this post relies on experience, but there are two sources where I'd start learning the information for every question.

How would you implement the structure of the London underground in a computer's memory?
Without having travelled to London or on its subway system, I'd guess a graph would be the right data structure. The set of vertices would represent the stations, and the edges connecting them would be the tracks.

Not safe for work (language):

I don't know the proper in-memory representation of tramps.
How would you store the value of a color in a database, as efficiently as possible?
Efficiently for retrieval speed, storage speed, size? I'm guessing size. After asking why such efficiency is needed, and assuming we're talking about a range of up to 16⁶ colors (FFFFFF), I'd just store it as the smallest integer type where it would fit.
What is the difference between a queue and a stack?
A queue is typically FIFO (priority queues don't quite follow that) while a stack is LIFO. Elements get inserted at one end of a queue and retrieved from the other, while the insertion and removal operations for a stack are done at the same end.
What is the difference between storing data on the heap vs. on the stack?
The stack is smaller, but quicker for creating variables, while the heap is limited in size only by how much memory can be allocated. Stack would include most compile time variables, while heap would include anything created with malloc or new. (This is for C/C++, and not strictly the case.)
How would you store a vector in N dimensions in a datatable?
I need a little direction for this question, as I know not what it means. I encourage my readers, who have on most occasions proven themselves more adept than me, to come through again.
What type of language do you prefer for writing complex data structures?
I can't imagine using anything higher level than C or C++. Anything more advanced has most anything already built and not very easily molded. Or perhaps I just wouldn't think of it as complex.
What is the number 21 in binary format? And in hex?
10101 in binary and 15 in hex, and no I didn't cheat and use a calculator. It works just like decimal. Take the following digits of an arbitrary number in base B:

UVWXYZ

The number in decimal is U*B⁵ + V*B⁴ + W*B³ + X*B² + Y*B¹ + Z*B⁰

As more digits are added, you just increase the power by which it is raised. Also note that any number raised to the zeroth power is 1, so the Z element is just itself, and the ones digit.
What is the last thing you learned about data structures from a book, magazine or web site?
As with my answer to this question with regard to algorithms, I'm certain I've used to web for reference here, but I'd guess my introduction and original knowledge acquisition came from a book.

However, I would add journal article to the list of answers, because in both cases that would have been my answer, even though I read them via the web.

How would you store the results of a soccer/football competition (with teams and scores) in an XML document?

<fixtures>

  <fixture>
    <team name="Chelsea FC">
	<score>0</score>
    </team>

    <team name="Fulham FC">
	<score>1</score>
	<!-- any other stats? -->
    </team>

  </fixture>
</fixtures>

That might be reasonable.

Can you name some different text file formats for storing unicode characters?
I have to be honest here and say I don't know what you're talking about. I can't think of a file format that wouldn't take it.

Again the reading material is similar to last week:

The Art of Computer Programming series by Donald Knuth
Wikipedia's list of data structures is another good place to start.
Perhaps a book or resource on data modeling, since there were XML and DB questions.

What advice would you give?

Every day, psychological barriers are erected around us, and depending on what task they are a stumbling block for, they can be helpful or a hindrance.

Ramit Sethi wrote a guest post on personal finance blog Get Rich Slowly about passive barriers that got me thinking about passive barriers in software development, or perhaps just any easily destroyed (or erected barrier) that prevents you from doing something useful (or stupid). One example he uses that comes up a lot in my own work references email: More...

I get emails like this all the time:
"Hey Ramit, what do you think of that article I sent last week? Any suggested changes?"

My reaction? "Ugh, what is he talking about? Oh yeah, that article on savings accounts ... I have to dig that up and reply to him. Where is that? I'll search for it later. Marks email as unread"

Note: You can yell at me for not just taking the 30 seconds to find his email right then, but that's exactly the point: By not including the article in this followup email, he triggered a passive barrier of me needing to think about what he was talking about, search for it, and then decide what to reply to. The lack of the attached article is the passive barrier, and our most common response to barriers is to do nothing.

(Bold emphasis is mine).

If I can't immediately answer an email, it gets put on hold until I have time to go through and do the research that I need to do to give a proper reply. Sometimes, that means your email never gets answered because eventually I look down at the receipt date and say to myself "I guess it'd be stupid to respond now." But I digress.

In everyday software development, there are a number of barriers that can help us:

Minimizing or closing the browser. When a compilation is expected to take up to a minute, or a test suite will run for too long, or a query takes forever, there's not much work that can be done, so I might fire up the feed reader, email, or twitter to pass the time away. The problem here is that you'll often spend far longer on your excursion than it takes for your process to complete. If you waste just 5 minutes each time, you've accomplished nothing - you're just skimming and certainly not getting anything out of it, and you could have been back working on what you were trying to accomplish. In these situations, I have my email, feed reader, and twitter minimized, and that significantly reduces the urge to open them up and start a side quest.

If you wanted to get more to the active side of barriers, you might just add the line
```
127.0.0.1	reddit.com
```
to your hosts file. That turns a passive barrier to time waste into a downright pain.
Having a test suite with continuous integration and code analysis tools running. At various points in a day you might be tempted to check in code that breaks the build or introduces a bug. This is especially true at the end of the day. However, if you have a test suite that runs on every commit, you're much more likely to run it to avoid the embarrassment of checking in bad code. If you've got static analysis tools that also report on potentially poor code, you're less likely to write it.
Annoyance Driven Development. This isn't one that I know how to turn on or off, but I think it would be a great feature to have in IDEs or text editors: it gets slow when your methods or classes or files get too big. This would be a great preventative tool, if it exists. I guess it falls back to using test suites and code analysis to provide instant feedback that annoys you into doing the right thing.
Working with others, or having others review your code. Most of us pay more attention to quality when we know others will be looking at the code we write. Imagine how much more of your code you'd be proud to show off if you just knew that someone would be looking at it later.

Just as well, there are also barriers that hinder us:

Interruptions. This one is obvious, but so pervasive it should be mentioned. IM, telephone calls, email, coworkers stopping by to chat or ask questions - they all prevent us from working from time to time. The easy answer is to close these things, and that's what I do. They all represent passive barriers to getting work done, and you can easily turn that around to be a passive barrier against wasting time (see above). Pair programming is an effective technique that erects its own barrier to these time wasters.

If you're really looking for something to help you focus, I'd have a look at The Pomodoro Technique, which divides your work into tomato time units of 25 minutes each, which really helps you focus on work. It's agile for time management. (The Pomodoro Technique PDF book is available for free.)
Rotting Design: Rigidity, Fragility, Immobility, and Viscosity. Bob Martin discusses these in his (PDF) article on Design Principles and Design Patterns. Quoting him for the descriptions, I'll leave it to you to read for the full story:
Rigidity is the tendency for software to be difficult to change, even in simple ways. Every change causes a cascade of subsequent changes in dependent modules. What begins as a simple two day change to one module grows into a multi- week marathon of change in module after module as the engineers chase the thread of the change through the application.
...
Closely related to rigidity is fragility. Fragility is the tendency of the software to break in many places every time it is changed. Often the breakage occurs in areas that have no conceptual relationship with the area that was changed. Such errors fill the hearts of managers with foreboding. Every time they authorize a fix, they fear that the software will break in some unexpected way.
...
Immobility is the inability to reuse software from other projects or from parts of the same project. It often happens that one engineer will discover that he needs a module that is similar to one that another engineer wrote. However, it also often happens that the module in question has too much baggage that it depends upon. After much work, the engineers discover that the work and risk required to separate the desirable parts of the software from the undesirable parts are too great to tolerate. And so the software is simply rewritten instead of reused.
...
Viscosity comes in two forms: viscosity of the design, and viscosity of the environment. When faced with a change, engineers usually find more than one way to make the change. Some of the ways preserve the design, others do not (i.e. they are hacks.) When the design preserving methods are harder to employ than the hacks, then the viscosity of the design is high. It is easy to do the wrong thing, but hard to do the right thing.
The point is that poor software design makes an effective barrier to progress. There are only two ways I know to tear down this wall: avoid the rot, and make a conscious decision to fix it when you know there's a problem. There are plenty of ways to avoid the rot, but books are devoted to them, so I'll leave it alone except to say a lot of the agile literature will point you in the right direction.
Unit Tests. I struggled with the idea of putting this on here or not. If you're an expert, you already know this. If you're a novice or lazy, you'll use it as an excuse to avoid unit testing. The point remains: unit testing can be a barrier to producing software, if you are exploring new spaces and having trouble determining test cases for it. I'll let the Godfather of TDD, Kent Beck, explain:
... I still didn't have any software. As with any speculative idea, the chances that this one will actually work out are slim, but without having anything running, the chances are zero. In six or eight hours of solid programming time, I can still make significant progress. If I'd just written some stuff and verified it by hand, I would probably have the final answer to whether my idea is actually worth money by now. Instead, all I have is a complicated test that doesn't work, a pile of frustration, eight fewer hours in my life, and the motivation to write another essay.

These are just a few examples, so I'm interested in hearing from you. What barriers have you noticed that positively affect your programming? Negatively?

A friend of mine from graduate school recently asked if she could use me as a reference on her resume. I've worked with her on a couple of projects, and she was definitely one of the top few people I'd worked with, so I was more than happy to say yes.

Most of the questions were straightforward and easy to answer. However, one of the potential questions seemed way off-base: I may be asked to "review her multi-tasking ability."

Is that a trick question? Is it relevant? More...

This is the "I'm trying my hardest to be late to that meeting that spans lunch where they don't serve anything to tide you over" edition of Programming Quotables.

If you don't know - I don't like to have too many microposts on this blog (I'm on twitter for that), so save them up as I run across them, and every once in a while I'll post a few of them. The idea is to post quotes about programming that have one or more of the following attributes:

I find funny
I find asinine
I find insightfully true
And stand on their own, with little to no comment needed

It's up to you decide which category they fall in, if you care to. Anyway, here we go:

More...

This is my Quality is Dead hypothesis: a pleasing level of quality for end users has become too hard to achieve while demand for it has simultaneously evaporated and penalties for not achieving it are weak. The entropy caused by mindboggling change and innovation in computing has reached a point where it is extremely expensive to use traditional development and testing methods to create reasonably good products and get a reasonable return on investment. Meanwhile, user expectations of quality have been beaten out of them. When I say quality is dead, I don't mean that it's dying, or that it's under threat. What I mean is that we have collectively- and rationally- ceased to expect that software normally works well, even under normal conditions. Furthermore, there is very little any one user can do about it.

James Bach, Quality is Dead #1: The Hypothesis

I haven't figured out yet exactly how I'm going to use this, but it'll probably look like this: one new beat per day, one new track per weekend. One new app per month, one milestone on the app per week. The goal is not to establish a far-off goal and find a way to hit it, but to establish a series of tiny, immediate goals that keep you forever moving forward. Aristotle argued that virtue was mostly a matter of having good habits; Lao-Tzu tells us that the voyage of a million miles starts with a single step. So the key is to get moving and keep moving.

Giles Bowkett No New Language in 2009; New Habits Instead

But in computer games, it's impossible to have an equal match. It's humans versus machines. One side has an advantage of being able to perform a billion calculations per second, and the other has the massively parallel human brain.

Any parity here is an illusion, and it's that illusion that we seek to improve and maintain via the introduction of intelligent mistakes and artificial stupidity.

The computer has to throw the game in order to make it fun. When you beat the computer, it's an illusion. The computer let you win. We just want it to let you win in a way that feels good.

Mick West, Intelligent Mistakes: How to Incorporate Stupidity Into Your AI Code

Problem

A Solution?

Problem

Goals

Progress

Help Wanted

Motivation

Launching a business

Deciding on technology

Code

Requirements

Motivation

Types of Cohesion and Coupling

What does it all mean?

Examples from each category of cohesion

Examples from each category of coupling

Concluding Remarks