At this stage, if you've heard of Rails and you haven't converted, it's entirely possible that you never will. It's also entirely possible that anybody who still isn't even taking Rails seriously by this point might just be some kind of idiot.
...
Every programmer should also read Chad Fowler's "My Job Went To India" book, where he explains that as larger and larger numbers of programmers adopt a particular skill, that skill becomes more and more a commodity. Rails development becoming a commodity is really not in the economic interest of any Rails developer. This is especially the case because programming skill is very difficult to measure, which - according to the same economics which govern lemons and used-car markets - means that the average price of programmers in any given market is more a reflection of the worst programmers in that market than the best. An influx of programmers drives your rates down, and an influx of incompetent programmers drives your rates way the fuck down.

Instead, I want to talk about my first attempt at solving the puzzle, which was an utter failure. A glorious, spectacular failure. Perhaps the single most impressive failure of my career. Failures are often much more interesting than successes, but for some unfathomable reason, people are often reluctant to discuss them.

And this is just the beginning of the ceremony:

Me: read file blah.txt and display it on system output
Java: How should I name the class?
Me: Test
Java: How should I handle errors?
Me: I don't care right now, I just need to display that data to system output
Java: But I need to know this, what if something unexpected happens?!
Me: I just want to make a prototype damn it!
Java: Sorry, can't do it.
Me: Ok, do nothing on error.
Java: And which implementation of Stream class should I use for reading?

Everyone knows that diversification is the key to managing financial risk, but few people seem to apply this principal to their professional careers. Most developer shops are relatively limited when it comes to the number of technologies and problem domains they deal with. If you want to diversify your resume without job hopping every year, then it makes sense to actively seek out technology experiences that are different from the ones you use in your day job.

Neal Ford and others have been talking about the distinction between dynamic and static typing as being incorrect. The real question is between essence and ceremony. Java is a ceremonious language because it needs you to do several dances to the rain gods to declare even the simplest form of method. In an essential language you will say what you need to say, but nothing else. This is one of the reasons dynamic languages and type inferenced static languages sometimes look quite alike - it's the absence of ceremony that people react to.

And launch as soon as you can, so you start learning from users what you should have been making.
...
So when you look at something like Reddit and think "I wish I could think of an idea like that," remember: ideas like that are all around you. But you ignore them because they look wrong.
...
Here it is: I like to find (a) simple solutions (b) to overlooked problems (c) that actually need to be solved, and (d) deliver them as informally as possible, (e) starting with a very crude version 1, then (f) iterating rapidly.

I've learned that developers who spend most of their time thinking about their project, rather than grueling over mundane tasks and busy work, do better. Their projects are stronger, easier to develop, and organized beautifully. These developers spend much of their free time doing things like evangelizing the MVC paradigm, reading non-fiction, and dining out. Their counterparts - those who fear the rush of extra web traffic, faulty error handling, and accidently falling into while() loops that never end - live in fear of their creation, dreading the day when it will turn on the parent.

The experiences discussed herein were valuable in their own right, but the challenge itself is rewarding as well. How often do we pause to reflect on what we've learned, and more importantly, how it has changed us? Because of that, I recommend you perform the exercise as well.

I freely admit that some of this isn't necessarily caused by my experiences with the language alone - but instead shaped by the languages and my experiences surrounding the times.

One last bit of administrata: Some of these memories are over a decade old, and therefore may bleed together and/or be unfactual. Please forgive the minor errors due to memory loss.


How QBASIC Made Me A Programmer

As I've said before, from the time I was very young, I had an interest in making games. I was enamored with my Atari 2600, and then later the NES. I also enjoyed a playground game with Donald Duck and Spelunker.

Before I was 10, I had a notepad with designs for my as-yet-unreleased blockbuster of a side-scrolling game that would run on my very own Super Sanola game console (I had the shell designed, not the electronics).

It was that intense interest in how to make a game that led me to inspect some of the source code Microsoft provided with QBASIC. After learning PRINT, INPUT, IF..THEN, and GOTO (and of course SomeLabel: to go to) I was ready to take a shot at my first text-based adventure game.

The game wasn't all that big - consisting of a few rooms, the NEWS directions, swinging of a sword against a few monsters, and keeping track of treasure and stats for everything - but it was a complete mess.

The experience with QBASIC taught me that, for any given program of sufficient complexity, you really only need three to four language constructs:

1. Input
2. Output
3. Conditional statements
4. Control structures

Even the control structures may not be necessary there. Why? Suppose you know a set of operations will be performed an unknown but arbitrary amount of times. Suppose also that it will be performed less than X number of times, where X is a known quantity smaller than infinity. Then you can simply write out X number of conditionals to cover all the cases. Not efficient, but not a requirement either.

Unfortunately, that experience and its lesson stuck with me for a while. (Hence, the title of this weblog.)

Side Note: The number of language constructs I mentioned that are necessary is not from a scientific source - just from the top of my head at the time I wrote it. If I'm wrong on the amount (be it too high or too low), I always appreciate corrections in the comments.


What ANSI Art taught me about programming

When I started making ANSI art, I was unaware of TheDraw. Instead, I opened up those .ans files I enjoyed looking at so much in MS-DOS Editor to see how it was done. A bunch of escape codes and blocks came together to produce a thing of visual beauty.



Since all I knew about were the escape codes and the blocks (alt-177, 178, 219-223 mostly), naturally I used the MS-DOS Editor to create my own art. The limitations of the medium were strangling, but that was what made it fun.

And I'm sure you can imagine the pain - worse than programming in an assembly language (at least for relatively small programs). Nevertheless, the experience taught me some valuable lessons:

• Even though we value people over tools, don't underestimate the value of a good tool. In fact, when attempting anything new to you, see if there's a tool that can help you. Back then, I was on local BBSs, and not the 1337 ones when I first started out. Now, the Internet is ubiquitous. We don't have an excuse anymore.
  
  
• I can now navigate through really bad code (and code that is limited by the language) a bit easier than I might otherwise have been able to do. I might have to do some experimenting to see what the symbols mean, but I imagine everyone would. And to be fair, I'm sure years of personally producing such crapcode also has something to do with my navigation abilities.
  
  
• Perhaps most importantly, it taught me the value of working in small chunks and taking baby steps. When you can't see the result of what you're doing, you've got to constantly check the results of the latest change, and most software systems are like that. Moreover, when you encounter something unexpected, an effective approach is to isolate the problem by isolating the code. In doing so, you can reproduce the problem and problem area, making the fix much easier.

The Middle Years (included for completeness' sake)

The middle years included exposure to Turbo Pascal, MASM, C, and C++, and some small experiences in other places as well. Although I learned many lessons, there are far too many to list here, and most are so small as to not be significant on their own. Therefore, they are uninteresting for the purposes of this post.

However, there were two lessons I learned from this time (but not during) that are significant:

1. Learn to compile your own $&*@%# programs (or, learn to fish instead of asking for them).
2. Stop being an arrogant know-it-all prick and admit you know nothing.

As you can tell, I was quite the cowboy coding young buck. I've tried to change that in recent years.


How ColdFusion made me a better programmer when I use Java

Although I've written a ton of bad code in ColdFusion, I've also written a couple of good lines here and there. I came into ColdFusion with the experiences I've related above this, and my early times with it definitely illustrate that fact. I cared nothing for small files, knew nothing of abstraction, and horrendous god-files were created as a result.

If you're a fan of Italian food, looking through my code would make your mouth water.

DRY principle? Forget about it. I still thought code reuse meant copy and paste.

Still, ColdFusion taught me one important aspect that got me started on the path to Object Oriented Enlightenment: Database access shouldn't require several lines of boilerplate code to execute one line of SQL.

Because of my experience with ColdFusion, I wrote my first reusable class in Java that took the boilerplating away, let me instantiate a single object, and use it for queries.


How Java taught me to write better programs in Ruby, C#, CF and others

It was around the time I started using Java quite a bit that I discovered Uncle Bob's Principles of OOD, so much of the improvement here is only indirectly related to Java.

Sure, I had heard about object oriented programming, but either I shrugged it off ("who needs that?") or didn't "get" it (or more likely, a combination of both).

Whatever it was, it took a couple of years of revisiting my own crapcode in ColdFusion and Java as a "professional" to tip me over the edge. I had to find a better way: Grad school here I come!

The better way was to find a new career. I was going to enter as a Political Scientist and drop programming altogether. I had seemingly lost all passion for the subject.

Fortunately for me now, the political science department wasn't accepting Spring entrance, so I decide to at least get started in computer science. Even more luckily, that first semester Venkat introduced me to the solution to many my problems, and got me excited about programming again.

I was using Java fairly heavily during all this time, so learning the principles behind OO in depth and in Java allowed me to extrapolate that for use in other languages. I focused on principles, not recipes.

On top of it all, Java taught me about unit testing with JUnit. Now, the first thing I look for when evaluating a language is a unit testing framework.


What Ruby taught me that the others didn't

My experience with Ruby over the last year or so has been invaluable. In particular, there are four lessons I've taken (or am in the process of taking):

• The importance of code as data, or higher-order functions, or first-order functions, or blocks or closures: After learning how to appropriately use yield, I really miss it when I'm using a language where it's lacking.
  
  
• There is value in viewing programming as the construction of lanugages, and DSLs are useful tools to have in your toolbox.
  
  
• Metaprogramming is OK. Before Ruby, I used metaprogramming very sparingly. Part of that is because I didn't understand it, and the other part is I didn't take the time to understand it because I had heard how slow it can make your programs.
  
  Needless to say, after seeing it in action in Ruby, I started using those features more extensively everywhere else. After seeing Rails, I very rarely write queries in ColdFusion - instead, I've got a component that takes care of it for me.
  
  
• Because of my interests in Java and Ruby, I've recently started browsing JRuby's source code and issue tracker. I'm not yet able to put into words what I'm learning, but that time will come with some more experience. In any case, I can't imagine that I'll learn nothing from the likes of Charlie Nutter, Ola Bini, Thomas Enebo, and others. Can you?

What's next?

Missing from my experience has been a functional language. Sure, I had a tiny bit of Lisp in college, but not enough to say I got anything out of it. So this year, I'm going to do something useful and not useful in Erlang. Perhaps next I'll go for Lisp. We'll see where time takes me after that.

That's been my journey. What's yours been like?

Now that I've written that post, I have a request for a post I'd like to see:

What have you learned from a non-programming-related discipline that's made you a better programmer?

Later, he moves on to describe four different types of design for your DSL. This was something I hadn't really thought about before, therefore it was the most interesting for me. Here are the four types:

• Instantiation: The DSL consists simply of methods on an object. This is not much different from normal programming. He says it is "DSLish," perhaps the way Blaine Buxton looks at it.
  
  
• Class Macros: DSL as methods on some ancestor class, and subclasses can then use those methods to tweak the behavior of themselves and their subclasses. This follows a declarative style of programming, and is the type of DSL followed by Ruby on Rails (and cfrails) if I understand him correctly.
  
  
• Top-level methods: Your application defines the DSL as a set of top-level methods, and then invokes load with the path to your DSL script. When those methods are called in the configuration file, they modify some central (typically global) data, which your application uses to determine how it should execute. This is like scripting, and (again) if I understood correctly, this is the style my (almost working correctly) Partial Order Planner uses.
  
  
• Sandboxing (aka Contexts): Similar to the Instantiation style, but with more magic. Your DSL is defined as methods of some object, but that object is really just a "sandbox." Interaction with the object's methods modifies some state in the sandbox, which is then queried by the application. This is useful for processing user-maintained scripts kept in a database, and you can vary the behavior by changing the execution context.

Note: These are "almost" quotes from the slides/talk (with a couple of comments by myself), but I didn't want to keep rewinding and such, so they aren't exact. Therefore, I'm giving him credit for the good stuff, but I stake a claim on any mistakes made.

The talk uses Ruby as the implementation language for the example internal DSLs, but I think the four types mentioned above can be generalized. At the least, they are possible in ColdFusion - but with Java, I'm not so sure. In particular, I think you'd need mixin ability for the Top-level methods style of DSL.

Next, Obie catalogs some of the features of Ruby you see used quite often in crafting DSLs: Symbols (because they have less noise than strings), Blocks (for delayed evaluation of code), Modules (for cleaner separation of code), Splats (for parameter arrays), the different types of eval (for dynamic evaluation), and define_method and alias_method.

Finally, he winds up the presentation with a bit about BNL. I liked this part because I found Jay Fields' blog and his series of articles about BNL.

As always, comments, questions, and thoughts are appreciated. Flames - not so much.

The first item where I see a distinction that needs to be made is on, "what do we mean when we are talking about XML?" I see two types - XML the paradigm where you use tags to describe data, and the XML you write - as in, the concrete tags you put into a file (like, "see that XML?"). We're talking about XML you've written, not the abstract notion of XML.

The second idea: what is code? What is data? Sean Corfield offers what I would consider to be a concice, and mostly correct definition: "Code executes, non-code (data) does not execute." To make it correct (rather than partially so), he adds that (especially in Lisp) code can be data, but data is not code. You see this code-as-data any time you are using closures or passing code around as data. But taking it a bit further - your source code is always just data to be passed to a compiler or interpreter, which figures out what the code means, and does what it has been told to do.

So is XML code? Certainly we see it can be: ColdFusion, MXML, and others are languages where your source code is written (largely) in XML. But what about the broader issue of having a programmatic configuration file versus a "data-only" config file?

Is the data in the config file executable? It depends on the purpose behind the data. In the case of data like


I think (assuming there is nothing strange going on) that it is clearly data. Without knowing anything about the back end, it seems like we're just creating a data structure. But In the case of DI (and many others uses for config files), I see it as giving a command to the DI framework to configure a new bean. In essence, as Peter notes, we've just substituted one concrete syntax for another.

In the case of XML, we're writing (or using) a parser to send data to an intepreter we've written that figures out what "real" commands to run based on what the programmer wrote in the configuration file. We've just created a higher level language than we had before - it is doing the same thing any other non-machine code language does (and you might even argue about the machine code comment). In the configuration case, often it is a DSL (in the DI case specifically, used to describe which objects depend on which other objects and load them for us).

While we don't often have control structures, there is nothing stopping us from implementing them, and as Peter also notes, just because a language is not Turing complete), doesn't mean it is not a programming language. In the end, I see it as code.

Both approaches are known to have their benefits and drawbacks, and choosing one over the other is largely a matter of personal taste, size and scope of problem, and problem/solution domain. For me, in the worlds of JIT compiling and interpreted langages, the programmatic way of doing things tends to win out - especially with large configurations because I prefer to have the power of control structures to help me do my job (without having to implement them myself). On the other hand, going the hard-coded XML route is especially popular in the compiled world, if not for any other reason than you can change configurations without recompiling your application.

I don't make a distinction between the two on terms of XML is data, while programming (or using an in-language DSL) in your general-purpose language is code. To me, they are both code, and if either is done incorrectly it will blow-up your program.

Finally, I'm not sure what value we gain from seeing that code is data (and in many cases config data is code), other than perhaps a new way of looking at problems which might lead us to find better solutions. But that isn't provided by the distinction itself, just the fact that we saw it.

Comments, thoughts, questions, and requests for clarifications are welcome and appreciated.

My interest lies in the second complaint though - that writing it to a live file (rather than dynamically figuring it out - which I've been calling "synthesis," as opposed to generation) means you can't get the benefits of upgrades to the scaffolding engine (and, it is not following DRY!). But, if you just use the default scaffolding, what happens if it doesn't come out just right (assuming you've even passed the notUgly test in the first place)? Well, thats a big part of what I'm trying to solve with cfrails, by using a DSL that provides tweakability to the scaffolding, without requiring you to write to a file (though, if the tweaks won't work, you are always welcome to go to a file with HTML and the like). The interesting part for me about the RoR plugins above, therefore, is that it appears (I haven't checked them out yet) that at least Hobo contains a DSL, called DRYML, to help along those lines.

I'll be having a closer look at those when free time becomes a bit less scarce. What do you think? Is it the holy grail, or can there be very useful "scaffolding"?

Next, of interest to Java programmers and ORM fans, Google has contributed code to Hibernate that allows people to "keep their data in more than one relational database for whatever reason-too much data or to isolate certain datasets, for instance-without added complexity when building and managing applications."

Then, Bruce Tate, who is probably best known for his books Beyond Java and From Java to Ruby (by me anyway) has a case study on ChangingThePresent.org, his new project that uses Rails. It looks like a good starting point if you are developing a high-traffic Rails site. And, the site does some good charity work - taking only the credit card processing fee from your donation to the causes of your choice.

Finally, Gilad Bracha speaks about dynamic languages on the JVM. Until recently he was the Computational Theologist (is that someone who studies the religion of computation?) at Sun. The talk is just over 30 minutes long, and is at a surprisingly low level - so geeks only! He discusses a few of the challenges with implementation details on getting dynamic languages to run natively in byte-code (as opposed to writing an interpreter in Java which would then interpret the dynamic language). This includes a new instruction for invoking methods - invokedynamic, as well as a discussion on possible implementations of hotswapping - the technique that allows you to modify code at runtime. Very interesting stuff.

For today, I should be done referencing InfoQ, except to give them credit in my next post for turning me on to a couple of good resources for implementing DSLs in Ruby (which is the subject of my next post). I'll post the link in the comments when I'm done with it - I try not to edit these posts anymore since MXNA likes to visit every 30 seconds and thinks the date is the unique identifier of the post... you know, instead of using the XML field called "guid." =) (sorry guys, just poking a little fun - thanks for aggregating me).

In a nutshell, the goal is to have such highly customizable "scaffolding" that there really is no scaffolding - all the code is synthesized - or generated "on the fly." Of course, this only gets you so far. For the stuff that really makes your application unique, you'll still have to code that. But you can compose views from others and such, so it's not like related tables have to stay unrelated, but I do want to stress that right now there is no relationship stuff implemented in the ORM.

I've skipped a few "mini-versions" from 0.1.3 to 0.2.0 because there were so many changes that I haven't documented one-by-one. That's just sloppiness on my part. Basically, I started by following Ruby on Rails' example, and taking my own experience about what I find myself doing over and over again. That part is done, except that the ORM needs to be able to auto-load and lazy-load relationships at the programmer's whim. In any case, once I got enough functionality to start using it on my project, I've been developing them in parallel. The problem is, I've fallen back on poor practices, so the code isn't as nice as it could be.

In particular, there aren't any new automated tests after the first couple of releases, which isn't as bad as it might otherwise be, since a lot of the core code was tested in them. But on that note, I haven't run the existing tests in a while, so they may be broken. Further, since I've been thinking in Ruby and coding in Coldfusion, you'll see a mix of camelCase and under_score notations. My original goal was to provide both for all the public methods, and I still plan to do that (because, since I can't rely on case from all databases for the column names -- or so I think -- I use the under_score notation to tell where to put spaces when displaying the column names). But right now, there is a mix. Finally, the DSL needs a lot more thought put behind it - Right now it is a mix-and-match of variables.setting variables and set_something() methods. Right now it is really ugly, but when I take the time to get some updated documentation up and actually package it as a zip, I should have it cleaned up. In fact, I shouldn't have released this yet, but I was just starting to feel I needed to do something, since so much had be done on it and I hadn't put anything out in quite some time. Besides that, I'm quite excited to be using it - it's been a pain to build, but it's already saved me more time than had I not done anything like it.

In the end, I guess what I'm trying to say is: 1) Don't look at it to learn from. There may be some good points, but there are bad points too, and 2) Don't rely too heavily on the interfaces. While I don't anticipate changing them (only adding to them, and not forcing you to set variables.property), this is still less than version 1, so I reserve the right to change the interfaces until then. =)

Other than that, I would love to hear any feedback if you happen to be using it, or need help because the documentation is out of date, or if you tried to use it but couldn't get it to work. You can contact me here. You can find cfrails at http://cfrails.riaforge.org/ .