I just bought the latest book on Erlang from the pragmatic progammers. If you haven't heard, Erlang is a functional programming language that has some unique architectural constraints that make it work really well with parallelism and concurrency. Erlang is "hot". This post isn't about Erlang, other than to note that it's what concludes the hype phase that has made Ruby on Rails the darling of fanboys everywhere and an annoyance to the rest of us. This means Ruby and Rails are going to have to stop riding on the wave of excitement by being the latest new thing and are going to have to start impressing people who have already heard about it, tried it, and unlike the fanboys, thought "nya -- needs some work, I'll check back in a few years".

You gets a lot of attention when you mouth off the way the rails zealots have. You get a lot of attention for a while, that is, until then everybody gets tired of you, exposes your flaws, and waits to see how you do fixing them in a climate of criticism. One of the reasons I still like and use java, despite its problems, verbosity, and all that, is that when valid criticisms are leveled against java, the people in the java community don't make excuses, they slowly quietly address the problems. That's a sign of maturity, and Ruby and Rails need to get it or they aren't going to be talked about in five years.

In terms of the Gartner Hype Cycle, Ruby and Rails are nearing the end of the "Peak of Inflated Expectations" and are moving into the "Trough of Disillusionment". For evidence, here's an article by a PHP guy who switched back. That is NOT supposed to happen.

My experience with Ruby is that I read the pick-axe book four or five years ago, before Rails, played with it, liked it, and added a few other books too. I liked ruby's syntax - very efficient for the original writer. I had my doubts about using it in team settings with code maintained over years. I've used Ruby to solve production problems in business settings on several occasions. I never considered adopting Ruby as my "main" language, for reasons I'll expound on below.

Then Rails appeared. Frankly, Rails ruined my enjoyment of Ruby because it attracted so many fanboys and zealots. The folks were making outrageous and ridiculous claims and were so arrogant about how Ruby would be the Java killer. These folks jumped the shark this week, with the posting of Top 10 Reasons Why Java Sucks Ass. Thank you Obie, for being so absurd that no one can deny how absurd the hype around Rails has been.

Now that the industry has played with Ruby and Rails seriously, the problems of Ruby and Rails have become well known. They are:

• No high quality app server
• No language specification.
• Custom runtime VM that isn't supported by any well resourced organization
• Open source is not a substitute for open standards at the language or runtime engine level.
• You can quickly write code than nobody can understand, and several years later, people have done exactly that
• Duck typing doesn't save code, as it requires more unit tests or occasionally blows up on you
• Mixins aren't so great: A class's state and behavior specification shouldn't depend on its runtime history. I shouldn't need unit tests to verify my class can do what I think it's supposed to do - the unit tests waste more time than I save. Java's new static imports are a much better solution.
• Total lack of high quality enterprise features: transaction support is weak, SOAP support is weak, messaging is weak, caching solutions are weak, directory integration is weak, clustering is weak, multithreading is weak, etc...
• What do I do in the real world where my database already exists with 8 years of data in it and I can't design my schema to match the Rails opinions of what's best?
• Dynamic typing and Mixins make it nearly impossible to have tool assisted refactoring and good code completion. They are appropriate for small tasks with a low reuse quotient for 3rd party code (aka non-enterprise development).
• Opinionated software is great, if you are a follower-the-leader type who doesn't want to think about what's really best to solve your problem.
• Overuse of DSL's and embedding DSL's in regular code basically assure difficulty in maintenance. Your DSL's aren't as obvoius as you think. The reason everybody hates XML in java is not because it's XML (they don't complain about XHTML or RSS do they?) it's because they are really DSL's, but at least with editor support.
• Ruby is unbelievably slow. Slow. Slow. Slow. Don't say it doesn't matter because the database is slower, because it does matter. Ruby is like 6-7 times slower than Java. 2-3 times slower than Python and Perl.
• Reliability and scalability are crummy

The funniest thing about the above list is that the Ruby app servers are so bad that what's going to end up happening is that the Java app servers will bolt on JRuby support and Rails will simply be absorbed into the borg that is Java. If you run Glassfish, Rails is just another java library! That's really, really funny. Nya-Nya to the "java sucks" fanboys! Once they go away, I might actually take up Ruby again -- if the Ruby community shows some interest in solving some of the problems above. Not all of them -- just show me you can listen.

It turns out that it was easier for Java to bolt on the few innovations that Ruby offered than for Ruby to absorb the many innovations that Java has offered over the years. I think the end result of Ruby is that it killed perl, and forced PHP developers to improve their sophistication (see the article from the guy who switched back to PHP). Unfortunately, it stunted the growth of enterprise language solutions in Python. Python is a lot more maintainable than Ruby, by the way. Ruby's effect on Java has been positive. The concepts of reasonable conventions are being used more. Groovy is looking good (better than Ruby except for being even slower). Most of all, Ruby has forced Java framework makers to think about simplicity, and they have. Ruby is helping make REST more mainstream. REST has a fanboy problem too, but with a lot of work could become a viable alternative to SOA. The final benefit of Ruby is that JDK 7 will finally bring closures to Java.

Speaking of closures... That's why I'm reading a book on Erlang. Ruby touts closures as their biggest strength, but if you buy into that, you should be using functional programming languages. Erlang (and Haskell and all the FP languages) are better for the purity of their closures than Ruby. Erlang gets other things right that Ruby struggles with. Ruby threading is a mess. Erlang gets this deeply and emphatically right (even Java should take note). Unlike Ruby, which falters as it tries to move up to enterprise settings, Erlang as been used for 10 years in carrier grade telecommunications applications with many 9's of reliability. Erlang is Rock solid. As in: bet your company on the fact that your apps won't be down EVER.

Do I think Erlang will take over? No. People are starting to expose it's flaws too. But this isn't as fun as doing so for Ruby, because it's origins in telecom make it immune to false euphoria. But, grid computing and multi-core processing are moving IT towards Erlang's strength. Erlang's unique insistence on non-reassignable variables allow it to deliver fault tolerance, concurrency, and parallelism that give it something that will not be easy to bolt on to Java. While functional programming can be added to Java. Though Scala is looking really interesting at bring FP to the JVM and it's performance doesn't disappoint like Ruby/Groovy, Erlang's inherent concurrency cannot be casually added the way many of the Rails innovations could be, and this means Erlang will have a suite spot that cannot be assimilated into the borg.

But this article is not really about Erlang. Erlang is here as a foil, and for it's cultural impact: the biggest feature that Erlang offers at this moment is that it will bring to a close the Rails 15 minutes of fame. Every time some Rails fanboy starts peddling their hype, the approved thing to do is to respond with Erlang. There is nothing sweeter than dropping the credible innuendo that hype is transient and fleeting and is almost over.

And because Erlang is old school telecom stuff, it will come without the fanboy culture. There something really refreshing about the the fact that it's a telecom industry's solution. After Ruby on Rails, it's hip and trendy to be stodgy and uncool, and Erlang fits the bill. Unsexy is sexy. God bless Erlang.

Technorati Tags: ruby fanboy erlang 15-minutes

I study service description contracts as a key difference between REST and SOAP, as part of my series comparing Resource Oriented Architectures and Service Oriented Architecture. In the previous article on ROA and SOA I examined what ROA and SOA are as objectively as I could. Now it's time to state some opinions on how the differences play out in an enterprise setting by studying the practical considerations regarding service description contracts with adopting ROA or SOA. These are non-technical considerations that an enterprise has to think about when deciding on what to adopt.

In the previous article, I found three key differences between ROA and SOA:

1. Contractual vs Not - SOA offers contractually, discoverable interfaces, while and ROA expects designers to solve problems as they see fit.
2. Rich vs Uniform Behavior - ROA has the uniform interface applied with minimally overloading, while SOA provides rich, discoverable behavior
3. Differentiation vs Equivalence of Data and Metadata - SOA relies on metadata to inform behavior, while ROA uses connectedness and design time conventions for representations and URI's to blur data and metadata together.

Today I study the value of web service description contracts, in a typical enterprise development setting.

My conclusion: I think enterprise settings mostly need service descriptions. A contract levels the playing field between service provider and client which is important when organizational politics might get involved. Otherwise the provider has all the power and the proliferation of unstandardized web service providers in an enterprise setting will take an huge toll. Lack of a contract certainly gives service providers freedom. That's great if you are a provider as there's no one to answer to. It's terrible if you need to be the client of 20 different kinds of web services, since basic questions like "what URL do I POST to and what do I POST to get a list of X's with property Y" will be documented 20 different ways. While efforts like WADL and NSDL offer solutions, these have to be institutionalized into ROA for it to be viable in the enterprise, and frankly the REST people resist contracts for reasons that appear cultural. The sad thing about all this is that REST is not incompatible with contractual service descriptions, there just are no warmly embraced standards for them to correspond to WSDL in SOAP.

Recall with horror how hard it is to maintain an application if SQL has leaked throughout the layers, especially to my view code. Somebody moves an entity attribute to another table and "ack!" we've got tons of duplicated code to fix. Now consider the weak correspondence between GET, POST, PUT, DELETE and the CRUD operations. Now consider that the URI query parameters are your new form for expressing WHERE clauses, and that, unlike SQL, web services are a distributed technology so you don't control the analog of the database schema. Now consider that you can't encapsulate these URI's because they are designed to be universal addresses. Links could be in your source code, in your data, or in your end user's bookmarks and emails. REST depends a little too much on links not breaking and POST parameters not changing to NOT express this in a contract.

In order to use any web service, the caller must know the technical details about how to invoke it. WSDL helps SOAP solve this problem. The WSDL spec is admittedly a bit over-engineered, and solves a much broader, more abstract problem than what SOAP over HTTP alone requires. The bloat is a minor flaw, and is greatly overstated by detractors. I often look at raw WSDL in an editor.

But really, who cares how ugly it is or isn't? As long as the tools eat it, I'm happy. And WSDL is very well tooled because the IT industry invested heavily in SOA. Most of the kinks have been worked out by now, so that if you follow the WS-I recommendations, you get great interoperability that's easy to use. Stick the WSDL it in your favorite SOAP stack, whether that's one of a dozen in Java, or C#, Ruby, Erlang or whatever. And then forget about it. Visual WSDL explorers are appearing, like the one the Eclipse web services plugin provides.

I hear the REST people saying "how hard is it to GET and POST to the URI - why do we need a web service description?". It's not very hard as long as you know everything in this list:

1. The server that hosts the service
2. The URL path structure to the resource you want
3. The required and allowable query parameters after "?" in the URL
4. What values are allowable for each parameter
5. What type of response you will get back

For an example, let's look at GData atom feeds from google. Items 1-4 are defined by the GData API Protocol Reference and item 5 is defined by the Atom RFC. Google and ATOM have done a great job of defining a nice RESTful architecture, and I expect it's pretty efficient and I think they got it right. We'll see below they're in the suite spot for using ROA.

But let's look at what it took to create this solution and ask if we can replicate this in the enterprise when we serve up enterprise data instead of blog feeds. Instead of an instance of a WSDL and a discoverable XML Schema contained in it, we have to find something that corresonds to the ATOM spec itself, which stands on its own 42 pages (and ATOM is a VERY simple mechanism). ATOM is nicely written and readable, but every ATOM tool maker should read and understand it because the spec is all you get.

ATOM is a widely known enough standard to have attracted tool makers to help. The problem is that that only helps when I make web services for things like ubiquitous blog feed formats backed with open standards. The tooling that chews on WSDL applies to *ALL* SOAP services. That's the difference without contracts: every service with a different payload paradigm is a one-off. I think this hints at an economic reality: ROA without service description contracts is indicated when the economics favor a very large, diverse set of web service clients that can collaborate on tooling up with format-specific solutions.

There's one special case that deserves comment: if the ROA web service supplies clients (such as a UI) within the same development team, then you can get reuse of tribal knowledge of the payload format. In such settings, you probably can be agile with what the web service provides, so long as no other parties are depending on it for critical business processes. I think of this as ROA in the MVC layer and not really as enterprise web services. This kind of ROA might often prefer JSON over XML and dazzle us with mash-ups. In summary, ROA without contracts works well within shouting range or globally, but not as well as SOAP/WSDL in between. Another way to state the same thing: if organizational politics are involved, you need contracts governing your web services.

Also, don't forget that to use a specific web service like GData, the GData API style information has to be created, maintained, communicated, and understood. These are all costs accrued per resource type. To be fair, before you call a web service to make a state change you need to understand what is going to happen in a way code reading the WSDL can't help with. But WSDL provides a natural place to describe the operations in human readable form, though admittedly like any form of code documentation, getting developers to create it will be a challenge in an enterprise setting. Still this improves over REST which offers no convention at all for a natural home for the description of what to POST to a given URL.

The key difference is this: with ROA, somebody has to create written specs like the GDATA API and the ATOM standard FOR EVERY distinct resource type. A resource provider might choose to use the same XML marshaling tools as in SOA to create XML Schemas or DTD's, or they might decide validation is to restrictive and force clients to read their stellar natural language description, like the ATOM team did. To describe what URL's GET or POST what entities, you have no help. REST is a programming language that hides the input and output types of it's call parameters and forces you to read documentation to know what they are.

In a corporate setting most of these specs corresponding to the GData API have to be created in house and would be maintained as documentation (and we all know how well internal IT shops do at maintaining documentation). Contrast this with WSDL that is going to roll out of your tools and be definitive, since it's a code level artifact.

Also consider what happens if you have 10 different RESTful services to deal with as a consumer. For each of them, you have to figure out all five of the elements above. Every service will have different semantics for invoking it and dealing with the response. That is, unless you adopt (and enforce) a standard for how to express these within your enterprise. Then the question arises, if instead of bolting on SOA features to ROA by spending your company's IT time and effort for no business value, why not just start with SOA and get the benefit of tooling created by the massive investments by Microsoft, IBM, Apache, Sun, Oracle, and so on? For this reason alone, I don't think ROA is ready for the enterprise in 2007. But let me be clear: this is a solvable problem and it only applies in settings where organizational interactions come into play.

Next time, I'll talk about the Uniform Interface vs rich behavior, and here, I think I see some wisdom in the ROA position.

Technorati Tags: roa soa web-services REST SOAP WSDL interfaces

As part of my ongoing look at enterprise architecture in 2007, I want to focus on the commonalities and distinguishing features between Resource Orientation and Service Orientation, comparing ROA and SOA. I'm witholding judgement as best I can, to try to be as objective as I can about what these two approaches to web services are. Accordingly, I'm not looking at practicalities. Those clearly matter, but they are separate and distinct from the purely technical differences. Practicalities change over time, but the technical pieces probably don't. I'm considering SOA implemented as SOAP web services, in angry defiance of people who say SOAP is just one implementation mechanism of SOA. While that's reasonable language, it has little value to me in the real world to continually say "SOAP based SOA" instead of just "SOA". There's no other widely adopted implementation of SOA with market and mindshare significance. ROA, on the other hand, is a new term for a particular perspective on REST. The flag bearer for ROA is the book RESTful Web Services by Richardson and Ruby and I follow their interpretation of ROA as faithfully as I can.

The principles of SOA are pretty well known:

1. Open Standards - HTTP, XML, XML Schema, SOAP, WSDL, various WS-*
2. Loose Coupling - Service consumers do not couple to implementation platform, but instead rely on open standards or metadata.
3. Reusability - Services are intended to be highly reused. The goal is a single authoritative service, meeting the needs of all consumers.
4. Contractual - Services adhere to an interface agreement artifact (a service description) that is functionally definitive of the interface.
5. Discoverability - The service contract is an outwardly descriptive and externalizable metadata artifact suitable for forwarding by 3rd parties to potential service consumers
6. Autonomy - Services have control over the implementation of logic and technical machinery they encapsulate.
7. Statelessness - Services generally do not maintain conversational state, Every call is self contained.
8. Rich Behavior - The service contract exposes many kinds of method invocations and associates them logically with the data structures described in the service description.
9. Metadata - Both requests and responses have a clear delination between metadata and payload.

The principles of ROA, as presented by Richardson and Ruby are easily understood:

1. Open Standards - HTTP, often XML, possibly XML Schema or DTD
2. Loose Coupling - Service consumers do not couple to implementation platform, but instead rely on open standards.
3. Universal Addressability - Resources have a URI which serves as a single global name that defines their identity.
4. Uniform Interface - All resources support a small and standard set of behavior methods (GET, POST, PUT, DELETE). Often only GET and POST are used.
5. Behavior Conventions - GET never has side-effects, and anything without side-effects should use GET. PUT and DELETE should be idempotent (repeating them has no additional affect).
6. Statelessness - Services generally do not maintain conversational state, Every call is self contained.
7. Representations - A resources can have tangible representations as bits in various MIME types, but the resource is different than the representations. Representations of a resource may vary with time, and the set of available representations can also grow and shrink over time.
8. Connectedness - A resource can (and should) reference other resources by name (URI) in the representation.
9. Autonomy - Resource providers have control over the implementation of logic and technical machinery they encapsulate, subject only to the uniform interface behavior conventions.
10. Minimal Overloading - parameters to state changing operations should not be used solely to select different behaviors. Heuristically, if an operation parameter could be converted to part of the URI, then it should be.

OK, so ROA and SOA each have open standards, a loose coupling concept, autonomy, and statelessness. Those are probably properties of "Web Services", which generalizes both.

Here are the key differences between SOA and ROA as I see it:

1. Contractual vs Not - SOA offers contractually, discoverable interfaces, while and ROA expects designers to solve problems as they set fit.
2. Rich vs Uniform Behavior - ROA has the uniform interface applied with minimally overloading, while SOA provides rich, discoverable behavior
3. Differentiation vs Equivalence of Data and Metadata - SOA relies on metadata to inform behavior, while ROA uses universal addressability, connectedness and design time conventions for representations and URI's to blur data and metadata together.

Let's examine these in depth.

In SOA, the client and server have a contract that specifies the service operations, endpoints, and input and output data formats. Typically the artifact for expressing the contract (WSDL) is closely tied to the physical implementation of both the service and the client, so that there it is expected to be definitive. In ROA, there is no contract guarantee and the interpretation of resources address mechanisms and response types must be based on knowledge not contained in the system. Often external standards for particular sets of resources arise. These can be open standards like RSS or ATOM, or local standards like a corporate schema or DTD for the output. The local standard might require the resource to self declare it's format, or it might not. There is no standard for how to structure these standards, so resource designers are free to meet needs as they arise, but resource consumers have little assurances over such designs.

The second difference is the types of supported operations. SOA allows a service to offer a rich set of discoverable operations. ROA has specified the operations as a small fixed set with standard meaning. Where an SOA service may be a logical grouping of many operations, each with an independent name within the service, ROA flattens each operation out to it's own URI. The logical grouping might exist within the path of the URI. The minimal overloading principle counsels the implementer to refactor an ROA resources that masks rich behavior within it's parameters, so that the URI fully differentiates separable operations.

The final critical difference is that SOAP messages have an envelope which contains a header and a body. The header concerns information about the handling of the message, while the body is the message itself which must comply to the validation rules for well formed messages defined by the contract. ROA, in contrast simply treats metadata as part of the resource representation, possibly leveraging connectedness and universal addressability to link to other resources that describe the orignal resource. It's up to the conventions and standards surrounding each resource to define the expected behavior associated with it.

This is my attempt to simply define what ROA and SOA are and to observe their common and differentiating characteristics as objectively as possible. In a future article I'll get into dynamic qualities and practicalities, and someday I'll try to combine the technical with the practical and draw conclusions about the suite spots of each.

Technorati Tags: roa soa web-services REST SOAP

At work we are going through a period of architectural planning, focused around integrating our internal systems. There seem to be a mix of architectural paradigms that are in fashion these days, and I intend to explore them over a series of posts here. In my estimation the reasonable architectural paradigms to advocate in 2007 are:

• Service Oriented Architecture (SOA), typically using SOAP as the dominant web service. This is probably the leading architecture motivating IT spending these days.
• Event Driven Architecture (EDA), typified by asynchronous message notifications and message buses. Messages are often XML or serialized objects.
• Resource Oriented Architecture (ROA), typically using REST based web services, often with XML payloads. This is a relative newcomer and represents a mesh of a backlash against SOA and a progressive fundamentalism over the symmantic web.

Ones I didn't include, and why:

• Event Stream Processing (ESP), stateful analysis of streaming messages turning SQL on its head by storing queries and issuing data. This is just too new, but it's definitely an exciting area to watch. Maybe in 2008 the big debate will be over stateful vs stateless bus architectures.
• Enterprise Service Buses (ESB), Many see EDA and web services as complimentary and wrap Enterprise Service Buses around both to handle routing, transformation, and encapsulation of providers and consumers. I see an ESB as an enabling ingredient, but not an architectural paradigm that stands alone. Bobby Woolf, co-author of Enterprise Integration Patterns agrees. That said, a good question is: "When should I mix in an ESB to my enterprise architecture?". I may blog on that separately.

OK, so the big question to resolve is SOA vs ROA for web services and then when to use messaging instead of web service calls. Maybe with good internal architectures, web services manifested as both SOAP and REST simultaneously are possible. Amazon S3 does this, for example.

SOA says to deploy solutions embodied within services that are reused at the system level. Service in the flavor I'm considering are generally constructed using XML and XML Schemas, and interoperate using SOAP and WSDL. WSDL really ties the whole thing together, providing a techncial description of the operations of the service and the XML inputs and outputs of each such operation. WSDL also provides the endpoint information too. Doing SOAP right requires tooling - nobody writes stuff by hand. Good tools take a WSDL document and generate everything you need. SOAP functions by taking an XML document, and wrapping it in an envelope and decorating it with headers. Operations, even simple ones are generally going to provide XML input and XML output.

ROA is a particular template for a RESTful web service, using URI's, HTTP, and XML. ROA names resources with URI's and accesses them via HTTP in a way that emphasizes conformance with the standard operations for HTTP. ROA adopts four principles: addressibility, statelessness, connectedness, and the "uniform" interface of HTTP operations.

I'll be exploring this in depth over the next few posts.

Technorati Tags: soa soap rest roa WebServices architecture