Over my development career I've had the opportunity to be exposed to 20+ languages and have seen the implementation of even more server side application frameworks. It's been a long trip that has included tons of ORM implementations and services above and beyond that. After a couple of years away from java development I embarked on a project and was unsuccessful finding a combination of services that I had gotten used to with different platforms.
As we assessed JPA, Hibernate, and Spring JDBC we found that these frameworks either over compensated with complexity and created other unwanted side effects or could not solve the problem by themselves alone. In a nutshell, we wanted something that could
- perform data access in a lightweight manner
- create and execute fixtures for loading data
- perform validation logic with the data services
- expose persistence functionality through REST resources easily
Before we built vessl, we looked into some ORM tools and found that it was complicated to decouple the aggregate relationships and define where the boundaries ended for individual results. This made the interfaces for them simple in that they took a single model object, but complex in that the services would be responsible for saving the aggregate types. This deeply nested value object made serialization difficult. In probably a mis-fit implementation of some of these, we saw extremely large payloads being passed back and forth when only a subset of the data was necessary. Our approach to resolve this was to design a set of accessors or Data Access Objects (DAOs) that were shallow.
Additionally we wanted a persistence mechanism that was course grained from a query standpoint to support criteria, sorting, and pagination. We knew based upon building large applications that building this into the DAOs would result in a very powerful and performant interface. A Spring JDBC implementation was created that allowed for us to define the DAOs and corresponding JdbcMappers.
We discovered that a lot of the code included scaffolding that was a responsibility of the developer. We improved on this and created a reflection based implementation of the DAO and voila we had something that was much simpler. We no longer had to create a JdbcMapper because of this, the performance tradeoff was negligible and the amount of coding reduced by a 5x factor.
Extending off of the relational data layer we created, we next started to focus on fixtures from the RoR world that were great for testing, seeding, and even load testing. We decided to define our fixtures in JSON instead of the RoR default yaml syntax and because we had the shallow object definitions, it was a great match.
Things were coming together for us, but we wanted to push this logic out further to the service layer so we moved our focus onto the REST resources. We had identified a "manager" layer in our architecture that really just delegated responsibilities to the underlying DAOs and did not much else. Since we knew we wanted nouns for our REST services, we were able to get close to 95% of the coding done by mapping the functionality directly into our DAO layer.
One problem with this was that the manager layer was where all the validation logic existed. Yikes, that seemed wrong to make the controllers fatter and kind of violated DRY for aggregates. So we rolled in a validation logic set. This we found was something we could roll across the project in spring to transparently apply validation to the models.
The consumption of the REST service as it stood was particularly chatty - an unwanted result of trading off shallow objects. We created the concepts of serializers which allowed for object augmentation in the REST layer and allowed us to change the contract for GET to include complex objects while POST/PUT could maintain that shallow contract. It was an excellent solution that allowed us to provide a simple interface without un-necessarily poluting the internal implementation.
Vessl isn't done, in fact there are several other aspects of the project that we didn't mention in this article including:
- environment configuration
- NOSQL persistence
- mega fixtures
- fixture generation
Rather than including a ton of code snippets in this article, we've tried to ensure that the README captured the essence of the project really well. Between vessl and the example web project vessl-web there is enough to explain the details with working code and snippets. In fact, you could copy aspects of vessl-webapp straight into your current java web project OR use it as a template for your next java project.
Understanding why the various functions in this project came about was a natural evolution. It didn't happen all at once, but it was well thought out and we are really glad to be able to return some of the good nature that the open source community has provided us.