May 14, 2013

Creating a Simple AMF Client for Testing

Writing about web page http://blog.ccare.me/blog/2013/05/09/creating-an-amf-client-for-testing/

A few months back I had to debug an Adobe Flex application that used Adobe Message Format (AMF) for client-server communication. In order to debug the web service independently of the Flex client, I used Adobe's Java bindings to build a Java Client.

Building a lightweight Java client provides an alternative way of accessing a BlazeDS endpoint, which I found very useful when debugging and comprehend AMF services. Having this a client also enabled me to write a programmatic API test that didn’t rely on running a Flash Player. I also found that I could significantly the amount of code I needed to write by making my data transfer objects inherit from java.util.HashMap.

Read more: http://blog.ccare.me/blog/2013/05/09/creating-an-amf-client-for-testing/


Correlating Log Entries Using Log4J and the Mapped Diagnostic Context

Writing about web page http://blog.ccare.me/blog/2012/12/21/correlating-log-entries-using-log4j-and-mdc/


Over the last couple of years, it has become increasingly common for web applications to be built using client-side frameworks that interact with a remote server via asynchronous requests. Unlike a traditional app, that makes a single request per page, a modern web application will probably make many asynchronous HTTP requests. This poses a challenge when analysing server logs, since it can be difficult to identify which requests belong to the same client.

I wrote a blog post I’ll describe a simple technique we’ve applied to retro-fit a client-aware correlation token to the server logs of an existing web application using a feature in Log4J known as the Mapped Diagnostic Context.


Read more: http://blog.ccare.me/blog/2012/12/21/correlating-log-entries-using-log4j-and-mdc/


Using Grails to Throw–up a Simple Interface Over JPA Entities

Writing about web page http://blog.ccare.me/blog/2012/03/29/building-a-demo-jpa-integration-for-kasabi/


I’ve been wanting to write a demo of integrating data from a database using object-relational mapping. The idea was that data in an existing database would be polled and loaded in an ETL fashion. However, in order to build a compelling demo, I wanted to generate a simple CRUD over my existing domain classes. I found that the Grails library really helped progress quickly.

Read more at: http://blog.ccare.me/blog/2012/03/29/building-a-demo-jpa-integration-for-kasabi/


Visible Ops

Writing about web page http://blog.ccare.me/blog/2012/03/23/visible-ops/

Last year I finally got around to reading the Visible Ops handbook. I’d been meaning to read it for a while – ever since I first heard Gene Kim speak at a conference last year.

All in all it’s an interesting read. I didn’t find the book particularly controversial. But, then again, it is nearly 7 years since the book was first published. I think few people would disagree that many of the practices are now part of the ‘received wisdom’ of operations. I also found it useful to get a reminder of the ITIL jargon. Using the correct terms for this sort of thing is something I’m a bit weak on, but as the authors say in the book, having a common language is vital.

Read more on my other blog: http://blog.ccare.me/blog/2012/03/23/visible-ops/


August 01, 2012

Final Velocity workshop

The final workshop I attended at Velocity was

"Managing the System Lifecycle and Configuration of Apache Hadoop and Other Distributed Systems"
/ Philip Zeyliger Cloudera

The talk introduced Hadoop and it's various components, which was handy as there always seem to be so many.

  • HDFS
  • MapReduce
  • ZooKeeper
  • HBase
  • Oozie
  • HUE

He discussed how different people were managing Hadoop hosts (PXE, kickstart, Cobbler) and software (packaging, file transfer, installer scripts).

An interesting comment made during the talk were that we still talk about distributed systems are clusters of nodes whereas Hadoop is all about programming a datacentre as if it were one node. So, for instance, puppet and chef are designed to manage N hosts running a number of services... but with hadoop you're trying to manage a N hosts running ONE distributed/failsafe service.

He introduced the Cloudera Service and Configuration Manager (SCM) which is essentially a central console with log and status aggregation and the Cloudera Activity Monitor, which he described as being like UNIX top for hadoop.


April 18, 2012

New blog

Hi! I'm in the process of migrating the content from this blog to http://blog.ccare.me. New posts will probably appear there.


March 23, 2012

Visible Ops, avoiding unauthorised change, and automated code–review

Last weekend I finally got around to reading the Visible Ops handbook [1]. I'd been meaning to read it for a while - ever since I first heard Gene Kim speak at a conference last year.

All in all it's an interesting read. I didn't find the book particularly controversial. But, then again, it is nearly 7 years since the book was first published. I think few people would disagree that many of the practices are now part of the 'received wisdom' of operations. I also found it useful to get a reminder of the ITIL jargon. Using the correct terms for this sort of thing is something I'm a bit weak on, but as the authors say in the book, having a common language is vital.

Working for a smaller company, I suppose some of the talk about audits and change-review boards sounded a bit overkill. Although it made me realise that the natural turn-off of these things is usually more to do with past experience of bureaucratic processes, rather than a feeling that audit and change-reviews are, in-themselves, a bad thing.

The other week I gave a lecture to students at Warwick about what it's like to be a software developer at Kasabi [2, 3]. I had lots of things to talk about: our product, our team, our culture, etc. However I wanted to share something about our working practices. The practices we use have certainly evolved over time, inspired from various methodologies and communities including: Scrum, Kanban, and dev-ops. However, if I had to choose one thing that I couldn't do without, I think it would be having a code-review process that's tightly coupled to the build/release process.

Nearly a year ago, our team moved to using Gerrit [4] as our principle review tool. The way it integrates with version control and continuous integration is fantastic. We were already following a continuous deployment model, but I feel that following the Gerrit workflow has helped us get closer to doing truly automated deployments. We're also a distributed team, so spending extra-time on code-reviews can make up for less time spent pair-programming.

So lets return to Visible Ops... for me, one of the biggest take-aways from reading the book was the idea that the acceptable number of unauthorised changes should be zero. Two points follow directly from this: 1) that code-review / change-review should be a straightforward and fast process (so that it's not routinely circumvented), and 2) that even emergency changes should go through some kind of review process. In fact, the authors of Visible Ops make the valid point that emergency changes often benefit from closer review. For me, the simplicity of using a review-tool (like Gerrit) that's built into the release process facilitates both of these goals. There have been times when I've needed to push an emergency fix. But because the tooling is relatively quick and simple to use, more and more I'm finding that I prefer to take these changes down the normal review path. Even though we're not doing heavy-weight audits, from an ITIL perspective, it's re-assuring to know that these changes are going through a robust, well-understood, and auditable process.

[1] http://www.amazon.co.uk/Visible-Ops-Handbook-Implementing-Practical/dp/0975568612

[2] http://www2.warwick.ac.uk/fac/sci/dcs/teaching/material/cs321/

[3] http://www.slideshare.net/cpcare/warwick-2012

[4] http://code.google.com/p/gerrit/


March 20, 2012

JPA and Kasabi

Over the last couple of days I've been trying to pull together a demo of integrating an existing JPA application with Kasabi. It's been a fun journey learning (and relearning) the JPA spec, and also working with some of the internals of the Hibernate framework. It's interesting that when a framework becomes really ubiquitous - like Hibernate, in this case - it's really easy to find online docs and howtos about how to use it, but it's far harder to sift through all that noise to find docs that help you extend it.

In the end, I did get it working: I re-used the annotation scanning plumbing that Hibernate provides to get access to the JPA configuration, then I walked the object graph marshalling entities as RDF (I used NTriples) before submitting them to a Kasabi dataset.

Of course, to complete the demo, I needed an actual JPA application, so I've had some fun re-learning Spring Roo and Grails in order to build a quick database-backed webapp. Normally I dislike scaffolding-based frameworks, but they're certainly useful for creating a quick functional app.

I'll submit a link to the code at some point soon.


Ways of Thinking, Ways of Seeing

It's nice to see Ways of Thinking, Ways of Seeing (ed. Bissell & Dillon) finally published [1]. I contributed Chapter 5 (see [2]), entitled 'Early Computational Modelling: Physical Models, Electrical Analogies and Analogue Computers'. The origins of this book came from a session which Chris Bissell organised at SHOT 2007. It's taken a while to find it's way into print, but I think it's an interesting collection of different perspectives on modelling.

Thanks to Chris Bissell and Chris Dillon for making it all happen!

[1] http://www.springer.com/engineering/robotics/book/978-3-642-25208-2

[2] http://www.springerlink.com/content/r071882m26486028/


July 05, 2011

Primitive types and Java performance

Reading article about primitives vs wrapper objects in Java. Nice to see some graphs and real evidence rather than hearsay:

http://vanillajava.blogspot.com/2011/07/low-gc-in-java-use-primitives-instead.html


Search this blog

Blog Tags

Galleries

Most recent comments

  • If you add [branch] autosetupmerge=true to your ~/.gitconfig , branches you push upstream will autom… by Chris May on this entry
  • What surprises me is that after having an expert look at my computer because of all the viruses he g… by Sue on this entry
  • Actually meant mvn –o install –Dmaven.repo.local=/path/to/repo :) by Charles Care on this entry
  • As you touched upon, 'digital' is sold to the consumer as newness. For example, digital TV and radio… by on this entry
  • Thanks Casey for your encouraging comment, I'm afraid that I can't take credit for the genius of the… by on this entry

Blog archive

Loading…
RSS2.0 Atom
Not signed in
Sign in

Powered by BlogBuilder
© MMXX