But lets be honest, that’s little awkward.

Why? because I’m forcing you to see a bit more about how RawkX works then you really need to know. you shouldn’t care what the method for parsing a line is called. You shouldn’t need to know what class it’s in. What if I change the method name? That would cause your parsing to break if you updated versions.  It wouldn’t be your fault, it would just be the result of me exposing too much to the user.

The good news is version 0.2.0 is an attempt to fix, or at least improve, that issue.

Now to specify how to parse your logs you can simply pass a block to RawkX which specifies the logic to handle your log. Lets just take a look at an example. Below is the same parsing logic from my previous example, made to work as version 0.2.0 intends:

require "rubygems"
require "rawkx"

RawkX.new do |line|
  fields = line.split
  [fields[1], fields[0]]
end

To me, this feels cleaner and safer. This block will be used in the same manner as the parse_line method was. it will get called for each line of the log, and it expects a key/value pair to be returned (or nil if the line is to be ignored). As with the previous version, if you do not pass a block or over-write the method, it will fall back to standard Rails log parsing.

Version 0.2.0 has been released to rubygems.org and, again, the source code for RawkX can be found here

Recently when I was at work we decided we wanted a Rawk-like report for another service our application was using, which was written in house, so we could observe its performance.  I was mildly familiar with Rawk, so I brought up the Rawk source code and started looking through to see what pieces we could reuse. Without much thought I began pulling pieces over, starting with the most obvious pieces and working my way down. It didn’t take very long for me to get what I wanted, but when all was said and done it started to sink in that I had ended up using almost all of Rawk’s original behavior, and simply changed how it parsed log lines. As any developer should, I realized I had created a significant amount of unnecessary duplication and felt compelled to find a way to prevent me from having to do it again.

Hence, the idea for RawkX was born and a couple nights later a first version has been published.

RawkX is simple. It is a slight refactoring of Rawk to allow it to be easily extended with different parsing logic. In short, the goal is to give you an easy way to generate Rawk reports for any log format without needing to think about anything besides how to parse your log.

For me, RawkX is also somewhat of an exercise and was a reason to play around with building and creating a gem for something I have actually had some use for. So maybe some of you will find it useful too!

Enough of that though, how do I get it and how does it work!

There’s no surprises or secrets getting the gem installed: gem install rawkx

Next, Lets say you have a log of actions that looks something like this:

3.24 Read user1
5.0 Write user1
2.245 Read user2
3.4 Read user3

Not complicated, but perhaps you had a large number of these actions and you wanted to generate a report to see the performance of them. With RawkX, that’s pretty simple. Just create a ruby file which does the following:

require "rubygems"
require "rawkx"

class RawkX
  def parse_line(line)
    fields = line.split
    return fields[1], fields[0]
  end
end
RawkX.new

Lets look at this for a second. We are reopening the RawkX class, and redefining the parse_line method. This method recieves a line from the log and is expected to return a key/value pair. The key is the action or item you want to measure, and the value is the time it took to complete. Nothing impressive, but a few things to note:

• The values returned must be in the order used above: key first, then the value/time
• Only two values are expected to be returned currently.
• If you want a particular line, or some type of line, to be ignored simply return nil and RawkX will move on to the next line without doing anything further.
• The parse_line method is called for each line of the log. If you want to save values between lines, you can do so using something like instance variables to remember things from previous lines (see how the default implementation of parse_line is done).
• If you use RawkX without over-writing this method, it will use the default Rails log parsing.

You can now run this file and pass in your log information (either through stdin or using the -f parameter to specify a file):

ruby [filename] -f mylog.log

ruby [filename] < mylog.log  mylog2.log   …..

Doing this should show a Rawk report for your log. I’m not going to go into details about the report. it should be fairly obvious and familiar if you’ve ever used Rawk.

Other options available when running RawkX reports are:

-w <count>

This is how many results you want displayed in the final “Worst Requests” list

-s <count>

This is how many results you want displayed in the results for the other report areas.

And, for the time being, that’s pretty much it. version 0.1.0 has been pushed out and available to download at rubygems.org. I have some stuff I want to try with this tool, but this seemed like the minimal amount required to be useful. As such, I wanted to get it released so I would feel like I accomplished something!

Also, the source code is posted on github here.

As always any feedback is welcome and if you ever end up using it, tell me and let me know what you think!

The puzzle itself can be found here. My javascript code can be found here.

What is it? It is an automated testing tool for Java Swing applications. There are lots of things I want to say/write about this project, but I think I’m going to hold off on all of those ideas for now since I feel they would do more good as a series of seperate posts instead of one disorganized post.

I took a little time a couple days ago to start setting up a simple website for the project though. Please note: The site in its current state of writing this does not have all the content, features, and design I’d like. However, I wanted to at least get something simple online quickly to create some sort of public presence about the project. As time continues in the weeks ahead, I will be adding and changing the site around as well as writing about the project and its development.

For now, check out Muffinman and keep an eye out for more information in the weeks ahead!

So what was the problem with the interactive shell?
Well, for starters, there was no solid organization for how I tried to parse or interpret the commands entered by the user.
What do you mean by that?
At the time of starting the project, I had absolutely no experience or knowledge of compiler theory or the general organization of a compiler/interpreter. In other words: I didn’t know that a compiler is usually broken up into a lexer and parser. As a result, no formal tokenization really ever takes place, which doesn’t make the process of parsing and evaluating expressions any easier.

I don’t want to spend too much time talking about what I didn’t do correctly, I just want to mention some points which inspired me to take another stab at a similar problem in a more intelligent way. As I mentioned, the interactive shell was primarily motivated by my desire to build something in Javascript and it was just something to keep me coding at the time.  It was not about trying to learn intelligent methods of parsing. Recently however, I decided to try to build a basic interpreter in C++ and I wanted to go about it in a more organized fashion.

All I wanted was the ability to, once again, evaluate arithmetic expressions and to allow the creation and use of variables in these expressions. Instead of being an interactive shell, it would simply read in a full script from a text file and evaluate the expressions line by line and print any results out to the console. I admit, this is not terribly useful, but I wanted to try and do things more intelligently.

What do I mean by ‘more intelligently’? I mean that I wanted to have a formal lexer which performed tokenization first, and then these tokens would be used by the parser to evaluate the expressions based off of a well defined grammer. So what all is needed?

First, I defined the valid set of tokens which are allowed:

NUMBER (an integer number. no decimal places)

IDENTIFIER (basically a variable name. can start with upper or lower case letter. can contain letters, digits, and underscores)

LINECOMMENTS(the usual ‘//’ for line comments is allowed. these arent actually technically a token since they are ignored by the lexer)

PLUS(+), MINUS(-), DIVIDE (/), MULTIPLY (*), EQUALS (=), SEMI(;), LPAREN( ‘(’ ), and RPAREN ( ‘)’ ).

Given these tokens, I define a very basic grammer for the different types of statements which are valid. Basically we only have 2 types of statements: a straight forward expression to be evaluated, or an assignment statement where the value of an expression is placed into a variable.

An assignment statement is defined as: IDENTIFIER EQUALS expression SEMI

where expression is defined as one of: NUMBER or IDENTIFIER or LPAREN expression op expression RPAREN SEMI

and op is defined as: PLUS or MINUS or DIVIDE or MULTIPLY

Furthermore, the value of a straightforward expression is printed to the console while nothing is printed in an assignment statement. So, an example script could end up looking like:

//This is just a simple script…

(3+4);
(12 - (3*7));
((10-5) * (3+5));
(((5-2) + (3*2)) + 10);
A = (3+4);
A = (A + 4);
B = A;
B;

The syntax is far from elegant, but that’s not really the point. The parsing is my attempt at something resembling recursive descent parsing, which you can read a bit more about here.  My actual code is heavily influenced in style from that in the article. However, I did do some things a bit differently, especially because I also wanted to actually evaluate the expressions as I was parsing them.

It’s still nothing amazing, but I feel like my code is more easily understood and organized for now. speaking of the code, you can get it here. I wrote it in Microsoft Visual C++ Express Edition and I simply zipped up the Visual Studio project to post. Take a look at it if you’re interested and feel free to share any comments or critisism. Its getting late, and I’m off to bed. Goodnight….

As I approach graduation I’ve become more and more torn as to how I ought to spend my time.  I have more ‘free’ time than I have had in years, and plenty of ideas and projects to use it for. However, over the last few weeks I have spent most of my time thinking about what I could be doing, instead of doing much of anything. I decided to create this blog in order to help stop this. In an attempt to no longer think or worry too much about jobs, moving, classes, or anything else. I have decided to focus on doing one thing: coding.

This blog might be used for all sorts of things. Perhaps sharing thoughts about programming and software development in general and anything else that is on my mind.  But most of all, it is a place for me to write about and share the projects I want to build in my spare time. This blog is a place for me to start seeing some sort of  organized results for the things I create.

And, obviously, it’s a place for you to see it all as well…

On the right side is a link to where I plan to post some info about projects as I work on them. I’ll probably write more about projects of interest in separate posts as well. For now, I’ve added a link and a little description about a simple interactive shell I wrote a while back. Check it out. More will be coming.