BloggingON

Hi Everyone,

I know it has been some time since I last posted (over two weeks), but rest assured, you have not missed one iota of science. These last two weeks have been somewhat slow. I have been finishing off my ICSOC'12 paper and have since sent it to my friends and family for brief review. I will be receiving the first critique back today (and I'm sure I'll feel bad about it) which means I can start doing the final tweaks to my paper. I hope to have it finally submitted and forgotten sometime next week.

I have also started the groundwork for better trade-off analysis in my framework so I will be reporting on the outcome of that soon. Thesis writing is very soon to start and tomorrow after my meeting with my supervisor I should have a clear direction in what I need to do.

I will keep you inform as I make progress.

As a write to you, hopefully my supervisor is dissecting my most recent paper, cutting content and correcting the remaining stuff. We have roughly 2 weeks until the paper deadline and I feel that we have certainly enough content - we just need to revise it.

As I mentioned earlier, I have been busy trying to profile my Phidget device at certain resource loads. This has been mostly successful, however memory contention is harder to prove on my system as we have no swap space allocated. Still, the results are good for CPU Load response time delay so at least we have more justification for our work.

Soon, I will upload the code to "Mallocinator" - a tool that Stephen Wattam originally made which I have added to. My improvements include allowing the user to specify what physical memory load the user would like (i.e. 65%). This is not dynamic however and will only take into account the initial memory load. Once it has been commented, I will upload it to my academic space. Hopefully it will benefit one of you - or some other poor academic trying to do system tests under constrained resources.

Hopefully by next week, I should be able to let you know how the paper is doing.

First off the bat is "I'm sorry". I have been very busy recently and as such haven't had the chance to update the blog.

Mostly, my work has been finishing off the WICSA'12 paper (corrections) and writing the new ICSOC'12 paper - which, hopefully will be accepted. As part of this new paper, we have wanted to get new data for certain scenarios the framework may encounter but also we have wanted to justify certain (previously arbitrary) things in our work. As such, I have been busy coding benchmarking tools to help justify why our framework acts at 85% CPU load rather than X%.

One new tool I made is CPU Thrasher, a program that attempts to maintain the CPU load of the platform at X%. The reason being is that I can run my framework at these CPU usage levels, allowing me to benchmark its performance at different levels of resource saturation. The link provided sends you to my academic site, where I will be posting all my tools (as I comment and tidy them). Currently, only the Linux version is available - Windows will appear soon hopefully.

I must continue writing my paper now, but I thought I'd keep you up to date and offer you a free tool, should you be doing the same as me.

See you next week.

Update : If I actually have any followers and you happened to download my "CPU_thrasher" tool, know that it was full of bugs and didn't work correctly. The new updated version works well(although finds it hard to get above 90% CPU load). It is typically accurate to 1%+- and the settle time has been greatly reduced.

I am writing to you, not from my bed, or couch watching TV and drinking beer, nor frolicking in the wilderness, from my office. I still have work to do regarding my paper and as such find myself in the office on a bank holiday.

I have pretty much finalised my WICSA12 paper, although I'm waiting for final feedback from my supervisor. I can't imagine him being too unhappy as I have done most of his suggestions. Hopefully he will be happy with the paper. Regardless of feedback, I intend to send the paper off to my dearest friends for proof reading and so forth in the hope that they can find any embarrassing mistakes and fix them.

Whilst I've been waiting, I have started work on the ICSOC12 paper (6th May deadline). Much of the paper content from WICSA12 can be reused (such as data and background), but much will also have to be rewritten with new content added in an attempt to placate the 'call for papers' content request. I should be able to get this paper out and hopefully being published there will mean a trip to Shanghai. It should also mean less stress on Viva day.

I have also decided that occasionally, I may write little technical blogs so that people might actually want to read my weekly blog. So far, it's just me ranting on about my Ph.D and how I've been working late when instead it should be about how I was able to save a dog from the path of a speeding train... or something like that.

I will keep you posted nonetheless.

I finally finished yet another draft of my WICSA'12 paper and have sent it off to be ripped apart by my supervisor. I now sit around waiting to see what his comments are, although since he wont start reading it until tonight, today has involved me floating around in Limbo with nothing to do - except write blog articles and write some mundane code.

That is basically all that has happened. I have been working towards this draft deadline and now that it has been and gone, I find myself doing nothing and feeling extremely guilty about it - this is what a Ph.D does... make you feel guilty about relaxing.

I submitted the abstract to WICSA'12 so I don't have to worry about forgetting about doing it and ruining my last opportunity to enter WICSA during my Ph.D. Below you'll find the abstract so you can read and decide whether the paper seems interesting enough...

A Runtime Resource-Management Framework for Orchestrated Embedded Service-Oriented Systems

Service-oriented Computing (SOC) has shown rapid growth in recent years, with numerous examples addressing complex software and system runtime problems. Embedded systems are particularly interesting in this respect. As they are typically physically isolated and critical in nature, embedded systems can greatly benefit from the SOC paradigm as it allows software systems to be dynamically composed and reconfigured using services discoverable at runtime. However, as embedded devices are often constrained by limited resources, resource-contention between services on the orchestration platform can cause significant falls in Quality of Service (QoS). This emergent QoS is difficult to anticipate before deployment as changes in QoS are often dynamic and influenced by a number of factors. In this paper, we present EQoSystem, a runtime resource-aware framework that combines resource monitoring with dynamic service orchestration and workflow management. We demonstrate the efficacy of our framework using a case-study deployed on an embedded platform.

Now that I've mildly entertained you, I'm off to fight lions and sky-dive... or whatever people do when they have no work to do...

P.S. I completely forgot to tell you how data collection went and whether the data benefit the paper... Long story short, we ended up with 20 instances of data ( 4 configurations with 5 simulations each configuration). We had plenty of data to play around with - hopefully my supervisor (and the research community) will be happy with the results.

Sorry for the delay in posting an update on my work. I have recently been distracted in paper writing (for WICSA'12) and as such have not found the time to post.

Working seems to be going well. Effectively, I have roughly 10 days until my paper needs to be submitted but already I have most of the content ready (9/10 pages). I have just sent off the first draft to my supervisor for review and hopefully he should get back to me by Wednesday. I hope to continue spamming him with new drafts until the deadline.

Mostly, I've been working on the paper, although since the beginning of last week, I've been frantically working towards gathering more data. After speaking to my friend and previous officemate Rachel (or should I say Dr Burrows), I decided that I needed to collect a set of data that measured resource of a system without resource management. Then, after speaking to Gerald (my supervisor), I needed 2 more datasets to cover scenarios that I had not thought about. As such, I was collecting data until Thursday evening, rather than only until the previous Friday. As such, I feel slightly behind where I wanted to be.

I should still make my deadline, and hopefully the content quality will not have suffered - in fact, with the extra data, I'm sure the paper will only benefit.

I will update you on Friday with my progress (hopefully it will be 10 pages of progress when we next speak).

I almost feel confident at this point to say that implementation has been done (or at least until the end of this current paper-writing session). Besides the occasional tweak, I really shouldn't have to touch anything to gather data. Some minor tweaking has been done already, such as artificially constraining the SBC resources further, but hopefully that should be it for now. This is still the issue of object profiling consuming system resources but this shouldn't pose too much of an issue - I might still fix this by pausing the application whilst the framework is gathering this data (purely for experiment reasons), but I don't think this will be required.

Today, I plan on running the simulations I had prepared earlier. These simulations should hopefully demonstrate various resource usage patterns, although I am inclined to continue gathering data as time goes now that it seems to work reliably enough. Compared to the last case study, this one has behaved itself rather amicably (although some problems have crept in - nothing that couldn't be fixed though).

Yesterday, I discovered an issue with task swapping that meant that a sequence could be executing whilst a sequence is being changed. This mean that certain parts of a sequence that depend on each other might be swapped out. Synchronisation of certain variables along with a well placed mutex has hopefully prevented this issue from appearing once again.

Other work done includes playing with the new toys my supervisor has acquired for me. Disappointingly, rather than the lego I requested, I have been given 2x20 LCD screen and a GPS receiver. Whilst it is too late to incorporate the GPS receiver into the current case study, I have added the LCD screen and wrapped its supplied library with my own code, allowing for 'ticker-tape' text to be displayed (if the text exceeds the 20 character limit).

I will keep you updated with my progress. Hopefully by next Friday, I hope to have completed data-gathering and will have performed the data analysis. I should have written the majority of the paper too!

Watch this space!

Today has marked the end of implementation and so begins data gathering (and bug fixing). The case study application has been completed and should hopefully allow us to demonstrate the efficacy of the framework we have developed. Not only has the application been implemented, but the XML document informing the framework of workflow and task annotation has also been done. Now, all that remains is to deploy it on my Phidget SBC.

Truth be told, some minor implementation changes to the framework will be needed to ensure that the data collected is correct and as accurate as possible, but I foresee this only taking a short while. All this should be done by the end of next week (including data gathering).

I will let you know how things progress the same time next week.

This is another update on the status of my Ph.D work.

During this week, I have been implementing the client side of my case-study; this will allow me to demonstrate the efficacy of my framework. So far, work has been going well although I am just very slightly passed the self-imposed deadline I set myself a few weeks ago. As mentioned previously, the client-side to the application I have been developing is to act as a pager, notifying the carrier of 'assets' within their area. The movement of the user is simulated and updated and specific time intervals. The simulation module passes events at specified time intervals, thus allowing me to test my framework in a deterministic manner.

The application itself has been implemented and seems to work correctly - and better than the previous case study application (which crashed and failed intermittently - causing me great pain towards my last deadline). I am currently fixing the implementation of my framework and workflow tool to allow for greater flexibility in changing the behaviour of my application. This I imagine should be completed by the end of Monday, thus allowing my over 3 weeks to write the paper and gather results. During this time, I hope to figure out different ways to monitor memory consumption of services - as it is a rather slow (and costly) process to do so.

Hopefully, this following week should be productive.

Hello my band of followers, today I continue you with the progress of my Ph.D.

Recently I have been working on my case study so that I will be able to gleam some data from it (all in preparation for my two new paper deadlines - mentioned in an earlier blog).

The case study is forming nicely and has been revised to be an application suite for tracking 'assets' around campus; assets can include such things as computers, printers and any other electrics around campus. A member of the on-site maintenance team roam campus (in a patrol) answering maintenance requests and performing general inspections. A server side is responsible for adding assets (to make my life easier) and to add annotations to them such as maintenance reports and other information. The client side (which I'm currently working on) functions as a beeper, alerting the maintenance team to maintenance requests in the area.

A few additions have also been made to the framework; primarily, these changes deal with task pre-emption.

We now have the ability for the application developer to call one method but for other methods to be invoked beforehand (transparently). Imagine the developer programs their code to execute method B. It is possible for us to now add tasks before method B. To illustrate that :

A -> B (trigger) -> C } In what I just described, despite B being the method just called, method A would execute first.

This new feature will be handy in circumstances where data needs to be treated and filtered before being passed to the next service (think encryption or searching/filtering).

I will leave you to dwell on this blog whilst I implement the module responsible for driving the simulation. I will let you know of progress made this coming Friday.

I'm sorry for the lack of an exciting title, but I couldn't think of any way to express what has happened in the recent weeks.

Publication News
Firstly, my paper to ICSE2012 was rejected. This would be fine, if not for the fact that it was my last opportunity to publish there during my Ph.D. Still, I have 2 new opportunities to publish and at extremely good conferences; WICSA and ICSOC. Both of these conferences have submission dates in April, so I have plenty of time to ready my case-study.

Currently, my work has been focused on implementing new services for the case study and implementing new features into my framework to allow for the swapping of tasks at runtime.

Task Swapping
Imagine the scenario where you have a method call being performed, such as encryption. You have a number of different types of encryption that can be performed, but you have gone for overkill with 1024bit RSA rather than 128bit AES. RSA is perfectly fine until its drain on system resources (more specifically CPU load) start to cause QoS issues. What my framework is able to do now is swap out given tasks for other tasks (which have been earlier defined) when certain resource usage patterns have been detected.

Services
Since we last spoke, I had a small handful of services which I used as part of an application. I have increased this number of services to 8 (although only 7 are planned to be used). These extra services are :

* mySQL DB Service - Stores (as a directory structure) elements and annotations of elements. A concept of 'files' and 'directories' exist - although it is still fairly basic (files only have one owner (though this is easy to change if I had the time).

* Metadata Caching Service - Stores date retrieved from a given data source (in this case, the database service mentioned above). operates similar to the map caching service previously implemented.

* Encryption Service - Allows for byte streams to be encrypted/decrypted using RSA, AES, DES, and any other encryption algorithm (when extended with a plugin).

Finally
I have decided to make this blog a weekly thing in order to try and maintain a record of what work I'm doing. Different to previous attempts to keep this blog up-to-date, I have added it as an official event in my calender (which I use often). I am also working longer hours during the week in order to maintain a workload befitting of a third year Ph.D student.

I will keep you informed of my progress - indeed next Friday, if my calender is to be believed...

P.s. I have also updated my workflow tool to incorporate the extra functionality of my framework and have generally tidied the UI. Pictures are included, but forgive the awful look of it - UI design is certainly not one of my skills..

Hello all,

I just thought I would keep you up to speed with my PhD - especially with the small projects that keep me sane along the way. Recently, I've been working on profiling the memory usage of my framework so that I am able to justify its usage within resource constrained environments (its raison d'etre).

Previously, I was using a memory profiler called classmexer, which is a fantastic library that allowed me to use the Java Instrumentation package to inspect memory usage of the object and all of its references recursively. This was working fine, although there were references to class types within my objects that I didn't wish to be profiled (such as the ClassLoader) as they would already exist in a system without my framework.

So, my journey began to implement my own memory profiling agent (Novamexer) that would allow for a blacklist of class types to avoid profiling. After 2 days of debugging and comparing the output of my agent vs classmexer, I have finally completed my sub-project. Soon, I will be making available the source and libraries - just in case anyone is mad enough to use my code. If the original authors of classmexer would like to add the functionality to their library that my agent has, they are more than welcome; I'm almost sure they are better coders than I.

I will edit this post with a link to the code once I've documented and tidied it.

Hello children,

After a month of working towards my ICSE12 deadline, I have finally completed my paper and submitted. Hopefully, the paper will be well received otherwise I'll find myself in the all-too-familiar foetal position. It was a short paper (4 pages IEEE format) discussing my approach to designing a framework capable of mitigating resource contention within service oriented applications deployed on embedded platforms.

I would provide you a link to my paper, however until it is published (which it may never be), I can't do that for fear of idea stealing and being sued... However, in case you happen to be interested in my work, feel free to read over my first (and only) publication here.

In addition to my paper, I was also required to give a presentation to the Software Engineering Forum within the department regarding my PhD - basically a presentation that mirrored my most recent paper. For the most part the presentation went well, although some hard questions were asked which caused me to rethink some of my work/positions. I also disregarded some of the questions/advice as people trying to 'steal' my project and use it for their own means - as all researchers do.

I also had a meeting with my supervisor in which we discussed the direction of my work and the plan for my remaining time as a PhD student - hopefully thesis writing will start sometime in January. I will also be chasing more paper deadlines between then and October next year. Until the next meeting, I guess I have nothing else noteworthy to report on.

Until then, have fun!

In the last few weeks, I have been working on my framework in an attempt to allow for changes in program composition at runtime. Today that was achieved.

Previously, my framework was only capable of altering the 'orchestration of the application at runtime - an example might be to disable a service/object at runtime that is consuming too many system resources; this service/object would be rebound when required. Very recently I have implemented the ability to also alter the work-flow of the application. As such, unnecessary tasks can be removed from the work-flow whilst all the essential task still execute.

Because image uploading is broken, I'll demonstrate with Ascii:

(A) -----> (B) -----> (C)

Tasks A, B, and C exist within a sequence of tasks; this sequence is executed once task A is invoked. Imagine task B (or indeed C) are consuming too much CPU time or some other resource required for reasonable QoS of the application. Depending on the importance level of a task (based on annotation provided before runtime), these tasks can be removed from the sequence. The example below shows the sequence after the removal of task (B) :

(A) -----> (C)

So, that's my work in a nutshell so far. I am now going to start writing a paper for the ICSE conference (deadline 1st-2nd Dec) and implementing the case-study. And then... it's just a short hop-skip and dash to thesis writing...

And I thought a PhD would be hard work...

So, Just to maintain a good level of procrastination, I thought I'd leave a quick report concerning the progress of my PhD workl

My framework is much closer to implementation - I have given myself until the middle of next week to have completely finished it (including some 'service policies' - the things that dictate my frameworks behaviour. I'm almost sure that with some testing, I'll find that my work has gone hideously wrong. Still, preliminary testing has shown fairly good results and as such I'm fairly confident it should work - cue hubris.

Most of this week has consisted of getting the XML parsing working again along with the transmission of the generate task model to the client. The next task is to use this task flow as part of the application and to alter service composition via messages between service and client.

More news to follow!