Thursday, August 21, 2008

Spring jmsTemplate threading issue

I recently encounter a problem while working on some Servicemix bug and it turned out that Servicemix jms provider is using spring-jms jmsTemplate and when there are multiple threads trying to do send receive (receiveSelected() to be precise) some of the messages are dropped.

While doing further investigation I found that multiple threads are using the same instance of spring-jms jmsTemplate that is created by the jms provider and this was causing some threading issue so thing to remember is to make the calls to jmsTemplate.receiveSelected() (atleast this one) synchronized.

UPDATE: I was too quick to diagnose the problem as jmsTemplate issue. With more drill down I was able to reproduce the problem with pure jmsTemplate and ActiveMQ combination and looks like a ActiveMQ issue than spring-jms. My friend Gary is getting it fixed in ActiveMQ currently.

Monday, August 11, 2008

Changes to Apache CXF JMS transport to allow jmsDestination name

Recently Apache CXF JMS transport is modified in order to allow use of jmsDestinationName and jmsReplyDestinationName. Till now CXF JMS only allowed using jndi destination names to lookup queue/destination and was not able to use jms destination names to create queues/topics.

Here is the diff of the schema change to specify the jmsDestinationName and jmsReplyDestinationName.

Example jms:address looks like this:

<jms:address jndiconnectionfactoryname="ConnectionFactory"
<jms:jmsnamingproperty name="java.naming.factory.initial"
<jms:jmsnamingproperty name="java.naming.provider.url"

This will allow CXF consumer and services to use the real queue names (where JMS provider will allow the creation of the queues) as opposed to use jndi lookup.

handy tip on how to increase jetty HttpClient threadpool size in ServiceMix 3.2.2

I was recently working on some jetty related issue in Servicemix 3.2 and got Servicemix people to update servicemix-http component to have configurable jetty httpClient thread pool size. Before I forget it I wanted to make a note of it so adding this entry.

In Servicemix 3.2.2 and onwards jetty HttpClient threadpool can be configured using following entry in file.

Also, you can configure Servicemix to use Jetty httpClient per httpProvider (by default a single client is shared among all httpProviderEndpoints) by adding


Also from ServiceMix 3.2.2 onwards Jetty HttpClient uses nio channel selector implementation of jetty. (previously it was using socket connector which was blocking I/O and was causing thread locking in thread pool in some stress conditions)

Friday, August 1, 2008

टेस्टिंग मराठी ब्लोंग एंट्री.

अरे वा मराठी एंट्री पण चालते आहे बर्यापैकी पण माझे मराठी कच्चे आहे।

Friday, June 6, 2008

Running CXF jms_queue sample in Tomcat

Recently, I wanted run Apache CXF jms_queue sample in tomcat and came across some problems setting up stuff so want to note the things that I had to do for it.

1. need to create the spring.xml and web.xml descriptor for deploying the jms_queue server inside Tomcat.
2. Need to add CXF and ActiveMQ jars to the container classpath. I haven't tried yet to bundle these inside the application war file that is next on my list.
3. Build the sample, create war file and deploy it to tomcat by copying it to webapp directory of the container.
4. start jms broker (I user ant task from the sample that starts embedded broker).
5. Start tomcat container. This will deploy the copied war and start the CXF jms_queue service.
6. run the client from the commandline against the service deployed into Tomcat.

Looks simple isn't it? But I had to spend 1 day figuring out how to make this work.

glitch 1: Service creation wasn't finding WSDL for my service. Reason: WSDL was bundled under WEB-INF/wsdl directory which is not on classpath so I have changed the build.xml to put it under WEB-INF/classes directory.

glitch 2: Wanted to enable logging by adding file but wasn't able to find it on classpath. Reason: same as glitch 1 and solved when added it to WEB-INF/classes folder.

I am attaching tar file of my sample directory for future reference here.

EDIT: I couldn't find the way to upload the tar file so I am editing the post to remind me to post the changes in my next post. Not so good but can't help.

Here is how web.xml looks like:

And my spring.xml looks like this:

Finally here is the link to zip file that has the full example.

Wednesday, May 14, 2008

Nexus 1.0.0-beta-2 testing

I managed to run Apache CXF and Apache Servicemix 3.2 builds with some glitches.
1. CXF build was fine without any problem (didn't run tests though)
2. ServiceMix 3.2.2 build failed to strangely due to 2 issue in first run.
a. Couldn't find log4j dependency (which should have been resolved indirectly from some other component dependency. I have to specifically add the dependency to POM file, maybe I should submit patch to Apache SMX for this.
b. Failed to run "crawl" goal for servicemix-archetype-catalog module. The puzzle is it doesn't do anything if I do a full build and reports failure but passes if I run mvn:install inside the component directory.

Overall, Out of 3 requirements of Builds 2 are successful., 3rd (IONA Fuse ESB's Logisticx Demo) I haven't yet tried. That is the next thing on my menu now.

All in all, I am pretty happy with the experience this time. (unlike my previous attempts to get proximity going without any UI based intuitive configuration option or lack of access to docs.)

I am surely looking forward to tune the configuration and use it for my regular work.

Oh.. I forgot to add the link to the download site and documentation that I accessed today.
Here are the links.

Nexus Home
Nexus Download

One thing I mentioned in last post was about beta-3. By the looks of the download are it seems the download page does the announcement of beta-3 but the actual download storage area doesn't have the necessary files in place yet.

My experiments continues......................

I think, I should do some other experiments also and keep the blog going. I still have to show my stab at Digital Picture frame out of old Toshiba laptop. Even though I got it to work properly in January on my Daughter's B'day Haven't got a time to touch it and refine and fix the backside of it properly. Hope to find some time to finish that and take some photos.

another experiment with Nexus now in progress

Today I stumbled upon Nexus repository manager while browsing and found a 1.0.0-beta-2 available for download. So with some excitement I went ahead and downloaded it and currently trying out.

Just now when I went back to check the link I realized that there is a new version nexus-1.0.0-beta-3. I will try patching my installation with it and will post the results.

One plus point, I can't resist mentioning here by initial looks is better UI based repository management which makes life much easier. Got the setup starting to work in no time and speed is also good.

I did stumbled upon the corruption of the pom file so doing clean build again to see if I can consistently get the same problem.

Wednesday, March 19, 2008

Something about the proximity problems I had.

It's been 4-5 days on and off I am trying the Maven Repository products to get working with Apache ServiceMix 3.2 build and test.

Goal: To get Apache ServiceMix 3.2 build and test working.

I think I should summarize the experience as I am pretty much giving up on them (or probably not but not sure yet). One thing is sure I liked the exercise even though frustrating some times.

1. Proximity :
* With no longer in development and Subversion repository, wiki, forums gone it is hard to get the bundle. I manage to get it from Bruce Snyder's Proximity blog entry.
* Hard to configure as no documentation around. (Thanks to Bruce Snyder for helping out)

Problems I faced are:
1. After adding the required bean entries and FSStores and Remote Peer locations It didn't work.
2. So tried sequencing the entries based on preference and updated for each beam entry accordingly.
3. After fixing this got the Repository logic properties wrong on some entires, fixed that and still proximity wouldn't try all the repositories listed.
4. After lot of time (without any docs) found that I was putting all the repository entries in separate group and as per my settings.xml maven was only requesting on public group so modified the configuration and updated for all required entries to public.
5. At this stage got maven to talk to the repositories and here I hit the big problem (may be a bug in proximity) for some of the dependencies which are available in more than one repositories it was corrupting the Jar files and POM files. This caused builds to fail on invalid POM and jar file. The POM file of the artifact in the local repository was replaced by some kind of Proximity generated artifact summary.

At this stage I gave up and moved on.

Nice to have: Some documentation around configuration, UI for adding or editing configuration. Hand editing the XML configuration is error-prone task.

Apache Archiva 1.0.1:
Similar to Proximity had some configuration issues. Didn't know need to add proxy connector for each repository entry. Once the repository entries, connectors and all entries in one group I was hit by a issue of cookies didn't like the url name of some of my entries saying they do not confirm to RFC2109. It didn't talk to that repository at all. Tried putting IP address but no luck archiva would resolve it and skip the repository with same error.
Result: Stopped experimenting.

One strong point is the ability to configure using UI in browser. It makes it much easier to view and edit configuration compared to other 2.

Nice to have: Little more speed in download artifacts. Found it slow even for cached artifacts. And also the fix/workaround (or visible documentation somewhere if it exists) for the RFC 2109 error.

Artifacotry 1.2.5 :
After playing with this I even if the build was not successful I started liking it for the speed of the downloads especially compared to Archiva. With some more efforts and configuration I got Artifactory to talk to all the repositories I wanted. It also gave the same RFC2109 error so changed to IP address and it still showed the error but was able to contact the repository alright.
Next issue was grouping and search order. From documentation page I manage to find out the search order and learned about the VirtualRepo. So Added new virtualRepo and added reference to my repository entries in the order I wanted.
So far so good stared downloading artifacts etc. but still my end goal was failure .

Even after so much trying out and tweaking the configuration the end result still was build failure, until while writing this blog entry I found the problem and after correcting it got it to work the way I wanted with successful build.

The final piece of the puzzle to get artifactory work the way I wanted was to add the exclusion for maven-plugins in apache.snapshots entry in configuration.

Lack of this exclusion was causing the build to pickup latest snapshot release and somewhere down the line the rest of the maven plugins picked up were either unstable or non-existent versions. I suspect this also would have been the problems with other 2 experiments also but anyway the other bug in Proximity still would mess -up the artifacts.

Next goal is to get the Apache CXF building with the same setup which I am 95% sure will work.

Monday, March 17, 2008

Nothing work for me , Archiva, Artifactory or Proxima

I am starting to feel like most dumb person as per my Blog title. After struggling for so many days managed to get the artifactory running I stumbled upon the problem of no legacy layout for maven repositories like so had to give up on this.
Next tried Archiva again and somwhere down the line my build doesn't find the requred maven-resources-plugin. No matter how I set it up it just doesn't like the setup and give up.

After these 2 option gone I turned back to proximity. After repeat configurations and getting build to download dependencies now I am seeing funny results from proximity for Some dependency POM files which gets replaced with some kind of artifact summary written by Proximity and also the corresponding jar file gets corrupted.

I still haven't given up on the setup. hoping to get something going soon. Let's see if I succeed or completely give up.

Sunday, March 16, 2008

No Archiva, no Proximity now trying artifacotry

I don't know what went wrong in my setup of Archiva and proximity it doesn't like to download maven-plugin-plugin whatever I do. After lot of time spent on it I decided to give a try to Artifactory again. As expected, failed on startup but got fixed and currently trying to do a Build for Apache ServiceMix 3.2.2 using Artifactory.
One thing I found from these exercise is the domain name doesn't confirm to RFC2109 so httpClient reject cookies. Also, it was rejecting to connect to it for artifact download. I had to do a ping and replace the name with IP address in configuration. The next problem I hit was due to mistake in entering the IP Address. (I am half blind at 1:30 a.m. due to lack of sleep but can't stop playing with this exercise)

I am hoping that atleast Artifacotry would work for me. I found it slightly faster also in downloading artifacts thanArchiva and Proximity.(Unless it just sleeps on some blocked connection or with NoRouteToHostException due to bad repo address url in which case it sleeps for 60 sec. on Linux)
One more problem I saw in my Artifactory setup is if I kill it won't shutdown fast enough and clean fashion I almost always had to kill java process.

I am hoping to get it working today. One big achievement is that I managed to get it talking to all the repositories I configured in configuration file.

Saturday, March 15, 2008

short-lived happiness of archiva

My happiness of getting Apache maven-archiva to work was short lived one. I wanted to see it working again with clean local maven repository and it now hangs. I even tried deleting archiva data directory to populate it fresh but no use mvn clean simply blocks and archiva even after adding admin user and giving repository management rights to default guest account pretend to have not received any communication from maven (atleast from the log at debug level)

Friday, March 14, 2008

My experiments of setting up Apache Archiva and proximity

From last 3 - 4 days I was playing with Apache Archiva 1.0.1, Proximity and artifactory.

One thing I found in common is the trouble to get it setup and lack of basic documentation or sample configuration.
After playing so long and spending long nights breaking my head over it finally mange to get Apache Archiva configured for my maven based projects.Initially I got proximity setup successful for Apache CXF build but couldn't get it to work with ServiceMix builds so started looking at artifactory. It was sure easy enough to get started with artifactory but soon I hit a problem where it won't like my repository mirror names and started rejecting them with cookie problems saying host-domain cannot contain "." character.

Finally got Archiva working with our inhouse Maven mirrors and external repositories.

I had to play around a lot and do the following things to let the Archiva talk to external mirrors.
1. I had to add repository proxy connectors for each of them.(which was expected)
2. Change managed repository to internal (snapshots which is shown in default selection didn't work for me.
3. Had to add */** pattern for WhiteList in proxy connector.
4. Biggest of all and most annoying part (since it took long time to find out) was to mark the java-net repository as maven 1.x repository because it has propitiatory directory structure which doesn't follow maven2 style.

After all this (looks to small to write but took 3-4 days to experiment in spare time) finally got Archiva to work.

Now next exercise is to clean local maven repository and run the builds. Also, would be interesting is to point repositories to nfs share if possible to speed up caching.

I guess whatever I wrote is not too useful so I might do another post with sample proximity configuration with public repositories.
The whole reason for this blogging exercise was to record it somewhere for later reference as I struggled to find the example configurations.

First Experience of Blogging

This is my first blog.
I felt I should record all the sensible (and more non-sensible) stuff, so that others can look at it and even I can refer to it again as I tend to forget things that I did to get something working .

Intention is to cover the low-tech stuff, mainly my crap experiments at work and home.

I should say, I am not a good writer (and my spelling/grammar is not good) so bare with me if there are any mistakes. I feel expressing the contents matters to me more than the style, spelling and grammar.