Mazya Uchapati (My low-tech dumb experiments)

ActiveMQ prefetch and Spring DMLC advice from Torsten

2011-11-14T14:02:00.001Z

A piece of advice on ActiveMQ prefetch and Spring DMLC from my colleague Torsten.

I just remembered a lesson learned about prefetch and Spring DMLC:
If you configure DMLC to use CACHE_SESSION or below (i.e. CACHE_CONNECTION or CACHE_NONE) *and* when you don't use a CachingConnectionFactory (e.g SingleConnectionFactory), then you should set the prefetch for the Spring consumer to only 1 or 0.
That is because when not caching the consumer, a new JMS consumer is recreated for every msg that Spring will ask the broker for.
And if there are many msgs on the queue, the broker will prefetch e.g. 1000 msgs to the consumer although the consumer will only process one msg.
Spring DMLC then recycles the JMS consumer with 999 unprocessed msgs. This obviously causes to much and unnecessary load on the network.

So possible solutions in this case:
1) prefetch=0 or 1,
2) configure DMLC to use CACHE_CONSUMER, or
3) use Springs CachingConnectionFactory

Short Note on How to install Servicemix 4 as windows service.

2011-11-08T15:59:00.006Z

To install Servicemix 4 as windows service need to start SMX from console and run features:install wrapper and wrapper:install and follow the instructions that are displayed by second karaf command.

Here is the output from a sample run :

karaf@root> features:install wrapper
karaf@root> wrapper:install
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-wrapper.exe
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\etc\karaf-wrapper.conf

Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\wrapper.dll
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\karaf-wrapper.jar
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\karaf-wrapper-main
.jar

Setup complete. You may want to tweak the JVM properties in the wrapper configu
ration file:
C:\work\apache-servicemix-4.4.1-fuse-01-06\etc\karaf-wrapper.conf
before installing and starting the service.

To install the service, run:
C:> C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat install

Once installed, to start the service run:
C:> net start "karaf"

Once running, to stop the service run:
C:> net stop "karaf"

Once stopped, to remove the installed the service run:
C:> C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat remove

Hope this helps and serves as note to myself for future.

Test post from Android phone.

2011-06-30T00:54:00.001+01:00

Test post.

Enabling JMX monitoring in Servicemix for pax-web jetty instance.

2011-02-02T16:59:00.005Z

In Servicemix 4.x OSGi HTTP service is provided using pax-web. By default, it doesn't expose MBean for underlying Jetty instance running inside pax-web.

To enable monitoring this jetty instance you need to use fragment bundle that would connect underlying jetty to Servicemix MBeanServer using additional jetty configuration.

Here are steps to do this:

1. Create a OSGi Fragment-Bundle which will attach to pax-web-jetty OSGi service. Bundle Fragment project pom.xml contains following:


<project xmlns="http://maven.apache.org/POM/4.0.0" 
            xsi="http://www.w3.org/2001/XMLSchema-instance"
            schemalocation=
               "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

 <modelversion>4.0.0</modelversion>

 <groupid>org.ops4j.pax.web</groupid>
 <artifactid>config-jetty-jmx</artifactid>
 <version>1.0.0-SNAPSHOT</version>
 <packaging>bundle</packaging>

 <name>OPS4J Pax Web - Configuring Jetty JMX connection</name>

 <properties>
   <bundle.symbolicname>org.ops4j.pax.web.config-jetty-jmx</bundle.symbolicname>
 </properties>

 
 <build>
   <plugins>
     <plugin>
       <groupid>org.apache.felix</groupid>
       <artifactid>maven-bundle-plugin</artifactid>
       <version>2.2.0</version>
       <extensions>true</extensions>
       <executions>
           <execution>
             <id>bundle-manifest</id>
             <phase>process-classes</phase>
             <goals>
               <goal>manifest</goal>
             </goals>
           </execution>
         </executions>
       <configuration>
           <instructions>
             <bundle-symbolicname>${bundle.symbolicName}</bundle-symbolicname>                     
             <bundle-version>${pom.version}</bundle-version>             
             <fragment-host>org.ops4j.pax.web.pax-web-jetty</fragment-host>
           </instructions>
       </configuration>
     </plugin>
   </plugins>
 </build>

</project>

2. This bundle (Fragment) would have additional Jetty configuration xml file jetty.xml which would (reside in src/main/resources folder) look like this:


<configure class="org.mortbay.jetty.Server">   
    <call id="jettyMBeanServer" class="java.lang.management.ManagementFactory" name="getPlatformMBeanServer">

   <!-- =========================================================== -->
   <!-- Initialize the Jetty MBean container                        -->
   <!-- =========================================================== -->
   <get id="Container" name="container">
     <call name="addEventListener">
       <arg>
         <new class="org.mortbay.management.MBeanContainer">
           <arg><ref id="jettyMBeanServer"></ref>
           <call name="start">
           </call>       
          </arg>
     </new>
   </arg>

</call>

3. Once the fragment bundle is built install it in Servicemix instance which is not currently running pax-web OSGi services. (I used Servicemix instance with minimal bundles only config feature was installed) I installed it using osgi:install file:///.... or something like osgi:install mvn:org.ops4j.pax.web/config-jetty-jmx where the groupId for my Fragment bundle project was org.ops4j.pax.web and artifactId config-jetty-jmx.

4. Once this is installed you can install pax-web. I deployed war feature which installs and starts pax-web feature.

5. At this stage your Fragment bundle would get attached to pax-web-jetty Host Bundle and enable JMX monitoring. To confirm this if you run osgi:list you should see something like (Note Fragment and Host entries):

[33] [Resolved ] [ ] [ 60] OPS4J Pax Web - Configuring Jetty JMX connection (1.0.0.SNAPSHOT)
Hosts: 39
[ 34] [Active ] [ ] [ 60] geronimo-servlet_2.5_spec (1.1.2)
[ 35] [Active ] [ ] [ 60] Apache ServiceMix :: Bundles :: jetty (6.1.26.1-fuse)
[ 36] [Active ] [ ] [ 60] OPS4J Pax Web - API (0.7.3)
[ 37] [Active ] [ ] [ 60] OPS4J Pax Web - Service SPI (0.7.3)
[ 38] [Active ] [ ] [ 60] OPS4J Pax Web - Runtime (0.7.3)
[ 39] [Active ] [ ] [ 60] OPS4J Pax Web - Jetty (0.7.3)
Fragments: 33
[ 40] [Active ] [ ] [ 60] OPS4J Pax Web - Jsp Support (0.7.3)
[ 41] [Active ] [ ] [ 60] OPS4J Pax Web - Extender - WAR (0.7.3)
[ 42] [Active ] [ ] [ 60] OPS4J Pax Web - Extender - Whiteboard (0.7.3)
[ 43] [Active ] [ ] [ 60] OPS4J Pax Url - war:, war-i: (1.1.3)
[ 44] [Active ] [Created ] [ 60] Apache Karaf :: WAR Deployer (2.0.0.fuse-02-00)

6. At this stage if you start JConsole and look connect to running Servicemix instance you should be able to see following MBeans appearing in MBeans tab:
a. org.mortbay.jetty.servlet
b. org.mortbay.thread
c. org.ops4j.pax.web.service.jetty.internal

I did see some InstanceAlreadyExists exceptions in Servicemix instance but wasn't able to resolve them quickly so just left there since in my opinion they are harmless.

I hope this helps anyone who wants to configure internal Jetty instance run by pax-web in Servicemix/Karaf.

How to fix broken grub when you have /boot on different partition.

2010-11-18T16:08:00.002Z

I recently upgraded my Ubuntu Desktop (64 bit) to 10.10. This upgrade broke my bootup and started throwing me to initramfs prompt.
After googling a bit and trying unsuccessfully instructions from here I found a simple solution.

I have a slightly differnt setup than what Grub assumes (boot and root on same partition).
I have my boot partition on (hd0,msdos3) and root partition on (hd0,msdos5). Grub assumed that both are on (hd0,msdos3) so added the root entry pointing to the same and tried to use kernel and initrd images from root.

To fix the issue all I had to do was use live USB key to boot into linux and mount partition (hd0,msdos3) and edit grub/grub.cfg to set root=(hd0,msdos5) and load kernel from (hd0,msdos3)/ and same for initrd.

This simple fix worked after lot of unsuccessful attempts to install grub and let it configure the boot configuration.

I am pretty sure this is going to break once the new kernel version is updated in future by update-manager. However, now atleast I know what to expect and how to fix with this post.

I would be happy to try suggestion on how to fix it in better way than what I did.

Finally sorted my duplicate photo collection.

2010-05-21T11:26:00.006+01:00

I figured out recently that I have accumulated 21GB of photos and movie clips from my small Pentax S4 camera and was wondering what should I do to remove the duplicates. Having searched google for linux photo management softwares I installed digiKam (from KDE) and it took almost half an hour to catalog them and used fuzzy search (I think I used the right term) to find duplicates and shocked me with many photos with as many as 11 copies in different folders from backups.

I took the painful way of manually cleaning them from duplicates list in digiKam and spent almost 1 whole day in all to do this. In the end I was expecting my original collection to be clean but it wasn't but digiKam showed it clean. So thought of deleting and recreating new album (biggest mistake) and was shocked again by it found all the duplicates and then I realized when you add an album it does the whole copy of the collection and edits the copy. (this is really annoying for me but it may there for good reason of not loosing stuff accidentally) Having said all this I do think it's a very good photo management application.

I was no way ready for the same exercise again so back to Google again and found another solution fslint which shows duplicate files but is still manual work to remove duplicates.

Further search gave me wonderful command on one of the forums using fdupes which did a very good job for me. I was bit skeptical in first place with fear to loose some files so ran it without -d option like
fdupes -r photos/clean_album and reviewed output and was happy with the listing so when ahead and used it as suggested by forum
yes 1 | fdupes -rd photos/clean_album and I had my 95% or duplicate removal work done. (I wish I should have used it in first place) Rest of the work was to clean up using Digikam and then flatten them into one directory where I have some name conflicts which I used as my blog name low-tech approach and added prefixes to avoid clashes.

End result 21GB is now reduced to 8.5GB.

I think I should be more careful going forward about duplicates and backups.

EDIT: Another note is I did miss the avi clips when I did manual cleanup using digiKam but fslint and fdupes sorted those out as well.

How to install Oracle xe on Ubuntu 64 bit?

2010-01-06T10:54:00.002Z

I wanted to install Oracle Xe on Ubuntu 64bit and was faced with the problem that the debian package that Oracle has with free edition is 32 bit so googled a bit for more information and found a link that helped me to configure it.

Here is a link to keep note of : http://littlebrain.org/2008/05/12/how-to-install-oracle-xe-in-ubuntu-64-bit/

gdm vs xdm surprise on my Ubuntu 8.04.1 x86_64 machine.

2009-04-20T11:58:00.006+01:00

While looking at some cleanup in Synaptic package manger for unused packages I thought of seeing what difference it would make to GNOME if I use xdm instead of gdm. Well
1. I saw not so nice startup text display (I am fine with it though)
2. I saw fine black and white mesh screen which is awfully painful for graphical login with thick text box for entering login/password. (I even managed to stand this one)
3. Login was much swift than previous into GNOME. (so it was worth the switch for now)/
4. I could see all the menus icons taskbar etc. so it's good sign.
5. Only thing I missed was shutdown option in GNOME System->Quit (or from the quit button).

The biggest surprise was without anything running I would normally see most of my 4 GB RAM was always in use with GDM which is now drop down to around 700-900Mb even now with Thunderbird, pidgin, firefox on it is at 1.1GB I think to me this is big improvement. (there may some others limitations but for now I thik I will see how it works for me). I past I always saw my system eating away into the 6 GB swap without running much of the applications.

I think I will also try another not so heavy window managers like Enlightenment etc.

The part of inspiration to think about it is DIY I am doing with my homemade Digital Picture Frame out of P-II for which I am trying various linux options after staring the project 2 years back and shelved it with initial success with Windows.

I wish to get more than a digital picture frame out of the DIY possibly low-end media player controllable via remote using lirc. (sounds too ambitious but will give it a go)

Quick and short post on jmxrmi exception in FUSE HQ.

2009-04-10T13:44:00.009+01:00

If you even see following exception when using FUSE HQ or Hyperic HQ it is most likely an issue with what JDK/JRE you (or rather HQ) is using. The problem occurred while collecting statistics from FUSE ESB (Servicemix) and FUSE Message Broker (ActiveMQ).

Here is the exception:


org.hyperic.hq.product.PluginException: javax.naming.NameNotFoundException: jmxrmi
 at org.hyperic.hq.product.jmx.MxServerDetector.discoverServices(MxServerDetector.java:404)
 at org.hyperic.hq.product.ServerDetector.discoverResources(ServerDetector.java:203)
 at org.hyperic.hq.autoinventory.agent.server.RuntimeAutodiscoverer.doRuntimeScan_internal(RuntimeAutodiscoverer.java:272)
 at org.hyperic.hq.autoinventory.agent.server.RuntimeAutodiscoverer.doRuntimeScan(RuntimeAutodiscoverer.java:205)
 at org.hyperic.hq.autoinventory.ScanManager.mainRunLoop(ScanManager.java:165)
 at org.hyperic.hq.autoinventory.ScanManager.access$000(ScanManager.java:41)
 at org.hyperic.hq.autoinventory.ScanManager$1.run(ScanManager.java:107)
Caused by: java.io.IOException: javax.naming.NameNotFoundException: jmxrmi
 at mx4j.remote.resolver.rmi.Resolver.lookupStubInJNDI(Resolver.java:100)
 at mx4j.remote.resolver.rmi.Resolver.lookupRMIServerStub(Resolver.java:72)
 at mx4j.remote.resolver.rmi.Resolver.lookupClient(Resolver.java:52)
 at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:119)
 at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:38)
 at org.hyperic.hq.product.jmx.MxUtil.getMBeanConnector(MxUtil.java:445)
 at org.hyperic.hq.product.jmx.MxServerDetector.discoverServices(MxServerDetector.java:401)

The cause in my case:
FUSE HQ agent was using JRE 1.4 bundled with it which was causing the problem.
Why : There are some incompatibilities in JMX stuff when it comes to using the same code with JDK 1.4 and JDK 1.5 and apparently moving to JDK/JRE 1.5 got rid of this problem.
I don't know much about JMX so don't know what all incompatibilities are between tow of them. Anyone who reads (in first place) and knows the differences and could comment on this post it would help everyone.

Accessing archetype while using nexus repository manager as proxy

2009-04-07T12:02:00.003+01:00

I was trying to use Eclipse with m2eclipse for creating some FUSE ESB project from the archetypes and was getting strange error :

"Unable to create project from archetype [org.apache.servicemix.tooling:servicemix-bean-service-unit:3.3.1.16-fuse -> null]
> The desired archetype does not exist (org.apache.servicemix.tooling:servicemix-bean-service-unit:3.3.1.16-fuse)"

After drilling down more and looking into it I found the problem was :
1. I did configured the nexus index catalog http://repo.open.iona.com/maven2/archetype-catalog.xml correctly.
2. However, I later realized that I am using Nexus repository manager which doesn't automatically download the catalog itself and so it is not getting all the requited information to download the correct artifacts.

The way I fixed this for now is manually downloading the archetype-catalog.xml via my Nexus repository instance like this:

wget http://my-nexus-proxy:8081/nexus/content/repositories/open.iona.m2/archetype-catalog.xml

once the catalog was available locally at my nexus repository I then used the Eclipse m2eclipse plugin in usual manner and it went and download required archetype plugins from the remote repository into my nexus instance.

I really would like to see some option in Nexus which would allow synchronization of such artifacts which are not downloaded normally by maven dependency. Some way of scheduling such stuff (I know we can schedule the task but some smart scheduling for such category of artifacts).

For now manual downloading and sync is the way to get around the problem.

Nexus Revisited after long time.

2009-03-25T16:45:00.005Z

After long time and not to say with after giving up my thought of setting up nexus/artifactory/archiva I came back to same stuff recently around month back.

I restarted my exercise with nexus 1.2.1 and to my surprise with very little efforts I manage to get it up and running for Apache projects I care about which are CXF, Servicemix.

I also upgraded our team nexus repository manager from older version to 1.3.1 and happily using it now. It is lot more improved than I thought it would have been. In past I was very much annoyed when my pom files got corrupt for the artifacts that were present in more than one repositories I had configured but it appears to have been fixed now and also the web interface for configuration is very helpful.

Having said all this I did spent some time today in figuring out why the artifact present in remote repository won't download even though I had it configured it correctly. After a lot of head scratching and looking at the configuration multiple times discovered that there is a parameter on each repository you configure called Not Found Cache TTL which by default was set to 1440 Minute once I changed it to smaller amount like 2 minutes my build went ahead happily.

I am overall very happy with the new Nexus Repository Manager (surely much better experience than I had with proximity and old nexus version).

SOAP RPC/Encoded and style related articles

2009-03-25T08:19:00.004Z

I recently looking for more information on RPC/encoded style came across two old but very good and simple articles so thought of bookmarking here via my Blog.

web-services-interoperability-between-j2ee-and-net-part-1
IBM-library-ws-whichwsdl

I have referenced the second one many times. It is very simple and handy to understand different between different SOAP message types.

Spring jmsTemplate threading issue

2008-08-21T11:26:00.001+01:00

I recently encounter a problem while working on some Servicemix bug and it turned out that Servicemix jms provider is using spring-jms jmsTemplate and when there are multiple threads trying to do send receive (receiveSelected() to be precise) some of the messages are dropped.

While doing further investigation I found that multiple threads are using the same instance of spring-jms jmsTemplate that is created by the jms provider and this was causing some threading issue so thing to remember is to make the calls to jmsTemplate.receiveSelected() (atleast this one) synchronized.

UPDATE: I was too quick to diagnose the problem as jmsTemplate issue. With more drill down I was able to reproduce the problem with pure jmsTemplate and ActiveMQ combination and looks like a ActiveMQ issue than spring-jms. My friend Gary is getting it fixed in ActiveMQ currently.

useful blog entry about how to detect/analyze with java memory leaks

2008-08-15T15:56:00.000+01:00

java-memory-leaks

SAP Memory Analyzer Eclipse plugin (MAT)

Changes to Apache CXF JMS transport to allow jmsDestination name

2008-08-11T17:52:00.006+01:00

Recently Apache CXF JMS transport is modified in order to allow use of jmsDestinationName and jmsReplyDestinationName. Till now CXF JMS only allowed using jndi destination names to lookup queue/destination and was not able to use jms destination names to create queues/topics.

Here is the diff of the schema change to specify the jmsDestinationName and jmsReplyDestinationName.

Example jms:address looks like this:

<jms:address jndiconnectionfactoryname="ConnectionFactory"     
              jmsdestinationname="dynamicQueues/routertest.SOAPService6Q.text"
              jmsreplydestinationname="dynamicQueues/SoapService6.reply.queue">
              <jms:jmsnamingproperty name="java.naming.factory.initial"
                            value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
              <jms:jmsnamingproperty name="java.naming.provider.url"
                                        value="tcp://localhost:61500"/>
</jms:address>

This will allow CXF consumer and services to use the real queue names (where JMS provider will allow the creation of the queues) as opposed to use jndi lookup.

handy tip on how to increase jetty HttpClient threadpool size in ServiceMix 3.2.2

2008-08-11T16:50:00.001+01:00

I was recently working on some jetty related issue in Servicemix 3.2 and got Servicemix people to update servicemix-http component to have configurable jetty httpClient thread pool size. Before I forget it I wanted to make a note of it so adding this entry.

In Servicemix 3.2.2 and onwards jetty HttpClient threadpool can be configured using following entry in component.properties file.
servicemix-http.jettyClientThreadPoolSize=33

Also, you can configure Servicemix to use Jetty httpClient per httpProvider (by default a single client is shared among all httpProviderEndpoints) by adding

servicemix-http.jettyHttpClientPerProvider=true

Also from ServiceMix 3.2.2 onwards Jetty HttpClient uses nio channel selector implementation of jetty. (previously it was using socket connector which was blocking I/O and was causing thread locking in thread pool in some stress conditions)

टेस्टिंग मराठी ब्लोंग एंट्री.

2008-08-01T14:08:00.000+01:00

अरे वा मराठी एंट्री पण चालते आहे बर्यापैकी पण माझे मराठी कच्चे आहे।

Running CXF jms_queue sample in Tomcat

2008-06-06T15:26:00.000+01:00

Recently, I wanted run Apache CXF jms_queue sample in tomcat and came across some problems setting up stuff so want to note the things that I had to do for it.

1. need to create the spring.xml and web.xml descriptor for deploying the jms_queue server inside Tomcat.
2. Need to add CXF and ActiveMQ jars to the container classpath. I haven't tried yet to bundle these inside the application war file that is next on my list.
3. Build the sample, create war file and deploy it to tomcat by copying it to webapp directory of the container.
4. start jms broker (I user ant task from the sample that starts embedded broker).
5. Start tomcat container. This will deploy the copied war and start the CXF jms_queue service.
6. run the client from the commandline against the service deployed into Tomcat.

Looks simple isn't it? But I had to spend 1 day figuring out how to make this work.

glitch 1: Service creation wasn't finding WSDL for my service. Reason: WSDL was bundled under WEB-INF/wsdl directory which is not on classpath so I have changed the build.xml to put it under WEB-INF/classes directory.

glitch 2: Wanted to enable logging by adding logging.properties file but wasn't able to find it on classpath. Reason: same as glitch 1 and solved when added it to WEB-INF/classes folder.

I am attaching tar file of my sample directory for future reference here.

EDIT: I couldn't find the way to upload the tar file so I am editing the post to remind me to post the changes in my next post. Not so good but can't help.

EDIT:
Here is how web.xml looks like:

And my spring.xml looks like this:

Finally here is the link to zip file that has the full example.

Nexus 1.0.0-beta-2 testing

2008-05-14T22:53:00.000+01:00

I managed to run Apache CXF and Apache Servicemix 3.2 builds with some glitches.
1. CXF build was fine without any problem (didn't run tests though)
2. ServiceMix 3.2.2 build failed to strangely due to 2 issue in first run.
a. Couldn't find log4j dependency (which should have been resolved indirectly from some other component dependency. I have to specifically add the dependency to POM file, maybe I should submit patch to Apache SMX for this.
b. Failed to run "crawl" goal for servicemix-archetype-catalog module. The puzzle is it doesn't do anything if I do a full build and reports failure but passes if I run mvn:install inside the component directory.

Overall, Out of 3 requirements of Builds 2 are successful., 3rd (IONA Fuse ESB's Logisticx Demo) I haven't yet tried. That is the next thing on my menu now.

All in all, I am pretty happy with the experience this time. (unlike my previous attempts to get proximity going without any UI based intuitive configuration option or lack of access to docs.)

I am surely looking forward to tune the configuration and use it for my regular work.

Oh.. I forgot to add the link to the download site and documentation that I accessed today.
Here are the links.

Nexus Home
Nexus Download

One thing I mentioned in last post was about beta-3. By the looks of the download are it seems the download page does the announcement of beta-3 but the actual download storage area doesn't have the necessary files in place yet.

My experiments continues......................

I think, I should do some other experiments also and keep the blog going. I still have to show my stab at Digital Picture frame out of old Toshiba laptop. Even though I got it to work properly in January on my Daughter's B'day Haven't got a time to touch it and refine and fix the backside of it properly. Hope to find some time to finish that and take some photos.

another experiment with Nexus now in progress

2008-05-14T19:32:00.000+01:00

Today I stumbled upon Nexus repository manager while browsing and found a 1.0.0-beta-2 available for download. So with some excitement I went ahead and downloaded it and currently trying out.

Just now when I went back to check the link I realized that there is a new version nexus-1.0.0-beta-3. I will try patching my installation with it and will post the results.

One plus point, I can't resist mentioning here by initial looks is better UI based repository management which makes life much easier. Got the setup starting to work in no time and speed is also good.

I did stumbled upon the corruption of the pom file so doing clean build again to see if I can consistently get the same problem.

Something about the proximity problems I had.

2008-03-19T17:06:00.000Z

It's been 4-5 days on and off I am trying the Maven Repository products to get working with Apache ServiceMix 3.2 build and test.

Goal: To get Apache ServiceMix 3.2 build and test working.

I think I should summarize the experience as I am pretty much giving up on them (or probably not but not sure yet). One thing is sure I liked the exercise even though frustrating some times.

1. Proximity :
* With no longer in development and Subversion repository, wiki, forums gone it is hard to get the bundle. I manage to get it from Bruce Snyder's Proximity blog entry.
* Hard to configure as no documentation around. (Thanks to Bruce Snyder for helping out)

Problems I faced are:
1. After adding the required bean entries and FSStores and Remote Peer locations It didn't work.
2. So tried sequencing the entries based on preference and updated for each beam entry accordingly.
3. After fixing this got the Repository logic properties wrong on some entires, fixed that and still proximity wouldn't try all the repositories listed.
4. After lot of time (without any docs) found that I was putting all the repository entries in separate group and as per my settings.xml maven was only requesting on public group so modified the configuration and updated for all required entries to public.
5. At this stage got maven to talk to the repositories and here I hit the big problem (may be a bug in proximity) for some of the dependencies which are available in more than one repositories it was corrupting the Jar files and POM files. This caused builds to fail on invalid POM and jar file. The POM file of the artifact in the local repository was replaced by some kind of Proximity generated artifact summary.

At this stage I gave up and moved on.

Nice to have: Some documentation around configuration, UI for adding or editing configuration. Hand editing the XML configuration is error-prone task.

Apache Archiva 1.0.1:
Similar to Proximity had some configuration issues. Didn't know need to add proxy connector for each repository entry. Once the repository entries, connectors and all entries in one group I was hit by a issue of cookies didn't like the url name of some of my entries saying they do not confirm to RFC2109. It didn't talk to that repository at all. Tried putting IP address but no luck archiva would resolve it and skip the repository with same error.
Result: Stopped experimenting.

One strong point is the ability to configure using UI in browser. It makes it much easier to view and edit configuration compared to other 2.

Nice to have: Little more speed in download artifacts. Found it slow even for cached artifacts. And also the fix/workaround (or visible documentation somewhere if it exists) for the RFC 2109 error.

Artifacotry 1.2.5 :
After playing with this I even if the build was not successful I started liking it for the speed of the downloads especially compared to Archiva. With some more efforts and configuration I got Artifactory to talk to all the repositories I wanted. It also gave the same RFC2109 error so changed to IP address and it still showed the error but was able to contact the repository alright.
Next issue was grouping and search order. From documentation page I manage to find out the search order and learned about the VirtualRepo. So Added new virtualRepo and added reference to my repository entries in the order I wanted.
So far so good stared downloading artifacts etc. but still my end goal was failure .

Even after so much trying out and tweaking the configuration the end result still was build failure, until while writing this blog entry I found the problem and after correcting it got it to work the way I wanted with successful build.

The final piece of the puzzle to get artifactory work the way I wanted was to add the exclusion for maven-plugins in apache.snapshots entry in configuration.

Lack of this exclusion was causing the build to pickup latest snapshot release and somewhere down the line the rest of the maven plugins picked up were either unstable or non-existent versions. I suspect this also would have been the problems with other 2 experiments also but anyway the other bug in Proximity still would mess -up the artifacts.

Next goal is to get the Apache CXF building with the same setup which I am 95% sure will work.

Nothing work for me , Archiva, Artifactory or Proxima

2008-03-17T05:31:00.000Z

I am starting to feel like most dumb person as per my Blog title. After struggling for so many days managed to get the artifactory running I stumbled upon the problem of no legacy layout for maven repositories like java.net so had to give up on this.
Next tried Archiva again and somwhere down the line my build doesn't find the requred maven-resources-plugin. No matter how I set it up it just doesn't like the setup and give up.

After these 2 option gone I turned back to proximity. After repeat configurations and getting build to download dependencies now I am seeing funny results from proximity for Some dependency POM files which gets replaced with some kind of artifact summary written by Proximity and also the corresponding jar file gets corrupted.

I still haven't given up on the setup. hoping to get something going soon. Let's see if I succeed or completely give up.

No Archiva, no Proximity now trying artifacotry

2008-03-16T01:11:00.000Z

I don't know what went wrong in my setup of Archiva and proximity it doesn't like to download maven-plugin-plugin whatever I do. After lot of time spent on it I decided to give a try to Artifactory again. As expected, failed on startup but got fixed and currently trying to do a Build for Apache ServiceMix 3.2.2 using Artifactory.
One thing I found from these exercise is the domain name repo.open.iona.com doesn't confirm to RFC2109 so httpClient reject cookies. Also, it was rejecting to connect to it for artifact download. I had to do a ping and replace the name with IP address in configuration. The next problem I hit was java.net.noRouteToHostException due to mistake in entering the IP Address. (I am half blind at 1:30 a.m. due to lack of sleep but can't stop playing with this exercise)

I am hoping that atleast Artifacotry would work for me. I found it slightly faster also in downloading artifacts thanArchiva and Proximity.(Unless it just sleeps on some blocked connection or with NoRouteToHostException due to bad repo address url in which case it sleeps for 60 sec. on Linux)
One more problem I saw in my Artifactory setup is if I kill it won't shutdown fast enough and clean fashion I almost always had to kill java process.

I am hoping to get it working today. One big achievement is that I managed to get it talking to all the repositories I configured in configuration file.

short-lived happiness of archiva

2008-03-15T16:10:00.000Z

My happiness of getting Apache maven-archiva to work was short lived one. I wanted to see it working again with clean local maven repository and it now hangs. I even tried deleting archiva data directory to populate it fresh but no use mvn clean simply blocks and archiva even after adding admin user and giving repository management rights to default guest account pretend to have not received any communication from maven (atleast from the log at debug level)

My experiments of setting up Apache Archiva and proximity

2008-03-14T18:29:00.000Z

From last 3 - 4 days I was playing with Apache Archiva 1.0.1, Proximity and artifactory.

One thing I found in common is the trouble to get it setup and lack of basic documentation or sample configuration.
After playing so long and spending long nights breaking my head over it finally mange to get Apache Archiva configured for my maven based projects.Initially I got proximity setup successful for Apache CXF build but couldn't get it to work with ServiceMix builds so started looking at artifactory. It was sure easy enough to get started with artifactory but soon I hit a problem where it won't like my repository mirror names and started rejecting them with cookie problems saying host-domain cannot contain "." character.

Finally got Archiva working with our inhouse Maven mirrors and external repositories.

I had to play around a lot and do the following things to let the Archiva talk to external mirrors.
1. I had to add repository proxy connectors for each of them.(which was expected)
2. Change managed repository to internal (snapshots which is shown in default selection didn't work for me.
3. Had to add */** pattern for WhiteList in proxy connector.
4. Biggest of all and most annoying part (since it took long time to find out) was to mark the java-net repository as maven 1.x repository because it has propitiatory directory structure which doesn't follow maven2 style.

After all this (looks to small to write but took 3-4 days to experiment in spare time) finally got Archiva to work.

Now next exercise is to clean local maven repository and run the builds. Also, would be interesting is to point repositories to nfs share if possible to speed up caching.

I guess whatever I wrote is not too useful so I might do another post with sample proximity configuration with public repositories.
The whole reason for this blogging exercise was to record it somewhere for later reference as I struggled to find the example configurations.

First Experience of Blogging

2008-03-14T18:20:00.000Z

This is my first blog.
I felt I should record all the sensible (and more non-sensible) stuff, so that others can look at it and even I can refer to it again as I tend to forget things that I did to get something working .

Intention is to cover the low-tech stuff, mainly my crap experiments at work and home.

I should say, I am not a good writer (and my spelling/grammar is not good) so bare with me if there are any mistakes. I feel expressing the contents matters to me more than the style, spelling and grammar.