ActiveMQ prefetch and Spring DMLC advice from Torsten

A piece of advice on ActiveMQ prefetch and Spring DMLC from my colleague Torsten.


I just remembered a lesson learned about prefetch and Spring DMLC:
If you configure DMLC to use CACHE_SESSION or below (i.e. CACHE_CONNECTION or CACHE_NONE) *and* when you don't use a CachingConnectionFactory (e.g SingleConnectionFactory), then you should set the prefetch for the Spring consumer to only 1 or 0.
That is because when not caching the consumer, a new JMS consumer is recreated for every msg that Spring will ask the broker for.
And if there are many msgs on the queue, the broker will prefetch e.g. 1000 msgs to the consumer although the consumer will only process one msg.
Spring DMLC then recycles the JMS consumer with 999 unprocessed msgs. This obviously causes to much and unnecessary load on the network.

So possible solutions in this case:
1) prefetch=0 or 1,
2) configure DMLC to use CACHE_CONSUMER, or
3) use Springs CachingConnectionFactory

Short Note on How to install Servicemix 4 as windows service.

To install Servicemix 4 as windows service need to start SMX from console and run features:install wrapper and wrapper:install and follow the instructions that are displayed by second karaf command.

Here is the output from a sample run :

karaf@root> features:install wrapper
karaf@root> wrapper:install
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-wrapper.exe
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\etc\karaf-wrapper.conf

Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\wrapper.dll
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\karaf-wrapper.jar
Creating file: C:\work\apache-servicemix-4.4.1-fuse-01-06\lib\karaf-wrapper-main
.jar

Setup complete. You may want to tweak the JVM properties in the wrapper configu
ration file:
C:\work\apache-servicemix-4.4.1-fuse-01-06\etc\karaf-wrapper.conf
before installing and starting the service.


To install the service, run:
C:> C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat install

Once installed, to start the service run:
C:> net start "karaf"

Once running, to stop the service run:
C:> net stop "karaf"

Once stopped, to remove the installed the service run:
C:> C:\work\apache-servicemix-4.4.1-fuse-01-06\bin\karaf-service.bat remove

Hope this helps and serves as note to myself for future.

Test post from Android phone.

Test post.

Enabling JMX monitoring in Servicemix for pax-web jetty instance.

In Servicemix 4.x OSGi HTTP service is provided using pax-web. By default, it doesn't expose MBean for underlying Jetty instance running inside pax-web.

To enable monitoring this jetty instance you need to use fragment bundle that would connect underlying jetty to Servicemix MBeanServer using additional jetty configuration.

Here are steps to do this:

1. Create a OSGi Fragment-Bundle which will attach to pax-web-jetty OSGi service. Bundle Fragment project pom.xml contains following:

<project xmlns="http://maven.apache.org/POM/4.0.0"
             xsi="http://www.w3.org/2001/XMLSchema-instance"
         schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
             <modelversion>4.0.0</modelversion>
             <groupid>org.ops4j.pax.web</groupid>
             <artifactid>config-jetty-jmx</artifactid>
             <version>1.0.0-SNAPSHOT</version>
             <packaging>bundle</packaging>
             <name>OPS4J Pax Web - Configuring Jetty JMX connection</name>
             <properties>
                  <bundle.symbolicname>org.ops4j.pax.web.config-jetty-jmx</bundle.symbolicname>
             </properties>
             <build>
                <plugins>  
                   <plugin>    
                         <groupid>org.apache.felix</groupid>   
                         <artifactid>maven-bundle-plugin</artifactid>    
                         <version>2.2.0</version>    
                         <extensions>true</extensions>   
                         <executions>        
                             <execution>         
                                <id>bundle-manifest</id>         
                                <phase>process-classes</phase>          
                                <goals>            
                                   <goal>manifest</goal>          
                                </goals>        
                             </execution>      
                         </executions>    
                     <configuration>        
                         <instructions>          
                             <bundle-symbolicname>${bundle.symbolicName}</bundle-symbolicname>
                             <bundle-version>${pom.version}</bundle-version>                      
                             <fragment-host>org.ops4j.pax.web.pax-web-jetty</fragment-host>        
                         </instructions>   
                     </configuration>  
                 </plugin>
             </plugins>
         </build>
</project>

2. This bundle (Fragment) would have additional Jetty configuration xml file jetty.xml which would (reside in src/main/resources folder) look like this:

<configure class="org.mortbay.jetty.Server">
<call id="jettyMBeanServer" class="java.lang.management.ManagementFactory" name="getPlatformMBeanServer">

<!-- =========================================================== -->
<!-- Initialize the Jetty MBean container                        -->
<!-- =========================================================== -->
<get id="Container" name="container">
<call name="addEventListener">
<arg>
  <new class="org.mortbay.management.MBeanContainer">
    <arg><ref id="jettyMBeanServer"></ref>
    <call name="start">
    </call>
   </arg>
</new>
</arg>

</call>

3. Once the fragment bundle is built install it in Servicemix instance which is not currently running pax-web OSGi services. (I used Servicemix instance with minimal bundles only config feature was installed) I installed it using osgi:install file:///.... or something like osgi:install mvn:org.ops4j.pax.web/config-jetty-jmx where the groupId for my Fragment bundle project was org.ops4j.pax.web and artifactId config-jetty-jmx.

4. Once this is installed you can install pax-web. I deployed war feature which installs and starts pax-web feature.

5. At this stage your Fragment bundle would get attached to pax-web-jetty Host Bundle and enable JMX monitoring. To confirm this if you run osgi:list you should see something like (Note Fragment and Host entries):

[33] [Resolved ] [ ] [ 60] OPS4J Pax Web - Configuring Jetty JMX connection (1.0.0.SNAPSHOT)
Hosts: 39
[ 34] [Active ] [ ] [ 60] geronimo-servlet_2.5_spec (1.1.2)
[ 35] [Active ] [ ] [ 60] Apache ServiceMix :: Bundles :: jetty (6.1.26.1-fuse)
[ 36] [Active ] [ ] [ 60] OPS4J Pax Web - API (0.7.3)
[ 37] [Active ] [ ] [ 60] OPS4J Pax Web - Service SPI (0.7.3)
[ 38] [Active ] [ ] [ 60] OPS4J Pax Web - Runtime (0.7.3)
[ 39] [Active ] [ ] [ 60] OPS4J Pax Web - Jetty (0.7.3)
Fragments: 33
[ 40] [Active ] [ ] [ 60] OPS4J Pax Web - Jsp Support (0.7.3)
[ 41] [Active ] [ ] [ 60] OPS4J Pax Web - Extender - WAR (0.7.3)
[ 42] [Active ] [ ] [ 60] OPS4J Pax Web - Extender - Whiteboard (0.7.3)
[ 43] [Active ] [ ] [ 60] OPS4J Pax Url - war:, war-i: (1.1.3)
[ 44] [Active ] [Created ] [ 60] Apache Karaf :: WAR Deployer (2.0.0.fuse-02-00)

6. At this stage if you start JConsole and look connect to running Servicemix instance you should be able to see following MBeans appearing in MBeans tab:
a. org.mortbay.jetty.servlet
b. org.mortbay.thread
c. org.ops4j.pax.web.service.jetty.internal

I did see some InstanceAlreadyExists exceptions in Servicemix instance but wasn't able to resolve them quickly so just left there since in my opinion they are harmless.

I hope this helps anyone who wants to configure internal Jetty instance run by pax-web in Servicemix/Karaf.

How to fix broken grub when you have /boot on different partition.

I recently upgraded my Ubuntu Desktop (64 bit) to 10.10. This upgrade broke my bootup and started throwing me to initramfs prompt.
After googling a bit and trying unsuccessfully instructions from here I found a simple solution.

I have a slightly differnt setup than what Grub assumes (boot and root on same partition).
I have my boot partition on (hd0,msdos3) and root partition on (hd0,msdos5). Grub assumed that both are on (hd0,msdos3) so added the root entry pointing to the same and tried to use kernel and initrd images from root.

To fix the issue all I had to do was use live USB key to boot into linux and mount partition (hd0,msdos3) and edit grub/grub.cfg to set root=(hd0,msdos5) and load kernel from (hd0,msdos3)/ and same for initrd.

This simple fix worked after lot of unsuccessful attempts to install grub and let it configure the boot configuration.

I am pretty sure this is going to break once the new kernel version is updated in future by update-manager. However, now atleast I know what to expect and how to fix with this post.

I would be happy to try suggestion on how to fix it in better way than what I did.

Finally sorted my duplicate photo collection.

I figured out recently that I have accumulated 21GB of photos and movie clips from my small Pentax S4 camera and was wondering what should I do to remove the duplicates. Having searched google for linux photo management softwares I installed digiKam (from KDE) and it took almost half an hour to catalog them and used fuzzy search (I think I used the right term) to find duplicates and shocked me with many photos with as many as 11 copies in different folders from backups.

I took the painful way of manually cleaning them from duplicates list in digiKam and spent almost 1 whole day in all to do this. In the end I was expecting my original collection to be clean but it wasn't but digiKam showed it clean. So thought of deleting and recreating new album (biggest mistake) and was shocked again by it found all the duplicates and then I realized when you add an album it does the whole copy of the collection and edits the copy. (this is really annoying for me but it may there for good reason of not loosing stuff accidentally) Having said all this I do think it's a very good photo management application.

I was no way ready for the same exercise again so back to Google again and found another solution fslint which shows duplicate files but is still manual work to remove duplicates.

Further search gave me wonderful command on one of the forums using fdupes which did a very good job for me. I was bit skeptical in first place with fear to loose some files so ran it without -d option like
fdupes -r photos/clean_album and reviewed output and was happy with the listing so when ahead and used it as suggested by forum
yes 1 | fdupes -rd photos/clean_album and I had my 95% or duplicate removal work done. (I wish I should have used it in first place) Rest of the work was to clean up using Digikam and then flatten them into one directory where I have some name conflicts which I used as my blog name low-tech approach and added prefixes to avoid clashes.

End result 21GB is now reduced to 8.5GB.

I think I should be more careful going forward about duplicates and backups.

EDIT: Another note is I did miss the avi clips when I did manual cleanup using digiKam but fslint and fdupes sorted those out as well.

How to install Oracle xe on Ubuntu 64 bit?

I wanted to install Oracle Xe on Ubuntu 64bit and was faced with the problem that the debian package that Oracle has with free edition is 32 bit so googled a bit for more information and found a link that helped me to configure it.

Here is a link to keep note of : http://littlebrain.org/2008/05/12/how-to-install-oracle-xe-in-ubuntu-64-bit/