Tuesday, October 21, 2014

The quest for an OSGI friendly web framework: Day 2 => Apache Karaf

Day 2 =>Apache Karaf

Synopsis

Karaf is an OSGI Container, but then it works on top of Eclipse Equinox or Apache Felix. mmh.. I feel my eyebrows raising as what it really is then. The website promises to support what I am after, which is getting a webframework which works nicely in OSGI.

This blog is one of the articles in a series named Quest for an OSGI friendly Webframework, this as OSGI bundles which can be started/stopped etc.. and we have 1 day for it!.

Requirements

To work along this exercise, you will need:
- An Eclipse installation (Kepler or Luna).
- Maven Plugin for Eclipse (m2e) and the p2-maven-plugin
- Knowledge of Eclipse PDE Concepts (Features, Plugins, Target Platform, P2 Repositories, Products).

WARNING: This is not a step-by-step instruction, you will need to make sure how to add missing OSGI and other features and bundles to make it all work. 

Result - PARTIAL SUCCESS

Karaf is amazing, it takes OSGI to the next level. However there are several issues for now.
First the EIK tooling is behind the Eclipse releases, or as the bug report points out, it's EIK in combination with later Karaf versions. And as it turns out, tooling is essential, as our Eclipse bundles, don't fit as Karaf features. It would be nice if Karaf could process P2 repositories, but that's not the case. It's very maven centric as to be expected and not Eclipse centric. Different worlds again....
As for the objective to deploy a web-app, well the attempt to use the Eclipse implementation of HTTPService failed. It simply won't load easily in Apache, the error I get still unresolved.

Diving-In!

The first question I asked myself after having browsed the Karaf website is: How do I bundle my application with Karaf? The instructions to get going with Karaf is to download it and run it, so why not give it a try:


Christophes-MacBook-Pro:bin Christophe$ ./karaf
        __ __                  ____     
       / //_/____ __________ _/ __/     
      / ,<  / __ `/ ___/ __ `/ /_       
     / /| |/ /_/ / /  / /_/ / __/       
    /_/ |_|\__,_/_/   \__,_/_/        

  Apache Karaf (4.0.0.M1)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown Karaf.


Hey we are up and running, that's pretty cool already.
So let's try a couple of Karaf commands:

Show the installed features:

karaf@root()> feature:list -i
Name            | Version  | Required | Installed | Repository        | Description                                      
--------------------------------------------------------------------------------------------------------------------------
aries-proxy     | 4.0.0.M1 |          | x         | standard-4.0.0.M1 | Aries Proxy                                      
aries-blueprint | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Aries Blueprint                                  
shell           | 4.0.0.M1 |          | x         | standard-4.0.0.M1 | Karaf Shell                                      
shell-compat    | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Karaf Shell Compatibility                        
deployer        | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Karaf Deployer                                   
bundle          | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide Bundle support                           
config          | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide OSGi ConfigAdmin support                 
diagnostic      | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide Diagnostic support                       
instance        | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide Instance support                         
jaas            | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide JAAS support                             
log             | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide Log support                              
package         | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Package commands and mbeans                      
service         | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide Service support                          
system          | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide System support                           
kar             | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide KAR (KARaf archive) support              
ssh             | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide a SSHd server on Karaf                   
management      | 4.0.0.M1 | x        | x         | standard-4.0.0.M1 | Provide a JMX MBeanServer and a set of MBeans in K
wrap            | 0.0.0    | x        | x         | standard-4.0.0.M1 | Wrap URL handler                  


Getting my bundles in there (with EIK)

Now, I am still puzzled, how I will package the whole application, or will I have to tell my users to install Karaf and then deploy my bundles. Well, that's not even such a bad idea after all, so let's try to get a couple of bundles in Karaf.

One of the things I look out for, is the availability of tooling for Eclipse and shabang, there is EIK, which stands for Eclipse Integration Karaf, which can be downloaded and installed.

It's documented here

Not sure about what this tooling can do, one hint is to type karaf in the "Quick Access" search box.
EIK comes with a Perspective which shows the Bundles and Services View also contributed by karaf.

It also has a wizard to create a project and tell EIK about the Karaf Installation:




Browse to the installation of Karaf and click finish.

Unfortunately, the wizard throws an error. (This is the Karaf builder). 

java.lang.NoClassDefFoundError: org/eclipse/pde/internal/core/target/provisional/ITargetPlatformService
    at org.apache.karaf.eik.ui.project.KarafProjectBuilder.createTargetPlatform(KarafProjectBuilder.java:332)
    at org.apache.karaf.eik.ui.project.KarafProjectBuilder.fullBuild(KarafProjectBuilder.java:165)
    at org.apache.karaf.eik.ui.project.KarafProjectBuilder.incrementalBuild(KarafProjectBuilder.java:386)
    at org.apache.karaf.eik.ui.project.KarafProjectBuilder.build(KarafProjectBuilder.java:85)
    at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:733)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:206)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:246)
    at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:299)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:302)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:358)
    at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:381)
    at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:143)
    at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:241)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)


And I am never the first of course: https://issues.apache.org/jira/browse/KARAF-2668


We can still continue though....

The create project, will basically link to various folders in the Karaf installation:

According to the documentation of EIK, it should be possible to deploy bundles in the workspace to Karaf with the EIK tooling. Also there is should be a Karafspecific .target, but the error above suggests the .target was not installed properly...

The EIK documentation (Which is not Apache worthy btw, the wording is so bad it hurts!),  lists compatibility with Eclipse versions, and I am surprised to see Kepler and Luna missing. (Kepler has been out for > 1 year at the time of writing).

mmh EIK is behind is my conclusion sofar..

Let's continue...(This doesn't feel right). 

The next step is to create a launch config. As we likely miss bundles in the TargetPlatform, I am curious to see if this will work... .another exception....


java.lang.NullPointerException
    at org.apache.karaf.eik.ui.KarafLaunchConfigurationDelegate.fixKarafJarClasspathEntry(KarafLaunchConfigurationDelegate.java:428)
    at org.apache.karaf.eik.ui.KarafLaunchConfigurationDelegate.getClasspath(KarafLaunchConfigurationDelegate.java:187)
    at org.eclipse.pde.launching.AbstractPDELaunchConfiguration.launch(AbstractPDELaunchConfiguration.java:70)
    at org.apache.karaf.eik.ui.KarafLaunchConfigurationDelegate.launch(KarafLaunchConfigurationDelegate.java:246)
    at org.eclipse.pde.launching.OSGiLaunchConfigurationDelegate.launch(OSGiLaunchConfigurationDelegate.java:47)
    at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:858)
    at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:707)
    at org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:1018)
    at org.eclipse.debug.internal.ui.DebugUIPlugin$8.run(DebugUIPlugin.java:1222)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)


Getting my bundles in there (without EIK)

So playing with EIK, has been very, very disappointing and sorry, I am not going back to Indigo to see it works then... I have an option to not use EIK tooling, let's try that.

Now the instructions on the Karaf website tell to produce a maven pom file. mmh, but I already have OSGI bundles, I want to deploy them, what are my options then?

Reading on, I can deploy a Karaf feature. There is a hot-deploy folder, where I can dump the feature.xml. Unless this gets implemented, I am afraid it will be a manual conversion process for now.

The Karaf feature will contain the references to bundles for this feature.

Here is what it looks like, I purposely do not fill in the bundle section, just to see what happens:

<?xml version="1.0" encoding="UTF-8"?>
<features name="oss2-features" xmlns="http://karaf.apache.org/xmlns/features/v1.2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.2.0 http://karaf.apache.org/xmlns/features/v1.2.0">
  <feature name="oss2" version="1.0.0">
    <bundle>file:///Users/Christophe/Documents/Projects/GIT_netxstudio/plugins/base/com.netxforge.base/target/com.netxforge.base-1.0.0-SNAPSHOT.jar</bundle>
  </feature>
</features>


Now copying this file to the karaf installation folder/deploy should process the feature.

And indeed, the file is process. typing log:tail shows that this happend:

2014-10-16 16:35:29,151 | INFO  | -4.0.0.M1/deploy | fileinstall                      | 5 - org.apache.felix.fileinstall - 3.4.2 | Updating bundle feature.xml / 0.0.0
2014-10-16 16:35:29,166 | INFO  | -4.0.0.M1/deploy | fileinstall                      | 5 - org.apache.felix.fileinstall - 3.4.2 | Started bundle: feature:file:/Users/Christophe/Downloads/apache-karaf-4.0.0.M1/deploy/feature.xml
2014-10-16 16:35:29,176 | INFO  | lixDispatchQueue | FeaturesServiceImpl              | 6 - org.apache.karaf.features.core - 4.0.0.M1 | Adding features:
2014-10-16 16:35:29,449 | INFO  | pool-29-thread-1 | FeaturesServiceImpl              | 6 - org.apache.karaf.features.core - 4.0.0.M1 | No deployment change.
2014-10-16 16:35:29,452 | INFO  | pool-29-thread-1 | FeaturesServiceImpl              | 6 - org.apache.karaf.features.core - 4.0.0.M1 | Done.


So now let's populate the bundle section, with a file:// reference to one of my bundles:

  <bundle>file:///Users/Christophe/Documents/Projects/GIT_netxstudio/plugins/base/com.netxforge.base/target/com.netxforge.base-1.0.0-SNAPSHOT.jar</bundle>

The feature is now visible:

oss2                          | 1.0.0                            |          |           | oss2-features           |          

and trying to install it:

karaf@root()> feature:install oss2

Error executing command: Unable to resolve root: missing requirement

[root] osgi.identity; osgi.identity=oss2; type=karaf.feature; version="[1.0.0,1.0.0]"; filter:="(&(osgi.identity=oss2)(type=karaf.feature)(version>=1.0.0)(version<=1.0.0))"

[caused by: Unable to resolve oss2/1.0.0: missing requirement [oss2/1.0.0] osgi.identity; osgi.identity=com.netxforge.base; type=osgi.bundle; version="[1.0.0.201409161447,1.0.0.201409161447]"; resolution:=mandatory

[caused by: Unable to resolve com.netxforge.base/1.0.0.201409161447: missing requirement [com.netxforge.base/1.0.0.201409161447] osgi.wiring.package; filter:="(osgi.wiring.package=org.eclipse.emf.common.notify)"]]



So this fails on dependencies (as expected), but I am a bit puzzled why these are not listed. I would expect to see a couple of required bundles. BTW It took me a bit of puzzling to understand the log,
but what it says is. 1) There is a problem with oss2 feature, 2) there is a problem with bundle com.netxforge.base 3) there is a problem with a requirement org.eclipse.emf.common.notify


As an alternative, I am going to use a karaf maven archetype facility to create a bundle:

mvn archetype:generate \
    -DarchetypeGroupId=org.apache.karaf.archetypes \
    -DarchetypeArtifactId=karaf-bundle-archetype \
    -DarchetypeVersion=2.2.5-SNAPSHOT \
    -DgroupId=com.netxforge \
    -DartifactId=com.netxforge.oss2.kbundle \
    -Dversion=1.0-SNAPSHOT \
    -Dpackage=com.netxforge.oss2.kbundle


The archetype will create a bundle the maven way.
That is typically slightly different than the way an Eclipse PDE bundle looks like.

Some noticable differences are the fact that the /src folder in Eclipse is now /src/main/java. That there is no META-INF folder holding the MANIFEST.MF file and that there is a pom.xml. So a typical maven project. The pom is interesting, as it includes the configuration to produce a bundle.


    <dependencies>
        <dependency>
            <groupId>org.osgi</groupId>
            <artifactId>org.osgi.core</artifactId>
            <version>${osgi.version}</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>${maven-bundle-plugin.version}</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
                        <Bundle-Version>${project.version}</Bundle-Version>
                        <Bundle-Activator>com.netxforge.oss2.kbundle.Activator</Bundle-Activator>
                        <Export-Package>
                            com.netxforge.oss2.kbundle*;version=${project.version}
                        </Export-Package>
                        <Import-Package>
                            *
                        </Import-Package>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>



Next, I ran run as -> maven build... with goals 'clean install'
This produces the well known target folder with the .jar file in it.

Now trying to install this bundle in Karaf works nicely:

karaf@root()> bundle:list
START LEVEL 100 , List Threshold: 50
ID | State  | Lvl | Version        | Name                            
----------------------------------------------------------------------
49 | Active |  80 | 0.0.0          | feature.xml                     
91 | Active |  80 | 0.0.1.SNAPSHOT | com.netxforge.oss2.kbundle Bundle
karaf@root()> bundle:info 91

com.netxforge.oss2.kbundle Bundle (91)
--------------------------------------

Back to the web 

Web is very much build into Karaf. There is support for classic .WAR files but also OSGI based .WAB files. The following instructions tell us how to install the feature.

The log in karaf will show this:

2014-10-16 21:23:48,024 | INFO  | pool-69-thread-1 | Server                           | 73 - org.eclipse.jetty.util - 9.0.7.v20131107 | jetty-9.0.7.v20131107
2014-10-16 21:23:48,056 | INFO  | pool-69-thread-1 | JettyServerImpl                  | 87 - org.ops4j.pax.web.pax-web-jetty - 4.0.0 | Pax Web available at [0.0.0.0]:[8181]
2014-10-16 21:23:48,117 | INFO  | pool-69-thread-1 | ServerConnector                  | 73 - org.eclipse.jetty.util - 9.0.7.v20131107 | Started default@7003e8b7{HTTP/1.1}{0.0.0.0:8181}


So here, we see a server running on port 8181. How cool.
Firing a browser and localhost:8181 shows indeed a Jetty server running and listing.

HTTP ERROR: 404

Problem accessing /. Reason:
    Not Found

Powered by Jetty://

So, the final task to create a .wab file and deploy it.
But before doing so, there already a couple of deployable web features.

feature:install webconsole


Navigating to the url: localhost:8181/system/console/gogo
gives us this:



Creating a Servlet and Karaf DS

Now, with Eclipse Equinox, I use Declarative Services with annotations.
I wanted to do the same with a Karaf (Or Maven POM first bundle) to reference the HTTPService , but this isn't a trivial thing. This entry explains it, and I will dive into it later on.

What Karaf recommends however is a .wab deployment. a wab is like a war, but then for OSGI.
There is some tooling for a .wab in the Virgo project, as we can see here. That's cool, but do I really need to install more tooling, and why isn't this part of EIK and many more questions....

So, in any event, I want to use this package: org.osgi.service.http (BTW: the full OSGI 4.2 API can be found here ). 
 
Now, I need to find an implementation for it. Now that's an issue, with with pom first driven development. (I am used to Eclipse, and for getting dependencies, I get a P2 location and bang it's my .target to compile against and use in features and products). For Maven driven dev. I need to look for an implementation from maven central. so I type in: org.osgi.service.http, but nothing shows. 

So, I know there is Equinox and there is Apache Felix (...and yes there are a couple more).

For Apache Felix, I can use these instructions, but for Equinox I can't do maven first. (Please prove me wrong!). The reason is that Equinox is Eclipse and Eclipse is P2 not Maven. (Wow that's blunt, but very true). Compare it with the Apache Felix instructions and you will see that I mean. 

So, first I will deploy an Equinox implementation of HTTPService in Karaf build with Manifest first and Tycho, and next I will deploy an Apache Felix implementation using POM first. 

Equinox HTTPService: 

  1. I create a plugin-Project. Configure it as a maven project and update pom.xml with tycho stuff. (I have a parent pom.xml which sets the target platform to use etc...).
  2. Add it to a Karaf feature, which dump in the /deploy folder. 
  3. Install the feature  (Same as above).
 karaf@root()> bundle:list
START LEVEL 100 , List Threshold: 50
ID | State  | Lvl | Version            | Name                 
---------------------------------------------------------------
49 | Active |  80 | 0.0.0              | feature.xml          
92 | Active |  80 | 3.0.0              | Apache Karaf :: Manual
99 | Active |  80 | 1.0.0.201410201328 | OSS2 HTTP Service    


Perfect, the bundle get's installed, but I haven't added the HTTPService dependency yet, so let's do that. The instructions are here but I don't want to use the extension registry, so I follow a regular HTTPService registration pattern.

The service looks like this:  (It uses OSGI Annotations and registers the service to respond to HTTP GET request).

@Component
public class WebDude{

    private HttpService httpService;

    @Activate
    public void activate() {
        try {
            httpService.registerServlet("/dudeme", new WebDudeServlet(), null, null);
        } catch (Exception exception) {
            exception.printStackTrace();
        }
    }

    @Reference
    public void setHTTPService(HttpService httpService) {
        this.httpService = httpService;
    }
   
    class WebDudeServlet extends HttpServlet {
        private static final long serialVersionUID = 1L;

        @Override
          protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
            resp.getWriter().write("I am dude");     
          }
    }
}


And then I build, and notice the dependencies which Karaf will need to know about.
So the jars are in the eclipse bundle pool. (Remember this is an eclipse tycho build which uses a target platform).



 The challenge remaining is to get these dependencies in a karaf feature or add the bundles manually.
(It dawns that a bundle pool scanner, could populate karaf automagically with a custom OBR, mmhhh...).

So I decide to refer the bundles from the eclipse installation bundle-pool in a Karaf feature and
install the feature:

This looks ok....

karaf@root()> bundle:list
START LEVEL 100 , List Threshold: 50
 ID | State    | Lvl | Version                | Name                                                
-----------------------------------------------------------------------------------------------------
 49 | Active   |  80 | 0.0.0                  | feature.xml                                         
 92 | Active   |  80 | 3.0.0                  | Apache Karaf :: Manual                              
110 | Active   |  80 | 1.0.0.201410201421     | OSS2 HTTP Service                                   
111 | Active   |  80 | 3.0.0.v201112011016    | Servlet API Bundle                                  
112 | Resolved |  80 | 1.0.401.v20130327-1442 | Transformer Hook Framework Extension, Hosts: 114    
113 | Resolved |  80 | 1.0.200.v20130327-1442 | Aspect Weaving Hooks Plug-in (Incubation), Hosts: 114
114 | Resolved |  80 | 3.9.1.v20140110-1610   | OSGi System Bundle, Fragments: 113, 112             
115 | Active   |  80 | 3.3.100.v20130513-1956 | OSGi Release 4.2.0 Services


but org.eclipse.osgi fails to start...maybe not a good idea to start org.eclipse.osgi on Karaf then?

2014-10-20 16:51:00,723 | INFO  | ool-206-thread-1 | FeaturesServiceImpl              | 6 - org.apache.karaf.features.core - 4.0.0.M1 |   org.eclipse.osgi / 3.9.1.v20140110-1610
2014-10-20 16:51:00,724 | ERROR | ool-181-thread-2 | FeatureDeploymentListener        | 24 - org.apache.karaf.deployer.features - 4.0.0.M1 | Unable to install features
org.apache.karaf.features.internal.util.MultiException: Error restarting bundles
    at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:765)[6:org.apache.karaf.features.core:4.0.0.M1]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:951)[6:org.apache.karaf.features.core:4.0.0.M1]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl$1.call(FeaturesServiceImpl.java:857)[6:org.apache.karaf.features.core:4.0.0.M1]
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)[:1.7.0_07]
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)[:1.7.0_07]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)[:1.7.0_07]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)[:1.7.0_07]
    at java.lang.Thread.run(Thread.java:722)[:1.7.0_07]
Caused by: org.osgi.framework.BundleException: Activator start error in bundle org.eclipse.osgi [114].
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2204)[org.apache.felix.framework-4.4.1.jar:]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2072)[org.apache.felix.framework-4.4.1.jar:]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:976)[org.apache.felix.framework-4.4.1.jar:]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:963)[org.apache.felix.framework-4.4.1.jar:]
    at org.apache.karaf.features.internal.service.FeaturesServiceImpl.startBundle(FeaturesServiceImpl.java:1030)[6:org.apache.karaf.features.core:4.0.0.M1]
    at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:757)[6:org.apache.karaf.features.core:4.0.0.M1]
    ... 7 more
Caused by: java.lang.ClassCastException: org.eclipse.osgi.framework.internal.core.SystemBundleActivator cannot be cast to org.osgi.framework.BundleActivator
    at org.apache.felix.framework.Felix.createBundleActivator(Felix.java:4362)[org.apache.felix.framework-4.4.1.jar:]
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2149)[org.apache.felix.framework-4.4.1.jar:]
    ... 12 more


Karaf can also be launched with Eclipse Equinox instead of Felix, let's start that then.

So, I tried to leave org.eclipse.osgi out, and configure Karaf using equinox.
The latter is done, by configuring the the entry:

#
# Framework selection properties
#
karaf.framework=equinox
 

Unfortunately this fails with:

Error executing command: Uses constraint violation. Unable to resolve resource org.apache.aries.blueprint.core [org.apache.aries.blueprint.core/1.4.1] because it is exposed to package 'org.osgi.service.framework' from resources org.eclipse.osgi [org.eclipse.osgi_3.9.1.v20140110-1610] and org.eclipse.osgi [org.eclipse.osgi_3.9.1.v20140110-1610] via two dependency chains.

Chain 1:
  org.apache.aries.blueprint.core [org.apache.aries.blueprint.core/1.4.1]
    import: (osgi.wiring.package=org.osgi.service.framework)
     |
    export: osgi.wiring.package: org.osgi.service.framework
  org.eclipse.osgi [org.eclipse.osgi_3.9.1.v20140110-1610]

Chain 2:
  org.apache.aries.blueprint.core [org.apache.aries.blueprint.core/1.4.1]
    import: (&(osgi.wiring.package=org.apache.aries.util.tracker)(version>=1.0.0)(!(version>=2.0.0)))
     |
    export: osgi.wiring.package=org.apache.aries.util.tracker; uses:=org.osgi.service.framework
  org.apache.aries.util [org.apache.aries.util/1.1.0]
    import: (&(osgi.wiring.package=org.osgi.service.framework)(version>=1.0.0)(!(version>=2.0.0)))
     |
    export: osgi.wiring.package: org.osgi.service.framework
  org.eclipse.osgi [org.eclipse.osgi_3.9.1.v20140110-1610]


So a problem, I am having a hard time understanding. A dead-end I am afraid.
I am confident Karaf will run with the default http service. (feature:install http), but this will be a dependency I will need to load in Eclipse IDE to work against. Eclipse targets and maven centric solutions are unfortunately still lightyears apart. (Tycho eases from Eclipse to maven, but the other way around??). 

Custom Deployment after all

And then I stumbled on this:

http://karaf.apache.org/manual/latest/developers-guide/custom-distribution.html

So with a bit of work, I could setup a distribution, fully configured with my Application specific artifacts.

What about head-less builds

Sofar I am building Eclipse artifacts with Tycho. This blog-entry is about a similar experience.

http://eclipsesource.com/blogs/2012/08/22/from-eclipse-tycho-to-apache-karaf-the-easy-way/

As Karaf is very much maven centric, our bundles would need to be deployed on a  maven repo. But that's for a later concern.

Thursday, October 16, 2014

The quest for an OSGI friendly web framework: Day 1 => Spring and Spring DM

Day 1 => Spring and Spring DM

Synopsis

Springframework has a solid user base and one of the most established Java Web Frameworks. It takes care of a lot of things for building Enterprise Web applications. But then there is OSGI which is a solid foundation for application Modularity.

This blog is one of the articles in a series named Quest for an OSGI friendly Webframework, this first time we turn to about making a minimal SpringFramework web application running as an OSGI bundle which can be started/stopped etc.. and we have 1 day for it!.

Requirements

To work along this exercise, you will need:
- An Eclipse installation (Kepler or Luna).
- Maven Plugin for Eclipse (m2e) and the p2-maven-plugin
- Knowledge of Eclipse PDE Concepts (Features, Plugins, Target Platform, P2 Repositories, Products).

WARNING: This is not a step-by-step instruction, you will need to make sure how to add missing OSGI and other features and bundles to make it all work. 

Result - FAILED

This failed. Spring mechanisms do not fit in OSGI. I ran into a class loading issue, which doesn't seem resolvable. I also tried Spring-DM, but the last version (1.2.1) is not dependency compatible with Spring 4 framework. A bit more reading, learned me it was donated to Eclipse in the Gemini project known as the Gemini-Blueprint. It's alive for sure. An evaluation of Gemini-blueprint will have to follow. So after one day, not even an HTTP service was up and running...

Still readers might still be interested in reading about the details and approach of this attempt.

Diving-in!

The OSGI container I use is Eclipse equinox. Alternatives could Apache Felix, Karaf or another OSGI container.  Depending on the OSGI implementation, the way bundles are deployed differs.

What I know well is Eclipse Equinox OSGI and all the tooling for it. I am less familiar with the SpringFramework. One way to run an Eclipse application is by creating a .product This will reference Eclipse Features, which will then reference the OSGI bundles. Now, it seems a good approach to create an Eclipse Feature which will contain all the needed SpringFramework bundles.

Make the SpringFramework Bundle available to Eclipse

So the first task at hand is to make sure the SpringFramework bundles are available in the Eclipse target platform or in the Workspace to be able to add them to a feature. Getting in the workspace as source code, could be done by cloning the source repository, but I decide on another approach, which is to produce a P2 repository which can then be used to populate the Eclipse Target Platform.

Now it turns out, there is this very cool Maven plugin, which can produce a P2 Repositories from bundles available on a Maven repository. The maven plugin name is p2-maven-plugin.

Following the instructions it is possible to produce a P2 repository for the Springframework bundles.
The produced P2 can then be published on an HTTP server kept locally for consumption by a Eclipse target definition.

In my case, I have pushed the P2 to a web server:

[Screen shot of the P2?]

Now, I can reference the P2 repository and the target definition will look like this:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?pde?>
<!-- generated with https://github.com/mbarbero/fr.obeo.releng.targetplatform -->
<target name="springframework" sequenceNumber="1413366361">
  <locations>
    <location includeMode="slicer" includeAllPlatforms="true" includeSource="true" includeConfigurePhase="false" type="InstallableUnit">
      <unit id="org.springframework.aop" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.asm" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.beans" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.context" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.core" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.expression" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.instrument" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.jdbc" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.orm" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.transaction" version="3.0.7.RELEASE"/>
      <unit id="org.springframework.security.core" version="0.0.0"/>
      <unit id="org.springframework.security.spring-security-crypto" version="0.0.0"/>
      <repository location="http://p2.netxforge.com/oss2/mvn.p2"/>
    </location>
  </locations>
</target>

Now, it's not said this is the complete list of required Spring Framework bundles, but the basic idea is there.

Now we can load this target platform in Eclipse. (Actually a more complete TP is required, but for illustration purpose, I only list the springframwork repo location in the target above).

Create an Eclipse feature for the SpringFramework (and it's dependencies). 

File -> New -> Feature Project
I named it org.springframework.artifacts.feature

To get started easy, I would recommend just adding

org.springframework.core
org.springframework.expression

...bundles to the feature. Just to see what happens and what can be consumed from this bundle already.

Add the feature to a .product file and run the product.

Assuming you are familiar with building Eclipse .product files (Which are feature based), Create a .product then add the feature you just defined plus the Equinox bundles.

From the product editor it is now possible to verify the dependencies, and you might be required to add a couple of missing plugins, like org.apache.commons.logging. The more Springframework bundles we will add to the feature, the more 3rd party dependencies will be required, hé!

Now run the product. (I usually specify -console 8811 in the product launch configuration setting, so I can get to the OSGI prompt). Oh and you will need the osgi.console etc... bundles as well...

[4] What's loaded.

Open a terminal and type: telnet 0 8811  to open the OSGI console.
and then perform:

osgi> ss org.springframework
"Framework is launched."


id    State       Bundle
16    RESOLVED    org.springframework.core_3.0.7.RELEASE
40    RESOLVED    org.springframework.expression_3.0.7.RELEASE

Now we will observe the springframework bundles are present in our OSGI application, hooray!
They remain in state RESOLVED, as they are not consumed or force started.

More details can be obtained with

osgi> bundle 16 (See output [1] org.springframework.core bundle details).

From this output we can see which packages this bundle exports and imports. It's interesting to see, the bundle can be activate, while some of the imported packages are not available

For example, the package here is imported by org.springframework.core, but is not exported by any of the bundles.

osgi> packages org.springframework.asm
No exported packages

This is Ok, as these are listed as optional (See [1]

Create an application context

The basic spring application which can be obtained from one of the Spring getting-started guides is an ApplicationContext  and a couple of java classes with Spring annotations.

The guide I followed is this one.

In the example, there is a java main method, but in OSGI we use bundle activators or declarative services to bootup code. So I have created a bundle, activator and a DS to do exactly this.

the DS Service looks like this: (Note: The @Component annotation is an OSGI annotation, not Spring).

@Component
public class SpringService {

    @Activate
    public void activate() {
        @SuppressWarnings("resource")
        ApplicationContext context = new AnnotationConfigApplicationContext(
                com.netxforge.oss2.spring.app.SpringApp.class);
        MessagePrinter printer = context.getBean(MessagePrinter.class);
        printer.printMessage();
    }

}

The SpringApp class looks like this:


@Configuration
@ComponentScan
public class SpringApp {

    @Bean
    MessageService mockMessageService() {
        return new MessageService() {
            public String getMessage() {
              return "Hello World!";
            }
        };
    }
   
}

This bundle is then packaged with the Eclipse product (As part of a feature), with feature dependencies to the required Springframework bundles.

When launching OSGI, the services are activated, but explodes with a stacktrace. (OSGI tries to activate the service several times. Showing the component status with:

osgi>ls

4    Unsatisfied        com.netxforge.oss2.spring.app.SpringService            com.netxforge.oss2.spring.app(bid=25)


More details...

osgi> comp 4
    Component[
    name = com.netxforge.oss2.spring.app.SpringService
    activate = activate
    deactivate = deactivate
    modified =
    configuration-policy = optional
    configuration-pid = com.netxforge.oss2.spring.app.SpringService
    factory = null
    autoenable = true
    immediate = true
    implementation = com.netxforge.oss2.spring.app.SpringService
    state = Unsatisfied
    properties =
    serviceFactory = false
    serviceInterface = null
    references = null
    located in bundle = com.netxforge.oss2.spring.app_1.0.0.qualifier [25]
]
Dynamic information :
  The component is satisfied
  All component references are satisfied
  Component configurations :
    Configuration properties:
      component.name = com.netxforge.oss2.spring.app.SpringService
      component.id = 3
    Instances:
    No instances were created because: Can not activate instance of component com.netxforge.oss2.spring.app.SpringService. The activation throws: java.lang.IllegalStateException: Cannot load configuration class: com.netxforge.oss2.spring.app.SpringApp

So, the SpringApp class can be loaded by Spring. In the stacktrace, the following entry is responsible:

ConfigurationClassPostProcessor

After a bit of code inspection and debugging, I realized the classloading mechanism will try to load the class from the spring bundle spring-context, but without knowledge of my application bundle and the SpringApp.class.

The obtained classloader is: Thread.currentThread().getContextClassLoader(); (ClassUtils of Spring).

Now in OSGI loading a class should happen using the Bundle, like this:
final Class<?> clazz = bundle.loadClass(clazzName);

This seems the likely cause of the class loading issue. I am also thinking about buddy-loading policies, but this would need us to modify the spring bundle MANIFEST.MF , which have been generated and archived, so that doesn't seem the right path.

sight... a dead-end here. I could dive into it, but it dawns, that a monolithic class framework like Spring will never fit well in OSGI unless..

Spring DM / Gemini-Blueprint

So, one of my bad-habits is not to accept the situation and move on. I had to spend a bit more time scanning the web for more information on the subject, and then I stumbled on Spring DM. Wow, this is what I needed in the first place, was the first reaction. I also found this very nice article dating 2012 about the topic: http://angelozerr.wordpress.com/about/eclipse_spring/

Wow, this is exactly what i want to do. Role up the sleeves and get going then.
So, I updated my P2 repo to include the Spring-DM bundles (Which are not called DM btw), where the version is 1.2.1

                                 <artifact>
                                    <id>org.springframework.osgi:spring-osgi-extender:${springDMVersion}</id>
                                    <source>true</source>
                                </artifact>


And the bundles showed up, so I added to my Eclipse feature, but then I got a version error!
What?!?! Spring DM is not compatible with Spring Framework 4?

A bit of more research revealed that Spring-DM was abandoned and given to Eclipse to become part of the Gemini project as the Gemini-blueprint. The development is active, but I am not sure about the pace, and I am still very much put off by the fact that it's not supporting the current Spring version (4.x). Someone on the forum asked about it, but the answer was "not yet", so when remains to be answered.

I don't want to give up on Gemini yet, but time's up. The sun is down, and a clear night-sky illuminates the my balcony. Time to shutdown for the night!


[1] 0utput from osgi>bundle 16

org.springframework.core_3.0.7.RELEASE [16]
  Id=16, Status=ACTIVE      Data Root=/Users/Christophe/Documents/Spaces/netxstudio/.metadata/.plugins/org.eclipse.pde.core/oss2app.product/org.eclipse.osgi/bundles/16/data
  "No registered services."
  No services in use.
  Exported packages
    org.springframework.core; version="3.0.7.RELEASE"[exported]
    org.springframework.core.annotation; version="3.0.7.RELEASE"[exported]
    org.springframework.core.convert; version="3.0.7.RELEASE"[exported]
    org.springframework.core.convert.converter; version="3.0.7.RELEASE"[exported]
    org.springframework.core.convert.support; version="3.0.7.RELEASE"[exported]
    org.springframework.core.enums; version="3.0.7.RELEASE"[exported]
    org.springframework.core.io; version="3.0.7.RELEASE"[exported]
    org.springframework.core.io.support; version="3.0.7.RELEASE"[exported]
    org.springframework.core.serializer; version="3.0.7.RELEASE"[exported]
    org.springframework.core.serializer.support; version="3.0.7.RELEASE"[exported]
    org.springframework.core.style; version="3.0.7.RELEASE"[exported]
    org.springframework.core.task; version="3.0.7.RELEASE"[exported]
    org.springframework.core.task.support; version="3.0.7.RELEASE"[exported]
    org.springframework.core.type; version="3.0.7.RELEASE"[exported]
    org.springframework.core.type.classreading; version="3.0.7.RELEASE"[exported]
    org.springframework.core.type.filter; version="3.0.7.RELEASE"[exported]
    org.springframework.util; version="3.0.7.RELEASE"[exported]
    org.springframework.util.comparator; version="3.0.7.RELEASE"[exported]
    org.springframework.util.xml; version="3.0.7.RELEASE"[exported]
  Imported packages
    org.apache.commons.logging; version="1.1.1"<org.apache.commons.logging_1.1.1.v201101211721 [11]>
    org.xml.sax.helpers; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    org.xml.sax.ext; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    org.xml.sax; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    org.w3c.dom; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    org.eclipse.core.runtime; version="3.4.0"<org.eclipse.equinox.common_3.6.100.v20120522-1841 [1]>
    org.apache.log4j.xml; version="1.2.15"<org.apache.log4j_1.2.15.v201012070815 [28]>
    org.apache.log4j; version="1.2.15"<org.apache.log4j_1.2.15.v201012070815 [28]>
    javax.xml.transform.stax; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.transform.sax; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.transform; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.stream.util; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.stream.events; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.stream; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    javax.xml.namespace; version="0.0.0"<org.eclipse.osgi_3.9.1.v20140110-1610 [0]>
    org.aspectj.bridge; version="[1.5.4,2.0.0)"<unwired><optional>
    org.aspectj.weaver; version="[1.5.4,2.0.0)"<unwired><optional>
    org.aspectj.weaver.bcel; version="[1.5.4,2.0.0)"<unwired><optional>
    org.aspectj.weaver.patterns; version="[1.5.4,2.0.0)"<unwired><optional>
    org.jboss.vfs; version="[3.0.0,4.0.0)"<unwired><optional>
    org.jboss.virtual; version="[2.1.0.GA,3.0.0)"<unwired><optional>
    org.springframework.asm; version="[3.0.7,3.0.8)"<unwired><optional>
    org.springframework.asm.commons; version="[3.0.7,3.0.8)"<unwired><optional>
  No fragment bundles
  Named class space
    org.springframework.core; bundle-version="3.0.7.RELEASE"[provided]
  No required bundles

































The quest for an OSGI friendly web framework

In a series of blog I am investigating and trying the available Java web-frameworks and how well they play in OSGI. I am spending exactly one day with each of them.

Background

NetXStudio is the application I have been working on the last few years. It allows Telecommunications Service Providers to collect performance data from Network components and resource, aggregate the data, and perform calculations and analysis on it. The main business goal is to act timely on congestion in the network.

It's an Eclipse RCP application with a server backend. The communication is through Eclipse CDO, which is a ORM solution on steroids. ( Besides mapping Java Objects to relational databases, it also works with non-relational DB's. On top CDO has an advanced 'audit' mode which allows to go back in time, and fetch an historical object graph, it's sort of ORM+GIT in one solution). Server communication happens through CDO communication protocol named Net4J. Additional communication, has been implemented through some HTTP calls, and a simple servlet implementation, which kicks of some processes. Here Eclipse Equinox HTTP bundles are used.

Both client and server are build with Eclipse, and grouped in OSGI bundles. On top, the Eclipse Equinox implementation provides additional functionality to deploy with products and features.

So what's next

So far so good, the application is in production and the customer is mostly happy. Mostly, because there is still lack of functionality, but also some impracticalities like not being web-based. 

As with many projects at some point there is time for reflection and planning for the future. In the case of NetXStudio, the idea is to bring the application to a wider audience by making it a modular platform with re-useable software components. Now with OSGI this is a breeze, applying OSGI Declarative Services allows true discovery of services, and service chaining and much more. With little effort most of the server side functionality was modularized and services can come and go. 

Thee is also a wish to make the UI fully web-based.

OSGI Ready web-framework 

So for a web-based UI, a web-framework is needed, with the capabilities to play nicely with OSGI.
Now, I don't really want to bother about the definition of "playin nicely", that would take up some time, I believe it's better to take 'them' for a spin and see what comes out.

The following solutions are explored:
  1. Springframework and Spring-DM
  2. Apache Karaf
  3. PAX Web 
  4. Equinox HTTP Services 
  5. Gemini-Blueprint 
  6. Vert.X (Added 17-120-2014)
  7. ... (The list will grow over time I believe). 

Setting the scene

So, this is a rather big task I am committing myself to. A couple of rules to make the comparison fair and the exercise, fun and painless.
  1. Get the framework up and running in one day.  
  2. Should run on Eclipse Equinox. 
  3. Have some interaction with my application bundles. (Serve objects to the web UI). 
  4. Give some feedback on deployment and configuration.