• EN
Solved

RabbitMQ JMS Client configuration

  • 18 July 2016
  • 9 replies
  • 68 views
alex_p
Badge +1

Hello FME users,

I am trying to configure the RabbitMQ JMS client for FME Server, without success. I copied the client files and dependencies directly in "FMEServer\\Server\\lib\\jms". Next, I configured a JMS subscriber to send message to my RabbibMQ Server :

- Protocol : JMS

- Provider Type Or Context : com.rabbitmq.jms.admin.RMQObjectFactory

- Provider URL : server-name:5672

- Connection Factory : com.rabbitmq.jms.admin.RMQConnectionFactory

Username and password : OK

Destination: a queue name

When I confirm my configuration the following error message appears : "Unable to find provider named com.rabbitmq.jms.admin.RMQObjectFactory. Ensure that the appropriate libraries are available and the correct provider is selected."

What I am missing?

Any help will be really appreciated

Thank you

Alex

icon

Best answer by johnnygalambos 9 December 2016, 18:29

View original

9 replies

M
Badge +2

@alexpillonel FME does not yet support Rabbit AMQP and therefore doesn't support RabbitMQ. I have added your post to the product change request we have to support RabbitMQ. Perhaps consider posting your request to the Ideas page to see if there are other users interested in RabbitMQ support

alex_p
Badge +1

I finally
had some success with my JMS RabbitMQ configuration on FME Server. Here the
results of my research :

RabbitMQ

For
information, I send message to the RabbitMQ server via an Exchange (ex-fme-flight-info) binded to a queue (qu-cep-flight-info) 

Configuration
on the server where FME Server is installed:

  1. I copied
         rabbitmq-jms client and its dependencies in the FME Server folder :
         "../Server/lib/jms"
  2. I configured a JNDI bindings
         file with the RabbitMQ configuration : C:\jndi\.bindings

Parts of the .binding file with the important configuration (where I
broke my head ;-)) :

#Factory parameters
ConnectionFactory/ClassName=javax.jms.QueueConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
#Destination queue
SendFlightQueue/ClassName=javax.jms.Queue
SendFlightQueue/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
SendFlightQueue/RefAddr/2/Content=ex-fme-flight-info
SendFlightQueue/RefAddr/2/Type=amqpExchangeName
SendFlightQueue/RefAddr/2/Encoding=String
…
… 
FME Server
  • I created a
         "Topic" : IODA_OUT_FlightCurrent
  • I created a
         "Subscription" with the protocol JMS and defined the
         "Destination" name with the name defined in the jndi binding
         files :


ProtocolJMSProvider Typecom.sun.jndi.fscontext.RefFSContextFactoryProvider URLfile:/c:/jndi/Connection FactoryConnectionFactoryDestinationSendFlightQueue
  • I created a FME Workspace
         with a "FMEServerNotifier" transformer configured to send data
         to the Topic previously created :

Unconverted
Try

I tried to send data
with the "JMSSender" transformer, but without success. FME engine
return the following error :

javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory]
Failed to connect to JMS provider with url "file:/c:/jndi/" and initial context "com.sun.jndi.fscontext.RefFSContextFactory"

 I hope it will help

mark2atsafe
Rank
Userlevel 4
Badge +26

I finally
had some success with my JMS RabbitMQ configuration on FME Server. Here the
results of my research :

RabbitMQ

For
information, I send message to the RabbitMQ server via an Exchange (ex-fme-flight-info) binded to a queue (qu-cep-flight-info) 

Configuration
on the server where FME Server is installed:

  1. I copied
         rabbitmq-jms client and its dependencies in the FME Server folder :
         "../Server/lib/jms"
  2. I configured a JNDI bindings
         file with the RabbitMQ configuration : C:\jndi\.bindings

Parts of the .binding file with the important configuration (where I
broke my head ;-)) :

#Factory parameters
ConnectionFactory/ClassName=javax.jms.QueueConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
#Destination queue
SendFlightQueue/ClassName=javax.jms.Queue
SendFlightQueue/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
SendFlightQueue/RefAddr/2/Content=ex-fme-flight-info
SendFlightQueue/RefAddr/2/Type=amqpExchangeName
SendFlightQueue/RefAddr/2/Encoding=String
…
… 
FME Server
  • I created a
         "Topic" : IODA_OUT_FlightCurrent
  • I created a
         "Subscription" with the protocol JMS and defined the
         "Destination" name with the name defined in the jndi binding
         files :


ProtocolJMSProvider Typecom.sun.jndi.fscontext.RefFSContextFactoryProvider URLfile:/c:/jndi/Connection FactoryConnectionFactoryDestinationSendFlightQueue
  • I created a FME Workspace
         with a "FMEServerNotifier" transformer configured to send data
         to the Topic previously created :

Unconverted
Try

I tried to send data
with the "JMSSender" transformer, but without success. FME engine
return the following error :

javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory]
Failed to connect to JMS provider with url "file:/c:/jndi/" and initial context "com.sun.jndi.fscontext.RefFSContextFactory"

 I hope it will help

Thanks for posting this information. Yes, I hope by having it here some other users will benefit from the knowledge.

 

 

johnnygalambos
Rank
Badge +1

Nice work, Alex! Your answer would have saved me a bunch of time--I must have missed it by a day or two. Just some additional details, I was able to get both the JMSSender and JMSReceiver working in a local Workspace as well as get FME Server Pub/Sub working. I've posted the .bindings file that works for us. 

For the local workspace, you need to make sure that the Jar containing com.sun.jndi.fscontext.RefFSContextFactory is somewhere in your class path.. I used fscontext-4.5-b25.jar from Maven here: https://mvnrepository.com/artifact/com.sun.messaging.mq/fscontext/4.5-b25. As a quick hack, I put this Jar as well as the RabbitMQ JMS jars into my default FME Java Plugins directory: C:\Users\<USERNAME>\Documents\FME\Plugins\Java. 

 

Configuring log4j on top of the slf4j was useful for debugging--the RabbitMQ client code does output some decent traces. I put a screenshot of all the jars I used to get this working below.

Bindings File  (in c:\RabbitMQBindings\.bindings for our config)

ConnectionFactory/ClassName=javax.jms.ConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory

#We are using a virtual host
ConnectionFactory/RefAddr/0/Content=fmetest
ConnectionFactory/RefAddr/0/Type=virtualHost
ConnectionFactory/RefAddr/0/Encoding=String
ConnectionFactory/RefAddr/1/Content=127.0.0.1
ConnectionFactory/RefAddr/1/Type=host
ConnectionFactory/RefAddr/1/Encoding=String


# JMS Queue where we have a RabbitMQ queue called "JMS2" boundto an exchange called "jms.durable.queues:.
# I named this very confusinginly, but FME will connect to this using the "JMS"
# destination. See screenshot below.
JMS/ClassName=javax.jms.Queue
JMS/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
JMS/RefAddr/0/Content=jms/Queue
JMS/RefAddr/0/Type=name
JMS/RefAddr/0/Encoding=String
JMS/RefAddr/1/Content=true
JMS/RefAddr/1/Type=amqp
JMS/RefAddr/1/Encoding=String
JMS/RefAddr/2/Content=JMS
JMS/RefAddr/2/Type=amqpRoutingKey
JMS/RefAddr/2/Encoding=String
JMS/RefAddr/3/Content=com.rabbitmq.jms.admin.RMQDestination
JMS/RefAddr/3/Type=type
JMS/RefAddr/3/Encoding=String
JMS/RefAddr/4/Content=jms.durable.queues
JMS/RefAddr/4/Type=amqpExchangeName
JMS/RefAddr/4/Encoding=String
JMS/RefAddr/5/Content=JMS2
JMS/RefAddr/5/Type=destinationName
JMS/RefAddr/5/Encoding=String
JMS/RefAddr/6/Content=JMS2
JMS/RefAddr/6/Type=amqpQueueName
JMS/RefAddr/6/Encoding=String

FME Workspace Configuration

0684Q00000ArJxAQAV.png

0684Q00000ArJqtQAF.png

0684Q00000ArJnnQAF.png

peter_s
Rank

Thanks for posting this!

I tried it out and had some success.

I found that to get the JMSReceiver-transformer to actually present a jms_content  I had to set the message header "JMSType" to  "TextMessage"

Example using python (pika) to post message with the header:

import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
msg_properties = pika.spec.BasicProperties(headers={'JMSType': 'TextMessage'})
channel.basic_publish(exchange='jms.durable.queues',
                        routing_key='JMS',
                        properties=msg_properties,
                        body='Hello World!')

I could not find a way to set this header using the JMSSender-transformer though so I don't know how to "eat my own dogfood" yet.

/Peter

steveatsafe
Rank
Userlevel 1
Badge +11

For the record, I was able to configure JMSSender and JMSReceiver to work in FME Desktop 2019.2 recently.

However, somewhere in the FME Server 2018.1 cycle the support for RabbitMQ and the JMSPublisher stopped working. We are hoping to resolve this in a later release of FME Server. Stay tuned.

Let us know - via an idea - if you are using or looking to use FME with RabbitMQ. Thanks.

 

SteveatSafe
steveatsafe
Rank
Userlevel 1
Badge +11

For the record, I was able to configure JMSSender and JMSReceiver to work in FME Desktop 2019.2 recently.

However, somewhere in the FME Server 2018.1 cycle the support for RabbitMQ and the JMSPublisher stopped working. We are hoping to resolve this in a later release of FME Server. Stay tuned.

Let us know - via an idea - if you are using or looking to use FME with RabbitMQ. Thanks.

 

I have found I can trigger topics in FME Server 2019 by setting the JMSType=TextMessage. If this isn't set in the header of the message, FME Server will not pass the message and thus won't trigger a related topic.

Here's an example of this in the RabbitMQ Web UI

SteveatSafe
steveatsafe
Rank
Userlevel 1
Badge +11

@alexpillonel FME does not yet support Rabbit AMQP and therefore doesn't support RabbitMQ. I have added your post to the product change request we have to support RabbitMQ. Perhaps consider posting your request to the Ideas page to see if there are other users interested in RabbitMQ support

Just a quick update. I recently had a Case related to this subject and it raised questions if FME was able to support RabbitMQ or not.

Back in 2016, this statement was true. However, some users were still able to configure FME and have success. More recently, I have been doing some testing and discovered that for a few releases of FME Server (2018.1, 2019.0, 2019.1) RabbitMQ messages were not triggering topics in FME Server. This was because of a change made.

For FME Server 2019.2 we resolved this. It still requires manual configuration but it is possible to get it to work. You'll need to provide the necessary RabbitMQ client libraries and be able to make sense of the .bindings file.

In summary, I have been able to configure FME Server to support RabbitMQ message types JMSType=TextMessage. I've also been able to use the JMSSender and JMSReciever in FME Desktop as well. This was all made possible with the help of @david_r when he discovered that things were not working in FME Server 2019.0/1 as they were before.

We have made changes to FME Server 2019.2 and even if JMSType is not set to TextMessage a notification topic will be triggered like it used to be in 2018.0 and older. Hope this helps.

If you have questions on this, please do reach out to us at support - add SteveatSafe to the message. I'm working on a KB article to share the configuration (similar to above) that was successful for me.

SteveatSafe
david_r
Rank
Userlevel 5

Just a quick update. I recently had a Case related to this subject and it raised questions if FME was able to support RabbitMQ or not.

Back in 2016, this statement was true. However, some users were still able to configure FME and have success. More recently, I have been doing some testing and discovered that for a few releases of FME Server (2018.1, 2019.0, 2019.1) RabbitMQ messages were not triggering topics in FME Server. This was because of a change made.

For FME Server 2019.2 we resolved this. It still requires manual configuration but it is possible to get it to work. You'll need to provide the necessary RabbitMQ client libraries and be able to make sense of the .bindings file.

In summary, I have been able to configure FME Server to support RabbitMQ message types JMSType=TextMessage. I've also been able to use the JMSSender and JMSReciever in FME Desktop as well. This was all made possible with the help of @david_r when he discovered that things were not working in FME Server 2019.0/1 as they were before.

We have made changes to FME Server 2019.2 and even if JMSType is not set to TextMessage a notification topic will be triggered like it used to be in 2018.0 and older. Hope this helps.

If you have questions on this, please do reach out to us at support - add SteveatSafe to the message. I'm working on a KB article to share the configuration (similar to above) that was successful for me.

I'm working on a KB article to share the configurationExcellent! :-)

Reply


Community Stats

28,869
Posts
109,985
Replies
36,672
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.