This article was reviewed in 2018.1 but applies to FME Server 2016 and above. For the FME Server 2015 version of this article please, see: Receiving email, Part 2: Processing email with a workspace (2015)
This tutorial is part of the series Email in FME Server.
Previous section: Receiving email, Part 1
Next section: Receiving email, Part 3
Introduction
This article is the second in a three-part tutorial series on receiving emails in FME Server. This section will cover how we can use a workspace published to FME Server via the Notification Service to process and log the details from the incoming email by subscribing to the Topic we created in Receiving email, Part 1.
Prerequisites
FME Server and FME Desktop installed
Completion of Receiving email, Part 1
Downloads
Instructions
Create the Workspace
We will set up FME Server to trigger a workspace to run in response to incoming messages to the RECEIVE_EMAIL topic. There are two workspace templates provided in the Downloads section of this article, respective of whether you set up an SMTP or IMAP Email Publication in the previous tutorial.
The workspace is designed to log all the information from the email as attributes so they can be viewed in the job log through the FME Server web interface.
You can either download the attached workspace template or create this .fmw file yourself using the following instructions.
- In a blank workspace, add a JSON reader to the canvas. For the dataset select the sample_email_notification_message.json from the Downloads zip file. This is a temporary JSON file that will be replaced by the incoming email.
- Connect a Logger to the JSON reader. There are two completed workspaces attached, one for SMTP and one for IMAP. The difference between these are the attribute prefixes of the incoming JSON file. Attributes of Emails received by the SMTP publisher are prefixed with email_ whilst the IMAP publisher prefixes the attributes with imap_.
Publish the Workspace
In order to be able to run this workspace using the notification service, we first need to publish it up to FME Server. When publishing the workspace, we have to set the topic to which it will be subscribed and specify which reader will receive the JSON file.
Go to File > Publish to FME Server. Connect to your server, and under the Register Services window check the Notification Service.
Publish the Workspace, registering it to the Notification Service
If you were to receive a new email now you should see this workspace listed under the Completed Jobs tab in the FME Server Web Interface.
Test the Configuration
To test if you have successfully set up the Subscription via registration to the Notification Service, simply send an email to the account you configured in the previous section of this tutorial series. As a reminder for SMTP this will by myusername@myserver.mydomain and for IMAP this is your own external mail account. This should cause the workspace to run, to confirm this you review Job History under Jobs > Completed.
Jobs triggered by the Notification Service list in the Completed Jobs page
Click on the Workspace Name to view the log. Review the log to find the list of attributes stored by the logger, including email subject, content, from and to address.
Example Log file
Email Subject Routing
If you need to take different actions based on the subject of the email - perhaps you are accepting files for different projects - you can filter and route emails to different sub-workspaces. This technique works well if you've set up regex-based subject filtering in the previous section.
If you would like to know more about this option, check out the other sample workspace provided in this tutorial which demonstrates how to use a TestFilter to route emails with three potential different project names in the subject line (FilterBySubject.fmwt).
Use the ‘Contains’ operation to test the email subject based on publisher filter
To set this up correctly, you will also have to return to your IMAP email publication in FME Server and set the Filter Type to Regex and then set the Subject Filter to:
project105|project481|project772
IMAP publication set filter type to regex
Test the Filter Configuration
To test whether the Filtering has been set up correctly we will send an email with the Email Subject ‘project105’.
Open the FilterBySubject.fmw and enable the FMEServerJobSubmitter for project105.
In the Parameter Editor select your FME Server connection. For the Repository select Samples and select the austinApartments.fmw.
Repeat the steps used above to publish the Workspace to FME Server under the Notification Service. Select RECEIVE_EMAIL as the Subscription Topic.
Send an email with the Subject project105
Return to the FME Server Web UI and go to the Completed Job History page to check both the FilterbySubject.fmw and the austinApartments.fmw workspace was run.
Email watch topic triggers FilterbySubject.fmw
If you send another email where the subject does not contain ‘project105’ then this will not pass on a topic message so no workspace will be run.
Conclusion
In this part of the tutorial, we created and published a workspace which processes emails. It simply logs the subject line of the email, but in the next part of the tutorial, we will expand it to process email attachments.
Troubleshooting
If your newly configured notification set up is not working as expected you can use FME Server's log files to help narrow down the problem. The diagram below shows how an email flows through each FME Server component and which log files are updated at each point. Check these messages and timestamps in these files to confirm that the email is reaching each component.
Flow chart of receiving email process with associated log files
Appendix
For SMTP the following attributes can be exposed in order to extract more information from the incoming email JSON:
email_publisher_attachment{} email_publisher_content_type{} email_publisher_content{} email_publisher_from email_publisher_to email_publisher_received email_publisher_sent email_publisher_subject fns_topic fns_type
For IMAP the following attributes can be exposed in order to extract more information from the incoming email JSON:
imap_publisher_attachment{} imap_publisher_content{} imap_publisher_sent imap_publisher_from imap_publisher_subject imap_publisher_content_type{} imap_publisher_received imap_publisher_to fns_type
Note: the "_publisher_received" attributes are not formatted in FME Date/Time format. Included in the download section above is a workspace that contains an example of how to convert it to the standard format. This should make this information easier to use with other transformers.
