The tutorial will work for FME Server 2016+ and has last been edited for FME Server 2018. For earlier versions of FME Server, please refer to this article.

Introduction

Resource folders are a convenient storage area for uploading and storing data, and data being stored in a resources folder can be used when publishing or running workspaces. In addition, existing directories (such as UNC shares) can be mounted and used when publishing from FME Workbench, or browsed when submitting a job through the web interface.

Please download the tutorial data before starting the examples below.

Downloads

required-data.zip

Example 1: Upload Data Via the FME Server Web Interface

This example will demonstrate how to upload data to a Resources folder on FME Server through the web interface, and then utilize the data when running a job from a published workspace

(1) Upload data to FME Server through the web interface

• Open the FME Server Web Interface > Resources
• Select the Data folder > click Create Folder > enter your first name
• Select the new folder and click Upload
• Click Choose Folder > browse to the downloaded tutorial data and select CityData.gdb

FME Server Web Interface Resources page

(2) Generate a simple workspace

• Open Workbench > File > New > Generate Workspace
• Source: ESRI Geodatabase (File Geodb Open API), TutorialDownloadPath/data/CityData.gdb
• Open the Parameters and select only CITY_GRID from the Table List
• Destination: ESRI Geodatabase (File Geodb Open API)

Generate Workspace wizard

• Run the workspace to ensure that it is working correctly
• File > Publish to FME Server > create new repository named "Resources Tutorial" > Use workspace name: example1.fmw (Alternatively you can use the Publish icon at the far end of the ribbon to get to this wizard )

Create a New Repository in the Publish to FME Server wizard

• Under the Register Services window, you can leave the Job Submitter service checked but also select the Data Download service.

(3) Browse and use data uploaded to FME Server

• Open the FME Server Web Interface. On the homepage under Last Published Workspaces, select the workspace you just published and this will bring up the Run Workspace page
• Under Published Parameters click the ellipse ‘...’ next to File Geodatabase > Resources > Data > your first name > CityData.gdb
• Select OK and Run Workspace

Example 2: Upload Data Via FME Workbench

This example will demonstrate how to upload data to a Resources folder when first publishing a workspace from FME Workbench using the ‘select files’ option. This data then becomes available for use at any time by other published workspaces.

(1) Generate a simple workspace

• Open Workbench > File > New > Generate workspace
• Source: MapInfo TAB (MITAB), <TutorialDownloadPath>/data/parks.tab
• Destination: MapInfo TAB (MITAB), data folder
• Run the workspace to test to see that it is working correctly

(2) Publish a workspace and upload data

• File > Publish to FME Server > Repository: Resources Tutorial > Use workspace name: example2.fmw
• Click "Select Files..."
• Check the ‘Files’ checkbox to select all the files
• Click "Select Location..."
• Check "Upload to a shared resource folder"
• Click Location… > Data > first name.
• Register with the Job Submitter service and finish publishing

Select location for files to be uploaded to when publishing a workspace

• Return to FME Server and run example2.fmw
• Under the Published Parameters notice this time the resource folder is already specified for the Source MapInfo TAB file
• Now that the data has been uploaded into the Resources folder you can reference it in any future Workspace that requires the Parks.tab dataset.

Example 3: Upload Data Via the FME Server System Share

This example demonstrates how to upload data into the Resources folder without logging on to the FME Server Web Interface by looking directly into the operating system’s file system.

(1) Copy data into the Resources Folder within the File Directory

• Open the File Directory and navigate to the data folder you downloaded at the beginning of this article. Right click on the data folder and select Copy.
• Now return to the Local Disk drive and select View and check the box for Hidden Items

Locate Hidden Items in file directory

• The folder ProgramData has now been revealed - this is where the FME System Share is located and is home to the Resources folders. Navigate to ProgramData > Safe Software > FME Server > resources > data.
• If you already completed the previous examples there will already be a folder under your name located here, if not then now you can Paste the data folder that we just copied and rename it to <your name>. When you return to the Web Interface, this data is now available.

Uploading a folder to FME Server Resources through FME System Share

Example 4: Using a Resource Folder Within a Workspace

This example demonstrates how to create a macro that contains a resource location and then utilize it within an FME Workspace. Note: if you see an error message such as "Failed to open...", you may need to change the user account that the FME Server Engines service is using. See guide here.

(1) Create a Macro allowing direct access to the Resource

• On the FME Server machine, open the /Server/ fmeEngineConfig.txt file as an administrator

• Find MACRO_DEF FME_SHAREDRESOURCE_BACKUP "C:/ProgramData/Safe Software/FME Server///resources/backups/"

• add a \ character to the end of the line:

MACRO_DEF FME_SHAREDRESOURCE_BACKUP "C:/ProgramData/Safe Software/FME Server///resources/backups/"\

• Press return and on a new line, add the following line, replacing <hostname> with your machine’s hostname:

MACRO_DEF FME_SHAREDRESOURCE_MYDATA "//<hostname>/data/"

Add a macro in fmeEngineConfig.txt file

• Save the file
• Restart FME Server under the FME Server folder in the Start Menu

(2) Add the Resource Folder as a Network Location

• Return to the FME Server Web Interface and under the Resources page select New.
• All the parameters require an input. Set the Name/Display Name and Description and then for Type select ‘Network based resource'. For the directory add \\<hostname>\data\ as in the previous step.

Create a new resource connection in the FME Server Web Interface

• Once you have pressed OK this folder will appear as a resource - if you check the box to the side of it and select Actions > Properties the System Path is now shown as the macro we just defined.
• Detailed instructions on how to add a resource connection either from an Amazon S3 Bucket or through a network based resource can be found here: Creating and Managing Resource Connections

(3) Testing the Macro

• Open example2.fmw in FME Workbench
• In the Navigator panel create a new private parameter setting the values using the image below ensuring the Parameter Value is set to $(FME_SHAREDRESOURCE_MYDATA)Parks.tab

Create a private parameter

• Now expose the Reader and change the Source parameter from its current setting to the private parameter we just created

Assign the private parameter as the reader source value

• Publish the workspace to FME Server under the Job Submitter service. Since this Workspace has already been published you may be able to republish it to save repeating the entire publishing process.
• Return to the Web Interface and Run Workspace example2.fmw. This time you will notice that there is no option to point FME Server to a Source file as this is no longer a parameter accessible to the User.

4a: Read Data within a Resource Folder

This example demonstrates reading data, using a macro to specify the location.

(1) Add the Macro to a simple workspace reader

• Open example2.fmw
• Expand the MITAB Reader in the Navigator panel and click on the Source Parameter
• Remove Source Parameter and add the Resource Macro created in the example above:

$(FME_SHAREDRESOURCE_MYDATA)Parks.tab

Use a macro to read data saved in FME Server Resources

• Instead of typing out the Resource macro you can also browse the Resources available on FME Server by looking exposing the drop down tab next to Dataset in the Add Reader Wizard and selecting ‘Select File From Web’

Select an FME Server Resource using the Select File From Web option

(2) Publish and run the workspace

• File > Publish to FME Server > Repository: Resources Tutorial > Use workspace name: example3.fmw > Register Services: Job Submitter > Publish
• Open the Web Interface. From the Last Published Workspaces, navigate to and select example3.fmw. Run the workspace.

If you have already uploaded data to a different folder in FME Server > Resources using one of the other examples in this article then you can view the macro to use in FME Workbench by hovering your mouse over the folder or selecting the folder and then Actions > Properties. However by rule of thumb they follow the pattern FME_SHAREDRESOURCES_<folder name>.

4b: Write Data to a Resource Folder

This example demonstrates writing data, using a macro to specify file location.

(1) Add the Macro to a simple workspace writer

• Open Workbench > File > New > Generate workspace
• Source: MapInfo TAB (MITAB), <TutorialDownloadPath>/data/parks.tab
• Destination: Esri Shape (SHAPE), (FME_SHAREDRESOURCE_MYDATA)Parks

Generate a workspace with the output written directly to FME Server

(2) Publish and run the workspace

• Publish the Workspace to FME Server making sure that the Upload data files option is left unchecked.
• Return to the FME Server Web Interface and select the newly published Workspace to Run.
• Upon completion return to Resources > MYDATA and now you should find a new folder has been added containing the Park shapefiles.

Folder created once the workspace was run containing the new shapefiles

4c: Use a Resource Folder File in a Parameter

This example demonstrates how to utilize a macro in order to access files in a Resource folder. This method can be used to allow files to be used in reader and writer parameters. For example seed, template or SDE connection files can be specified and shared between workspaces.

(1) Add the Macro to a simple workspace writer

• Open Workbench > File > New > Generate Workspace
• Source: MapInfo TAB (MITAB), TutorialDownloadPath/data/parks.tab
• Destination: AutoCAD Map 3D (AUTOCAD_OD)
• Create a new Private Parameter;
• Type: Text Name: Template_File Prompt: My Template File: Value: $(FME_SHAREDRESOURCE_MYDATA)template.dwt

Add a private parameter with the value connected to an FME Server Resource

• Expand the Parks [MITAB] Writer and right-click the "Template File" Parameter and link it to the Template_File parameter we just created

Link the private parameter to the reader template file parameter

(2) Publish and run the workspace

• Publish the workspace to FME Server and then return to the Web Interface to run it.
• In Resources > MYDATA the Parks.dwg has been created - to view this file you can select and download it and open it in the FME Data Inspector.
• When creating the Template File user parameter if you were to leave the published box checked the template file location could be changed by users in FME Server.