• EN
Question

FME Challenge: 12 Days of FME Hackathon

  • 30 November 2017
  • 79 replies
  • 11 views
fmelizard
Rank
Userlevel 4
Badge +13

Voting is now open! This question is closed for submissions, but you can vote for your favourites! Voting is open until Monday Dec 18th at 12:00pm PST.What is the 12 Days of FME Hackathon?

Recently at Safe we had an internal event where the FME Experts had the opportunity to spend a couple of days away from their normal jobs, and build whatever they like with FME. We named this event ‘Demo Daze’, and you can find out what we got up to in this blog post.

So, inspired by Demo Daze, we decided to extend the opportunity to our FME community, for the chance to win a one-of-a-kind, holiday inspired, FME prize pack.

There will also be reputation points and a Knowledge Center badge up for grabs - 100 points for the winner and 20 points for each contestant.

We’re giving FME users 12 days to work on and submit a project and share it with the rest of the community. Perhaps you’ll build something holiday inspired, give the 2018 beta a whirl or put that home licence to good use!

To enter the 12 Days of FME Hackathon, answer this question with:

a) A title.

b) A brief description. For example, explain why you chose this project, what it does, and how FME was used to create the project.

c) A file containing your project demo or result. This could include a video, image, FME Hub submission, or workspace.

This question will be closed at 12pm PST Tuesday 12th December, where the answers will be available to vote on but submissions will no longer be allowed.

The winner will be the submission that has the most votes at on Monday Dec 18th at 12:00pm PST.

VOTES ON ANSWERS BEFORE THE VOTING PERIOD WILL NOT COUNT

(Please don’t vote until then, as they will be removed)

T&Cs; apply

79 replies

G
Badge +3

Do the fme ultra slims feature a string searcher ??? :0 ....

dfresh
Badge

Creating an Object Detector for identifying Donald Trumps Hair.

Description: Using FME to prep and build a dataset for training an Object Detector to recognize Donald Trumps Hair. FME will create inputs for tensorflow/machine learning for object detection using an Single Shot Multibox Detector (SSD) with Inception V2 model.

I will post all of the files once it is completed, within the next couple of days.

Cheers,

Doug

runneals
Rank
Badge +14

Midwest Region Road Conditions

Description: Many State DOT Traveler Information websites believe that people only drive in their state and don't provide inter-state information in regards to road conditions or traffic incidents. These workspaces were put together to show how useful it is to have data from surrounding states in one common application. Luckily many of the states that I pulled have an easy to use API (and more importantly people that were willing to provide access to their APIs) that I was able to easily integrate. Obtaining access to the data feeds is a struggle in today's world of people and agencies wanting to keep their data closed by not sharing or requiring that developers go through a ridiculous complex process of obtaining a token key.

Workspaces of how I pull the data can be found here (minus some of the credentials/token keys that can be easily obtained).

Live Feature Collection or Live Feature Service (Please use these production quality feeds for inspiration or integration into other projects!)

See the layer live on our TrackaPlow application (powered by ArcGIS Online services that are updated every 3-5 minutes with FME Server!)

Finally a shout out to the SAFE Software staff for developing this awesome contest. Hopefully it will become an annual contest/tradition and encourage further sharing of interesting new ideas - and workspaces!

jdh
Rank
Badge +22

PolygonEliminator

 

A Custom Transformer that eliminates candidate polygons by dissolving them into an adjacent base polygon. The choice of adjoining polygon is based on area (largest or smallest) or edge length (longest or shortest) that does not cross a breakline. With optional fuzzy tolerance and group-by this transformer gives more fine grained control than the SliverRemover, but does not eliminate gaps.

Pro Tip: You can create candidate polygons for gaps by using the AreaOnAreaOverlayer with the input data+ BoundingBoxAccumulator.

 

polygoneliminator.fmx

jdh
Rank
Badge +22

PolygonEliminator

 

A Custom Transformer that eliminates candidate polygons by dissolving them into an adjacent base polygon. The choice of adjoining polygon is based on area (largest or smallest) or edge length (longest or shortest) that does not cross a breakline. With optional fuzzy tolerance and group-by this transformer gives more fine grained control than the SliverRemover, but does not eliminate gaps.

Pro Tip: You can create candidate polygons for gaps by using the AreaOnAreaOverlayer with the input data+ BoundingBoxAccumulator.

 

polygoneliminator.fmx

This is just for fun, I'm not eligible for the contest due to residency.

 

 

A
Badge +1

GeoGeek Xmas Shopping List Generator

Description: Input a space delimited list of the GeoGeeks you need to shop for and get a handy shopping list that tells you if they should get Beer, Wine, FME Socks or Coal. (Hey who really needs anything other than that!) Uses a random number generator and attribute manager to conditionally map the number to a gift.whatdoigetmyfavgeogeek4xmas.fmw

davideagle
Rank
Badge +14

A title. The FlexNetAnalyser.

A brief description. "On the 12th day of Christmas my true love gave to meeee... a divorce for spending too much time with FME."

Every now and again I'm asked to help someone work out what their Esri, FME or Autodesk license utilisation is. Or, and this is usually with a level of urgency in their voice they want to know, "who's hogging MY floating license right now!". I've several times built workspaces that were variations on this theme but always resolved to build something a little more generic, something that could get detail out of several types of license server and answer these questions in a more structured way. So, it seems like a Christmas hackathon is the best time to have a quick crack(er) at it...So, the FlexNetAnalyser is step 1 along the road to a Custom Transformer FME Hub submission. In the new year and with a bit of feedback from the community I'll whip it into shape for 2018.

The FlexNetAnalyser is able to analyse a FlexNet license server (FME, Esri, Autodesk etc) and will output the current users who have a license checked out and it will also highlight the total license count checked out per license type.

Output Ports:

LicenseTypeCount - A tabular structure designed to be output on a schedule at a regular interval to a database or spreadsheet so that analysis can be done on license use peaks and troughs, perhaps using structured queries or more likely with some form of dashboard. In the structure:

date, time, fme, oracle

20171207, 10:21, 1, 1

CurrentLicenseUsers - One feature per user with a license checked out, designed to be sent to a dashboard or HTML page so that the individual or machine blocking a license can be identified (and perhaps neutralised, as humanely as possible). In the structure:

license, allocated, host, user

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:32, SLEIGH1, Dancer

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:01, SLEIGH2, Rudolf

Both output ports include schema features so that they can be hooked up to a Dynamic FeatureType and your work is done. Alternatively the AttributeExposer will need to be used after the LicenseTypeCount port in order to expose and then manipulate the license type count columns based on the FlexNet license server that you are analysing. For example for an FME license server it would be fme, oracle etc and for an Esri license server, editor, publisher, viewer, arc/info etc.

Blatant heart string tug for votes:

Merry Christmas from the FlexNetAnalyser Puppy!

A file containing your project demo or result. flexnetanalyser.fmx

C
Badge

a) A title. FME blog scraper.

b) A brief description. This workspace was created to scan FME blog and extract metadata, like author, date, category, post url and text preview. The result will be saved to Excel file.

c) A file containing your project demo or result. fme-blog-reader.fmwt

P
Badge +4

Creating an Object Detector for identifying Donald Trumps Hair.

Description: Using FME to prep and build a dataset for training an Object Detector to recognize Donald Trumps Hair. FME will create inputs for tensorflow/machine learning for object detection using an Single Shot Multibox Detector (SSD) with Inception V2 model.

I will post all of the files once it is completed, within the next couple of days.

Cheers,

Doug

I would love to see the result!

 

 

redgeographics
Rank
Userlevel 5
Badge +26

Title:

 

The ColorConverter custom transformer.

Description:

 

Sometimes it's necessary to convert colors from one notation to another and sometimes it's necessary to do that for a lot of colors at a time (like that one time where I had to transform a bunch of CMYK colors from a printed map into Web RGB to use them in a SLD). I've built a custom transformer that handles this. Currently it supports the following color models:
  • FME (RGB triplet in range 0..1)
  • RGB (RGB triplet in range 0..255
  • Web RGB (a 6-character hexadecimal string containing an RGB triplet in range 00..FF)
  • CMYK (in ranges 0..1 and 0..100)

With that I can confirm that my Evil Duckie is 1,0,0 or 255,0,0 or #FF0000 or 0,1,1,0 or 0,100,100,0

(since I can't compete with @1spatialdave's puppy...)

File containing my submission:

 

https://hub.safe.com/transformers/colorconverter

Note that especially the conversion from RGB to CMYK color space is a bit of a dark art since the two color models don't actually cover the same colors. Under the hood it all gets converted to an FME color first before heading out to the desired output format (much like how FME works), a to-do item would be for any conversion between the two CMYK flavors to skip that step. I would also like to add HSV and HSL or even Pantone.

redgeographics
Rank
Userlevel 5
Badge +26

Title:

 

The ColorConverter custom transformer.

Description:

 

Sometimes it's necessary to convert colors from one notation to another and sometimes it's necessary to do that for a lot of colors at a time (like that one time where I had to transform a bunch of CMYK colors from a printed map into Web RGB to use them in a SLD). I've built a custom transformer that handles this. Currently it supports the following color models:
  • FME (RGB triplet in range 0..1)
  • RGB (RGB triplet in range 0..255
  • Web RGB (a 6-character hexadecimal string containing an RGB triplet in range 00..FF)
  • CMYK (in ranges 0..1 and 0..100)

With that I can confirm that my Evil Duckie is 1,0,0 or 255,0,0 or #FF0000 or 0,1,1,0 or 0,100,100,0

(since I can't compete with @1spatialdave's puppy...)

File containing my submission:

 

https://hub.safe.com/transformers/colorconverter

Note that especially the conversion from RGB to CMYK color space is a bit of a dark art since the two color models don't actually cover the same colors. Under the hood it all gets converted to an FME color first before heading out to the desired output format (much like how FME works), a to-do item would be for any conversion between the two CMYK flavors to skip that step. I would also like to add HSV and HSL or even Pantone.

K
Badge +5

Title:

 

The VertexAngleCalculator custom Transformer

Brief description:

 

A Transformer that, given an Area or Line feature, calculates the directional change at each vertex. Output is available as both a measure on the original feature, or as seperate vertices containing the angles as attributes.

A file containing your project demo or result:

 

An entry on the FME Hub.

I love doing these kinds of calculations on geometry, and I had already researched the required steps once before. Seems rude not to try it in FME and to share it with others. I originally needed the angles to determine whether a polygon was clockwise or counterclockwise.

Since I can't compete with a puppy, here's a typical Dutch winter scenery of enthousiastic folk going to work:

K
Badge +5

Title:

 

The VertexAngleCalculator custom Transformer

Brief description:

 

A Transformer that, given an Area or Line feature, calculates the directional change at each vertex. Output is available as both a measure on the original feature, or as seperate vertices containing the angles as attributes.

A file containing your project demo or result:

 

An entry on the FME Hub.

I love doing these kinds of calculations on geometry, and I had already researched the required steps once before. Seems rude not to try it in FME and to share it with others. I originally needed the angles to determine whether a polygon was clockwise or counterclockwise.

Since I can't compete with a puppy, here's a typical Dutch winter scenery of enthousiastic folk going to work:

takashi
Rank
Userlevel 3
Badge +17

A title.

Spatial Search and Clipping on Terrain Model Stored in DB

A brief description.

This is a part of a big project whose goal is to develop a database system to store a bunch of 3D datasets including terrain surface models, BIMs, and point clouds. Regarding terrain models, the major requirements are:

  1. The system should allow the user to perform spatial search any part of stored surfaces.
  2. The system should clip the found surface by an area which the user specified for searching.
  3. The original surface could have a texture appearance, and the system should restore the texture on the clipped surface.

A file containing my project demo.

The attachment contains two workspaces and a sample data, which demonstrate how FME works to handle terrain models according to the requirements. The workspace "load.fmw" loads the sample data to a SpatialLite database, and the workspace "extract.fmw" performs spatial search and clipping. Firstly run the "load.fmw" to create the SpatiaLite database before running "extract.fmw".

load-and-extract-terrain-surface.zip (created with FME 2017.1.1.1)

Left: Sample Data, Right: Spatial Search and Clipping Result

takashi
Rank
Userlevel 3
Badge +17

A title.

Spatial Search and Clipping on Terrain Model Stored in DB

A brief description.

This is a part of a big project whose goal is to develop a database system to store a bunch of 3D datasets including terrain surface models, BIMs, and point clouds. Regarding terrain models, the major requirements are:

  1. The system should allow the user to perform spatial search any part of stored surfaces.
  2. The system should clip the found surface by an area which the user specified for searching.
  3. The original surface could have a texture appearance, and the system should restore the texture on the clipped surface.

A file containing my project demo.

The attachment contains two workspaces and a sample data, which demonstrate how FME works to handle terrain models according to the requirements. The workspace "load.fmw" loads the sample data to a SpatialLite database, and the workspace "extract.fmw" performs spatial search and clipping. Firstly run the "load.fmw" to create the SpatiaLite database before running "extract.fmw".

load-and-extract-terrain-surface.zip (created with FME 2017.1.1.1)

Left: Sample Data, Right: Spatial Search and Clipping Result

australia
Badge +1

Title: Create map with index

Description: The ‘IndexMap’ Transformer was developed to automate the process of producing a map with an associated Index. This workspace work with points, lines and polygon features and can be used in any hemisphere in the world - NE, NW, SE, SW.

There are two links on the FME HUB

 

Canada (Vancouver Parks)

 

Associated Index

Australia (Canberra Roads)

 

Associated Index

 

Output to Google Earth

Regards

Greg Patterson

mark2atsafe
Rank
Userlevel 4
Badge +25
Not long now before the closing date. Don’t get so absorbed in your project that you miss the deadline!

 

 

erik_jan
Rank
Userlevel 2
Badge +16

I love this idea, but at this moment I don't have the time to participate.

Maybe next year.

jdh
Rank
Badge +22

A title. The FlexNetAnalyser.

A brief description.  "On the 12th day of Christmas my true love gave to meeee... a divorce for spending too much time with FME."

Every now and again I'm asked to help someone work out what their Esri, FME or Autodesk license utilisation is. Or, and this is usually with a level of urgency in their voice they want to know, "who's hogging MY floating license right now!". I've several times built workspaces that were variations on this theme but always resolved to build something a little more generic, something that could get detail out of several types of license server and answer these questions in a more structured way. So, it seems like a Christmas hackathon is the best time to have a quick crack(er) at it...So, the FlexNetAnalyser is step 1 along the road to a Custom Transformer FME Hub submission. In the new year and with a bit of feedback from the community I'll whip it into shape for 2018.

The FlexNetAnalyser is able to analyse a FlexNet license server (FME, Esri, Autodesk etc) and will output the current users who have a license checked out and it will also highlight the total license count checked out per license type.

Output Ports:

LicenseTypeCount - A tabular structure designed to be output on a schedule at a regular interval to a database or spreadsheet so that analysis can be done on license use peaks and troughs, perhaps using structured queries or more likely with some form of dashboard. In the structure:

date, time, fme, oracle

20171207, 10:21, 1, 1

CurrentLicenseUsers - One feature per user with a license checked out, designed to be sent to a dashboard or HTML page so that the individual or machine blocking a license can be identified (and perhaps neutralised, as humanely as possible). In the structure:

license, allocated, host, user

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:32, SLEIGH1, Dancer

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:01, SLEIGH2, Rudolf

Both output ports include schema features so that they can be hooked up to a Dynamic FeatureType and your work is done. Alternatively the AttributeExposer will need to be used after the LicenseTypeCount port in order to expose and then manipulate the license type count columns based on the FlexNet license server that you are analysing. For example for an FME license server it would be fme, oracle etc and for an Esri license server, editor, publisher, viewer, arc/info etc.

Blatant heart string tug for votes:

0684Q00000ArLStQAN.jpg

Merry Christmas from the FlexNetAnalyser Puppy!

A file containing your project demo or result. flexnetanalyser.fmx

Interesting.

 

I recommend better error catching if there is an invalid port.  Currently it is

 

2017-12-11 15:09:53| 0.6| 0.0|WARN |Python Exception <TypeError>: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|WARN |Traceback (most recent call last):
File "<string>", line 98, in close
File "<string>", line 23, in throw
TypeError: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|FATAL |f_70(PythonFactory): PythonFactory failed to close properly
2017-12-11 15:09:53| 0.6| 0.0|ERROR |f_70(PythonFactory): A fatal error has occurred. Check the logfile above for details

 

I would also be very interested in checking all the licences on a specific server in one shot, or at least all the licences for a specific version of flexlm.  (We have 6 software requiring 3 versions of flexlm on the same license server)

 

jneujens
Badge

A title. The FlexNetAnalyser.

A brief description. "On the 12th day of Christmas my true love gave to meeee... a divorce for spending too much time with FME."

Every now and again I'm asked to help someone work out what their Esri, FME or Autodesk license utilisation is. Or, and this is usually with a level of urgency in their voice they want to know, "who's hogging MY floating license right now!". I've several times built workspaces that were variations on this theme but always resolved to build something a little more generic, something that could get detail out of several types of license server and answer these questions in a more structured way. So, it seems like a Christmas hackathon is the best time to have a quick crack(er) at it...So, the FlexNetAnalyser is step 1 along the road to a Custom Transformer FME Hub submission. In the new year and with a bit of feedback from the community I'll whip it into shape for 2018.

The FlexNetAnalyser is able to analyse a FlexNet license server (FME, Esri, Autodesk etc) and will output the current users who have a license checked out and it will also highlight the total license count checked out per license type.

Output Ports:

LicenseTypeCount - A tabular structure designed to be output on a schedule at a regular interval to a database or spreadsheet so that analysis can be done on license use peaks and troughs, perhaps using structured queries or more likely with some form of dashboard. In the structure:

date, time, fme, oracle

20171207, 10:21, 1, 1

CurrentLicenseUsers - One feature per user with a license checked out, designed to be sent to a dashboard or HTML page so that the individual or machine blocking a license can be identified (and perhaps neutralised, as humanely as possible). In the structure:

license, allocated, host, user

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:32, SLEIGH1, Dancer

"FME" v1.0, vendor: safe, expiry: 30-sep-2018, Start Thu 12/7 22:01, SLEIGH2, Rudolf

Both output ports include schema features so that they can be hooked up to a Dynamic FeatureType and your work is done. Alternatively the AttributeExposer will need to be used after the LicenseTypeCount port in order to expose and then manipulate the license type count columns based on the FlexNet license server that you are analysing. For example for an FME license server it would be fme, oracle etc and for an Esri license server, editor, publisher, viewer, arc/info etc.

Blatant heart string tug for votes:

Merry Christmas from the FlexNetAnalyser Puppy!

A file containing your project demo or result. flexnetanalyser.fmx

The workspace is password-protected?
david_r
Rank
Userlevel 5
The 12 days of xmas generator

Workspace that generates the lyrics for the song and uses the system speech synthesizer to "sing" it out on your loudspeaker. A guaranteed hit at the family dinner!

Workspace: 12daysofxmas.fmw

Output: 12 day of xmas.mp3 (Dropbox link, the forum didn't like my mp3)

Requirements:

If you don't want to install anything and just want to play around, simply disable the PythonCaller at the end.

Ho ho ho and happy holidays!

david_r
Rank
Userlevel 5
The 12 days of xmas generator

Workspace that generates the lyrics for the song and uses the system speech synthesizer to "sing" it out on your loudspeaker. A guaranteed hit at the family dinner!

Workspace: 12daysofxmas.fmw

Output: 12 day of xmas.mp3 (Dropbox link, the forum didn't like my mp3)

Requirements:

If you don't want to install anything and just want to play around, simply disable the PythonCaller at the end.

Ho ho ho and happy holidays!

srg
Badge +10

Title: FME Santa Nose based Air Quality Index for Chennai, India

Description:

If you ever thought that Santa Claus was old fashioned and not tech savvy?

You got it wrong, at least this Christmas season and not atleast the FME powered Santa Claus.

With the nose as his IoT device (real time air quality sensor) doubling up as an indicator, this FME Santa Claus is all geared up to indicate in real time, if it is safe to wander around the city this Christmas season.

SaintGIS has been helping FME Santa learn a bit of FME to perform a trick or two this season, ever since the FME hackathon was announced.

FME Santa has learnt:

1. To consume real-time air quality data from the federal government open data portal web services api in xml format

2. Source: https://data.gov.in/resources/real-time-air-quality-index-various-locations

3. Filter the data for places of interest he will be visiting.

4. Flatten the xml and generate real time reports on air quality

Well these html reports are unique to FME Santa, for it has his magical touch.

Yes, the reports for a key location ( https://goo.gl/maps/jEX7YvijHjG2 ) in the city of Chennai, India; not only contain real time tabular data on the prominent pollutants but also a colour changing image of FME Santa to graphically indicate the real time air quality of that place.

FME Santa's own nose and beard are the IoT sensors and displays.

They change colours to display the air quality in real time.

Very nice of FME Santa to have lent his nose for a noble social and environmental cause.

FME Santa wants to add a word for improvement that the html report can be made dynamic by scheduling the workspace using FME Server.

Attached is the FME Workbench template, a sample output html report and the screen grab of the html report

iot-fme-santa-sensornose-srg.fmwt

santaclaus-iot-nosesensor-realtimeairqualityindex.htm

david_r
Rank
Userlevel 5

Title: FME Santa Nose based Air Quality Index for Chennai, India

Description:

If you ever thought that Santa Claus was old fashioned and not tech savvy?

You got it wrong, at least this Christmas season and not atleast the FME powered Santa Claus.

With the nose as his IoT device (real time air quality sensor) doubling up as an indicator, this FME Santa Claus is all geared up to indicate in real time, if it is safe to wander around the city this Christmas season.

SaintGIS has been helping FME Santa learn a bit of FME to perform a trick or two this season, ever since the FME hackathon was announced.

FME Santa has learnt:

1. To consume real-time air quality data from the federal government open data portal web services api in xml format

2. Source: https://data.gov.in/resources/real-time-air-quality-index-various-locations

3. Filter the data for places of interest he will be visiting.

4. Flatten the xml and generate real time reports on air quality

Well these html reports are unique to FME Santa, for it has his magical touch.

Yes, the reports for a key location ( https://goo.gl/maps/jEX7YvijHjG2 ) in the city of Chennai, India; not only contain real time tabular data on the prominent pollutants but also a colour changing image of FME Santa to graphically indicate the real time air quality of that place.

FME Santa's own nose and beard are the IoT sensors and displays.

They change colours to display the air quality in real time.

Very nice of FME Santa to have lent his nose for a noble social and environmental cause.

FME Santa wants to add a word for improvement that the html report can be made dynamic by scheduling the workspace using FME Server.

Attached is the FME Workbench template, a sample output html report and the screen grab of the html report

iot-fme-santa-sensornose-srg.fmwt

santaclaus-iot-nosesensor-realtimeairqualityindex.htm

Using Santa's nose as an indicator: priceless!
M
Badge

Santa's trip around the world!

This Workbench connect to the API of Openstreet Map.

- It queries all the Christmas related features of OSM input by users like me!

- It creates a path between all the trees

- It creates a Web page to display the result using Leaflet

xmas.fmw

Reply


Community Stats

28,799
Posts
109,770
Replies
36,617
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.