point to line to point, retain original attributes

@notmyname, this is an interesting subject. As my self-training, defined a Python script to perform the processing efficiently. I'd share it with you, but note it has not tested thoroughly. Also this is a test for posting codes in this new community site. Hope the script will be shown correctly ;)

Test result: Bad, unfortunately.
First several lines of the script are not shown in the code block.
< (less than) and > (greater than) in the script have been changed to their HTML entity ("&lt;", "&gt;").

# Python Script Example (PythonCaller)
# Filters out spike vertices from input points and transforms remnants into lines without spikes.
# Assuming that every input feature has a point geometry,
# the features have a group ID attribute called "groupID" (positive integer),
# the maximum spike angle is given by a user parameter called "SPIKEANGLE" (positive numeric),
# and the order of input features is sorted by the group ID and connecting order.
# Determination on whether a point is spike vertex will be performed by only the angle.
# If you need to consider the spike length as well, modify and add some codes. 
# You can classify output features into points and lines using the GeometryFilter.
import fme, fmeobjects, math

class FeatureProcessor(object):
    def __init__(self):
        self.features, self.coords = [], [] # storage for input features and their coordinates
        self.groupID = -1
        self.spikeAngle = float(FME_MacroValues['SPIKEANGLE'])
        
    def input(self, feature):
        # Retrieve group ID and coordinates from the input feature.
        gid = int(feature.getAttribute('groupID'))
        coord = feature.getCoordinate(0)
        
        # If the feature is the first one of a group, create and output line for the previous group,
        # clear the lists of features and coordinates, and update current group ID. 
        if gid != self.groupID:
            self.outputLine()
            self.features, self.coords = [], []
            self.groupID = gid

        # If the number of stored features is greater than 1,
        # determine wheter the last feature (point) in the lists is spike vertex or not.
        if 1 < len(self.features):
            x0, y0 = self.coords[-2][0], self.coords[-2][1] # second last point
            x1, y1 = self.coords[-1][0], self.coords[-1][1] # last point (candidate of spike vertex)
            x2, y2 = coord[0], coord[1] # current point (input feature)
            ax, ay = (x1 - x0), (y1 - y0) # vector: second last -> last
            bx, by = (x2 - x1), (y2 - y1) # vector: last -> current
            
            # Calculate the absolute angle formed by the two vectors.
            # If the angle is less than or equal to specified maximum spike angle,
            # output the last feature (point: x1,y1) and remove it from the lists.
            angle = 180 - abs(math.degrees(math.atan2(ax * by - ay * bx, ax * bx + ay * by)))
            if angle <= self.spikeAngle:
                self.pyoutput(self.features[-1]) # output the last feature (spike vertex)
                self.features.pop()
                self.coords.pop()
        
        # Append the input feature and its coordinates to the lists.
        self.features.append(feature)
        self.coords.append(coord)
            
    def outputLine(self):
        # Create a line geometry and set it to the first feature of the group,
        # then output the feature (line without spike).
        if 1 < len(self.features):
            self.features[0].setGeometry(fmeobjects.FMELine(self.coords))
            self.pyoutput(self.features[0])
        
    def close(self):
        # Create and output line for the final group.
        self.outputLine()

After trying both methods (I actually only had 1.7 million points). On my 64bit desktop.

-Baseline Method: No Attribute Assignment: 5min
-Takashi Method: Spatial Filter To Assign: 17min 50s
-Jonathan Method: ListSearch Coordinates To Assign: 7min 22s

Jonathan's method is obviously faster, but the ListSearcher (I did not try alternatives) seems to only match based on the first observation. The situation in my dataset does exist where an incorrect point has the same geometry as a correct point and accordingly I can't be sure the of which attribute is being assigned.

Takashi's method is slower but the spatial filter outputs all points which have the same geometry, from this point I can work out which of the points is the incorrect one (likely using the order value).
The whole time I was hoping (as I am not overly familiar with FME) that the point connector would track a relation (store a key) back to the source point for easy assignment of attribute values at a later time period if the line is broken back to its points.

Thanks for the advice you were both right, Takashi's solution better fits my actual data but Jonathan's method is better for the example I provided I hope I can accept both answers.

Thanks,
David

Thanks I will try both and report back. I know joining by location will work, but I was hoping it could come from the list for speed and that there could be a situation where a point that is out of sequence "flagged" has the same coordinate as an earlier point making it a one to many location join that I will then need to clean. Also I have several million points so it will be a good test.

Thanks

@takashi 's answer is a good one, but there is another way.

First you concatenate your input coordinates into a string (_coordinates).
Then after the point is flagged, you extract and then concatenate its coordinates. The two strings should be identical, and so all you need to do is run them through a ListSearcher. Be sure to "Demote Found List Element".

I suspect this method will be more efficient than takashi's for larger datasets as I'd hope the ListSearcher is faster than the SpatialFilter which can be quite resource intensive (and typically acts as a blocker for grouping purposes). I've not tested it on larger datasets myself so may be wrong.

capture.jpg(58.0 kB)