span8
span4
span8
span4
Hi all,
Would anyone be able to suggest a workflow of exporting non-spatial records from wfs service to csv format? It is a big file with property title information (2+ million records), so I was thinking about narrowing the search by specifying Land District (GetFeature?). I did not work much with wfs services so simplified explanation would be great.
Maybe someone from New Zealand with data.linz experience, especially their wfs services?
Regards,
Maksym
I ended up running workbench with SQLExecutor, which uses required parcels an an initiator. Also I set syntax, that @kimo suggested, exposed attributes and run it through AttributeManager to retrieve duplicated values. On the output I received a table of title_no relations between two sources. It has duplicated values for title_no, as was expected.
Bruce's two workbenches do work, but for spatial layers only. They do not use CSV and they do not make a subset of non-spatial records which was the question. They also leave all field widths at 2048+ chars which will crash ArcGIS when you try to index these fields.
The problem is that there is no easy query you can do on the non-spatial tables so you have to download the whole table and make a subset using some other related spatial layer afterwards.
Trying to download very large tables with WFS is sure to end badly. My system halts suspiciously at exactly 1 M records for some layers so there is some sort of limit that I could not remove, even though LINZ says there is no limit on their part and it works for them.
For very large tables LINZ provide a changeset mechanism to just download the changes to merge into an existing table that has been downloaded previously using the zip-and-ship interface. This is quite a bit more complex but much faster in the end.
My solution is to use the HttpCaller transformer to download the changesets to a CSV file and then update a geodatabase with a predefined schema. The GML format is no better at defining field widths than CSV, both give you the field names. The advantage of CSV is that it is much more compact, human readable and flexible.
Even CSV spatial layers are able to be converted into featureclasses because the shape field is WKT format which can be transformed.
Web services are unreliable. Often you get a failure to connect, or you get an interruption. It is helpful to check if you have got the expected number of records and allow for a restart if it is not correct before you plough on and corrupt your database.lds-bde-sqlite-changeset-sample.fmw
SELECT * from title_memorial, mytitle WHERE title_memorial.title_no = mytitle.title_memorial AND current = 'T'
Hi Maksym
MakeFeatureTypes creates a Geodatabase table handy for browsing LDS data but primarily useful for creating the picklist of available layers for DownloadFeatures, which does what it says; tables will ignore the bounding box parameter.
Output is Geodatabase.
DownloadFeatures is suitable for embedding in a ModelBuilder tool to pick up map extent, I'll make a sample when I get time.
You will need to edit the URL parameters to insert your API key.
© 2019 Safe Software Inc | Legal