Questions in topic: "r rcaller"
https://knowledge.safe.com/questions/topics/single/37689.html
The latest questions for the topic "r rcaller"RCaller Returning Empty Columns
https://knowledge.safe.com/questions/85014/rcaller-returning-empty-columns.html
<div class="fr-view clearfix"><p>Hi All,</p><p>I am attempting to use the RCaller to interpolate barometric pressures in water loggers from a set of data taken from a barometric logger in the project site. The R script I am using works in RStudio when I run it and returns correct results, and it runs without throwing an error in FME however the values calculated (interpolated barometric pressure) are <null> when they are taken out of the RCaller into FME.</p><p>Any advise would be greatly appreciated!</p><p>Thanks</p><p>-Jack<img src="/storage/attachments/24148-rcaller.jpg" /></p></div>scriptingrcallerr rcallerTue, 08 Jan 2019 20:55:31 GMTjecollinsHow do you use the RCaller and FME?
https://knowledge.safe.com/questions/53551/how-do-you-use-the-rcaller-and-fme.html
<p>Here at Safe, we are always trying to improve FME as well as our Knowledge Center articles. It recently came to our attention that we donâ€™t know much about how our users integrate R and FME using the RCaller transformer and the great things they are accomplishing with it!</p><div>Our questions to you: </div><ol>
<li>How do you use the RCaller transformer?</li><li>Are there any RCaller related articles you would like to see?</li></ol><p></p>r rcallerTue, 19 Sep 2017 21:24:44 GMTlizsandersonRCaller timeout error
https://knowledge.safe.com/questions/48564/rcaller-timeout-error.html
<p>I'm using an RCaller within a workbench that's processing some reasonably large datasets. The R script I'm using works fine in R itself, and the workbench runs perfectly with a smaller test dataset. </p><p>However with a full-size dataset I get the following error, approximately 30 seconds after the last feature enters the RCaller and the rscript.exe processes begin:</p><pre>RCaller(InlineQueryFactory): InlineQueryFactory encountered an error when starting R interpreter. Message was: Process operation timed out</pre><p>Running with the same size dataset in RGui or RStudio, the processing I'm doing takes approximately 10 minutes. However the processing is by group and so I use the ddply package in parallel mode to split the processing across 10 cores, making the process take approximately 1 minute.</p><p>I've transferred this over to the RCaller in FME, and it starts the 10 processes, but still times out after about 30 seconds. </p><p>How can I use the RCaller to do work that takes more than 30 seconds to produce an output? Is there any way of spitting out "something" so that FME knows it's running ok and should wait?</p><p></p>rcallerrr rcallerMon, 24 Jul 2017 17:05:36 GMTharrygibsonTutorial: Getting Started with the RCaller
https://knowledge.safe.com/articles/37979/tutorial-getting-started-with-the-rcaller.html
<p><a target="_blank" href="https://knowledge.safe.com/articles/37690/rcaller-ins-and-outs-of-using-the-rcaller.html"><strong>Next Section: RCaller: Ins and outs of using R in FME</strong></a><strong><br></strong></p><h1>Introduction</h1><p>If it is desirable to perform more advanced statistics than available in the <a href="https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/statisticscalculator.htm">StatisticsCalculator</a> transformer, the <a href="https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/rcaller.htm">RCaller transformer</a> makes much more statistical analysis possible in FME. The RCaller gives users the ability to run R scripts in FME, through the use of temporary R <a href="http://www.r-tutor.com/r-introduction/data-frame">data frames</a>.</p><p></p><h2><a target="_blank" href="https://knowledge.safe.com/articles/37690/rcaller-ins-and-outs-of-using-the-rcaller.html">RCaller: Ins and outs of using R in FME</a></h2><p>This example covers important RCaller basics, a few examples of its potential uses, and some of the RCaller challenges. </p><h2><a target="_blank" href="https://knowledge.safe.com/articles/33819/is-tree-height-and-tree-width-correlated.html">RCaller: Is Tree Height and Tree Width Correlated?</a></h2><p>Determine whether or not there is a correlation between the height of trees and the diameter of the trees in this RCaller demo.</p><h2><a target="_blank" href="https://knowledge.safe.com/articles/33822/interpolate-points-to-raster-through-kriging.html">RCaller: Interpolate Points to Raster Through Kriging</a></h2><p>The RCaller can be used to create a raster from points. In this example, kriging, an interpolation method, is used to create an estimated surface from cell signal point data.</p><p></p><p><a target="_blank" href="https://knowledge.safe.com/articles/37690/rcaller-ins-and-outs-of-using-the-rcaller.html"><strong>Continue to RCaller: Ins and outs of using R in FME</strong></a><strong><br></strong></p><p></p>statisticsrcallerr rcallerThu, 22 Dec 2016 23:27:04 GMTnataliewlockRCaller: Ins and outs of using R in FME
https://knowledge.safe.com/articles/37690/rcaller-ins-and-outs-of-using-the-rcaller.html
<p><strong><a target="_blank" href="https://knowledge.safe.com/articles/37979/tutorial-getting-started-with-the-rcaller.html">Intro: Getting Started with the RCaller</a> | <a target="_blank" href="https://knowledge.safe.com/articles/33819/is-tree-height-and-tree-width-correlated.html">Next: RCaller: Is Tree Height and Tree Width Correlated?</a></strong></p><h1>Introduction</h1><p>If you need to perform more advanced statistics than is available in the <a href="https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/statisticscalculator.htm">StatisticsCalculator</a> transformer, the <a href="https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/rcaller.htm">RCaller transformer</a> makes much more advanced statistical analysis possible in FME. The RCaller gives you the ability to run R scripts in FME.</p><h2>Downloads</h2><p><a id="5914" href="/storage/attachments/5914-rcallerlinearregression.fmwt">rcallerlinearregression.fmwt</a></p><p><a id="5910" href="/storage/attachments/5910-rcallerlinearregressionwithgroups.fmwt">rcallerlinearregressionwithgroups.fmwt</a></p><h2><a href="https://drive.google.com/open?id=0B3gvAFGWr2s6MGM2VkVYTEJHTTg"></a>Getting Started</h2><p>Before you can use the RCaller you need to install the appropriate R packages. See the section on <a target="_blank" href="https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/rcaller.htm">Installing the R Interpreter </a>in the FME User Documentation.</p><p>Before you get going with the RCaller there's a couple of useful things to remember:</p><ul><li>R doesn't like UNC path names so you can't run a FME Workspace stored on a UNC path, i.e.: \\myprojects\fmeWorkspaces. You have to be running your FME workspaces on a mapped drive, i.e: f:\myprojects\fmeWorkspaces</li><li>Do a bit of reading about R, if you're not already familiar with the concepts. This is a pretty good <a target="_blank" href="http://www.r-tutor.com/r-introduction">tutorial</a>. More resources are listed at the end of this article.</li></ul><h2>Source Data</h2><p>FME adds new ports to the RCaller as you connect transformers or feature types to the Connect Input port. The new input port will inherit its name from the source object (i.e the transformer name or the Feature Type Name). </p><p><img src="/storage/attachments/5894-rcaller.png"></p><p>The port names are used as the data frame names in R, so rename the port names to something you'll be able to use in your R scripts. </p><p>FME loads your data into a temporary SQLite database, so for both performance and clarity, only select the attributes you're going to use in your R scripts. Make sure the data types are correct. </p><p><img src="/storage/attachments/5895-rcallertables.png" style="width: 392px;"></p><p>FME transfers the data into R as a R <a target="_blank" href="http://www.r-tutor.com/r-introduction/data-frame">data frame</a>. You can access the data frame or data frame columns in your R script by dragging items from the Data Frames menu:</p><p><img src="/storage/attachments/5896-rscript1.png"></p><p>So to access the vector of Estimated values drag the Data - Estimated item into the script window and you'll see Data$Estimated in your R script window.</p><h2>Building an R Script</h2><p>This is not an R tutorial. To learn more about R, please see the resources section at the end of this article. If you're new to R, I'd recommend that you use the R Console to develop and debug your scripts - you'll get better feedback and it's a little easier to see the intermediate results. Then copy and paste the script into the RCaller. Build your script incrementally in the R Console so it's clear where any issues arise. You can load a sample of your data using the <a target="_blank" href="http://www.r-tutor.com/r-introduction/data-frame/data-import">R readers</a>: i.e. : </p><p> Data = read.csv("D:/tmp/SampleData.csv") </p><p> # <strong>Note</strong> R uses UNIX paths, i.e. '/' not '\'.</p><p><img src="/storage/attachments/5897-rconsole.png" style="width: 433px;"></p><p></p><h2>Getting the Results out of R...</h2><p>... can be tricky! RCaller passes data back to FME via a data frame called 'fmeOutput' . Each row in the data frame will become a separate output feature in FME. If you know how to build and append to R data frames you can probably skip this section. </p><p>To populate fmeOutput data frame, you can simply pass back a list of values (vectors of length one), i.e.:</p><pre>> Data = read.csv("D:/tmp/SampleData.csv")
> meanAct = mean(Data$Actual)
> meanEst = mean(Data$Estimated)
> fmeOutput = data.frame(meanEst, meanAct)
</pre><div>This will result in a single FME feature with the two mean values that have FME attribute names meanX & meanY</div><p>But many R functions return more complex results. For example a linear regression function solving for y=mx+k:
</p><pre>lm.linear <- lm(Data$Actual ~ Data$Estimated) </pre><p>Use the R <em>summary()</em> function to see the results: </p><pre>> summary(lm.linear)
Call: lm(formula = Data$Actual ~ Data$Estimated) Residuals:
Min 1Q Median 3Q Max
-9.9667 -2.1022 0.2679 2.3813 8.3354
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 12.051001 9.149612 1.317 0.211
Data$Estimated -0.009291 0.861531 -0.011 0.992
Residual standard error: 5.045 on 13 degrees of freedom
Multiple R-squared: 8.946e-06, Adjusted R-squared: -0.07691
F-statistic: 0.0001163 on 1 and 13 DF, p-value: 0.9916 </pre><p>How to get that back into FME? </p><p>The <em>names()</em> function will give you back the variable names in the summary, i.e.: </p><pre>> names(summary(lm.linear))
[1] "call" "terms" "residuals" "coefficients" "aliased"
[6] "sigma" "df" "r.squared" "adj.r.squared" "fstatistic"
[11] "cov.unscaled" </pre><p>But... some of these are more complex objects in their own right, like the "coefficients": </p><pre>> summary(lm.linear)$coefficients
Estimate Std. Error t value Pr(>|t|)
(Intercept) 12.051001492 9.1496116 1.31710525 0.2105480
Data$Estimated -0.009291111 0.8615308 -0.01078442 0.9915592 </pre><p>So what to do if you want to return to FME the common characteristics of the y=mx+k analysis such as the r squared value, m & k? </p><p>You have to pick-out the values you need and pass them to the fmeOutput data frame. In the above example, m is given by the Data$Estimated Estimate = -0.009291111 and k (y intercept) is given by (Intercept) Estimate = 12.051001492 and the r squared result is simple value: r.squared. So you can use: </p><pre>k <- summary(lm.linear)$coefficients[1,1] (the first column of the first row)
m <- summary(lm.linear)$coefficients[2,1] (the second column of the first row)
r2 <- summary(lm.linear)$r.squared</pre><p>That was easy!</p><p>The workspace <a id="5976" href="/storage/attachments/5976-rcallerlinearregression.fmwt">rcallerlinearregression.fmwt</a> illustrates the example described above.</p><p>One final tip: expose the result variables in the RCaller to make life easier in workbench:</p><p><img src="/storage/attachments/5916-rcalleroutputattributes.png" style="width: 338px;"></p><p></p><h4>Exporting R Results to a Temporary File</h4><p>In some cases it may not be appropriate to use a data frame for your R results, i.e for a large raster or an image. In this case you can export your R results to a temporary data file and have FME re-read those results. The article <a target="_blank" href="https://knowledge.safe.com/articles/33822/interpolate-points-to-raster-through-kriging.html">RCaller: Interpolate Points to Raster Through Kriging</a> illustrates how you can do this.</p><p></p><h2>Grouping with looping</h2><p> For many statistical problems, you have a qualitative value, i.e. Codes ABC ABD TXU, that have some bearing on the quantitative values. So simple grouping makes a lot of sense.</p><p>For example, you might want to calculate the mean of each Code value: </p><pre>Date Code Estimated Actual
11/29/2016 TXU 46.14 59.5
11/28/2016 ABD 43.89 34.1
11/27/2016 TXU 42.15 25.8
11/27/2016 ABC 9.3 20.3
11/26/2016 ABD 42.15 50.6
11/25/2016 ABC 11.04 11.7</pre><p>You can put your analysis in a loop, sample the data by the Code and then calculate the regression. Something like: </p><pre>for ( currentCode in unique(Data$Code)) { # assuming the input data.frame is 'Data'
tmpData = Data[Data$Code == currentCode,]
lm = lm(tmpData$Actual ~ tmpData$Estimated) # linear model on y = mx+k
}</pre><div><div>But you can't just use: </div><div style="margin-left: 20px;"> r2 <- summary(lm)$r.squared </div><div>to return the result, since you'll just return the last r2 out of the three calculated values. <div>One approach is to build vectors for each result and then copy these result vectors to the fmeOutput data frame.</div><pre># initialize vectors to hold results
r2 <- c()
m <- c()
k <- c()
Code <- character()
# y = Actual x = Estimated
for ( currentCode in unique(Data$Code)) {
tmpData = Data[Data$Code == currentCode,]
# linear regression for y = mx+k
lm.linear = lm(tmpData$Actual ~ tmpData$Estimated)
# linear model result vectors y = mx+k
r2 = c(r2, summary(lm.linear)$r.squared)
k = c(k, summary(lm.linear)$coefficients[1,1])
m = c(m, summary(lm.linear)$coefficients[2,1])
Code = c(Code, currentCode)
}
fmeOutput<-data.frame(Code, m, k, r2) </pre></div></div><p>You can assign the results directly to a data frame which would be more efficient, if you can figure it out. </p><p>The workspace <a id="5977" href="/storage/attachments/5977-rcallerlinearregressionwithgroups.fmwt">rcallerlinearregressionwithgroups.fmwt</a> illustrates the example described above.</p><p></p><p><strong>Debugging your R Script</strong></p><p>If you are relatively new to R, then I would recommend that you first develop your script in the R Console and then transfer to RCaller. It's a lot easier to debug there, see the section Building an R Script above. If you encounter the RCaller error:</p><p> ERROR |RCaller(InlineQueryFactory): InlineQueryFactory failed with exit code 1 when executing R script. Output was: Loading required package: gsubfn
Loading required package: proto
Loading required package: RSQLite </p><p>This seems to be a common error response if there is a syntax error in your script, or an undefined variable reference, so carefully check your script for unassigned variables or misspellings. </p><p>Remember, like FME, R is case sensitive.</p><h2>Other Resources</h2><p>Here are some useful resources around using R in FME:</p><p>FME RCaller <a target="_blank" href="http://docs.safe.com/fme/2017.0/html/FME_Desktop_Documentation/FME_Transformers/Transformers/rcaller.htm">documentation</a></p><p>'R' tutorials: <a href="http://www.r-tutor.com/">http://www.r-tutor.com/</a> and <a target="_blank" href="https://www.datacamp.com/courses/free-introduction-to-r?utm_source=adwords_ppc&utm_campaignid=644508384&utm_adgroupid=35605800561&utm_device=c&utm_keyword=r%20tutorial&utm_matchtype=e&utm_network=g&utm_adpostion=1t1&utm_creative=117835649361&utm_targetid=kwd-18697619167&utm_loc_interest_ms=&utm_loc_physical_ms=9060808&gclid=CIObwJLh9tACFU9rfgod_IUC2g">here</a></p><p>Extracting 'summary' information using summary(): <a href="https://psycnotes.wordpress.com/extracting-values-from-r-summary-objects/">example here</a></p><p>Appending to a data frame <a target="_blank" href="http://stackoverflow.com/questions/20689650/how-to-append-rows-to-an-r-data-frame">examples</a></p><p>Knowledge Center RCaller articles:</p><p><a href="https://knowledge.safe.com/articles/33822/interpolate-points-to-raster-through-kriging.html">RCaller: Interpolate Points to Raster Through Kriging</a></p><p><a target="_blank" href="https://knowledge.safe.com/articles/33819/is-tree-height-and-tree-width-correlated.html">RCaller: Is Tree Height and Tree Width Correlated?</a></p><p></p><p><strong><a target="_blank" href="https://knowledge.safe.com/articles/33819/is-tree-height-and-tree-width-correlated.html">Continue to RCaller: Is Tree Height and Tree Width Correlated?</a></strong></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p>statisticsrcallerr rcallerWed, 14 Dec 2016 22:56:13 GMTmarkatsafe