Exploring Open Source Data Management Solutions

And so it's been ages since I've posted... Had to get the Masters done... If there's interest take a look here.  Now that's done I've been working on some interesting projects for which the question always seems to be... How can complex, specific data management and analysis tasks be carried out in a simple foolproof way... This means graphical front ends for spatially capable databases and analysis software scripts. Now this gets interesting... upcoming posts will therefore extol the virtues and evils of of incorporating R scripts within QGIS or producing a custom front-end for R using the Shiny package.  We will also using PostGIS with PostgreSQL and building a custom front-end for this using Wavemaker or direct interaction with the database within QGIS... happy travelling!

Writing a QGIS Plugin

Such an easy title... such a smart idea... HARD WORK!!!

Anyway... after stumbling around for a bit with the PluginBuilder plugin... QGIS Cookbook etc... I finally settled on grinding through a tutorial... Which has proven the most rewarding.

I have learnt a few things about python and compiling and using the linux terminal that have proven useful.

• The shortcut for opening the terminal window is Ctrl +Alt + t   (Yaay...)
• Some compiling (the deadly c-word!) needs to be done.  Fortunately the pluginbuilder produces a super special file called a makefile which kind of knows what needs to be compiled and how.  All that I need to do after changing the user interface or editing the UI dialog code is to open the terminal :) select the directory in which the plugin is situated (cd ...) and type "make". The compiling is done!
• To get the directory string (to type after the cd command) I simply open it in a window and right click on the relevant directoy tab and copy it and paste as file name into the terminal window.
• at first the compiling di not work properly, I found that by simply typing pyrc4 or whatever package is necessary for the compiling, I was given instructions in the terminal as to how to go about installing it.

Those are just somelessons... as for writing the plugin... I suggest that anyone who wants to do this and is not a knowledgeable programmer finds a mentor and grinds throught the tutorial!

to be continued...!

Ramblings...

OK so what's been happening...

• I've tried to get some work done meanwhile the fish have gone hungry...
• I've switched from Mac to Windows to Linux Ubuntu 12 and I am finally happy... except I miss an MSOffice that works properly.
• I've got (more) clarity on the direction of my research, visited the study site, a beautiful pine plantation not too far away...
• I've been figuring out R... modelling relationships between various tree parameters... or at least trying to!
• I read an good book last night... The Go-Giver.  The principle being that we will be more successful if we give... stuff, of ourselves, of what we know, etc.... Hence the public sharing of my blog... Maybe someone could get something somewhere in it!
• I'm currently working on developing a plugin for QGIS that would contribute a useful tool as part of my research "package" that I hope to deliver by the end of next year.
• I found a great song for motivation :) ... The Script - Hall of Fame
  

Happy weekend!

Getting the SDA2PP plugin to work in QGIS. RPy2

Installing this plugin has proved to be a prolonged process.  I want to do Krigin and other interpolations in QGIS and have tried to install this plugin for these purposes.

To start off with the plugin was not even available on the list under Plugins>Fetch Python Plugins

After some searching and experimeting i found the plugin by selecting show all plugins, including experimental ones, although it does not appear to be experimental.

After selecting this option it became clear that much previous packages etc are required. These include rgdal, sp, RPy2, pywin32, numpy and an update of the python framework on my machine.  I am using a 64bit machine but it is not easy to work within a 64 bit pytho framework as many of the additional packages and moodules are not easily obtainable for 64 bit but only in 32 bit.  I therefore have kept to 32 bit to save headaches.

Most of the installs are not too complicated and can be found easily.  However I have struggled to install RPy2.

There is no precompiled self installing package for Rpy2 so it has to be done in another way.  The recommended way that i have had the most success with is using easyinstall, part of a package that is called setup tools.  I was able to install this package without any problems but could not get easy install to work.  Easy install downloads and installs packages etc from PyPI with the simple command

>easy_install rpy2

It is not all that simple however.  For this command to work the following is necessary after installation:

• Ensure that the Command Prompt is being used and not the Python command line.
• The Path variable needs to contain the directory in which easy_install.exe is... C:\Python27\Scripts\
• The path also needs to contain a directory for the R executable.

Changing the Path variable is done as shown in the following figure

http://code.google.com/p/sda4pp/downloads/detail?name=SDA4PP-System-0.9.0.exe&can=2&q=#makechanges

Installing Rpy2 on windows does not appear to be possible at this stage.  I am switching to linux.

Modelling individual tree in R using R Commander

And so I experiment modelling my tree data to get some kind of correlation between the coordinates and the tree size...  Also just to get a feel of the R commander GUI and the possibilities that R presents.  What I actually want to do is get an equation for the regression graph that was obtained in the previous post:

First it is necessary to create a model... tree size as a function of x and y coordinate!

Plotting Individual tree data in R. Windows,R Commander

I have switched to a 64-bit windows operating system and have also collected some of my own individual tree data using a surveyors total station.

For analysis of this data I started using  the steps as explained here.  However, I was unused to the windows interfadce and could not locate the command history window.  Doing some research to find the solution I came across an interesting GUI for R called R commander. It offer a variety of the analysis tools in drop down list form to from the commands that are then used as command lines.  This helps beginners like myself to get quicker results and also helps me to see what can be done in R.

Being a lover of 3d representations, plotting my data in 3d is one of the first things that I tried.

Importing the .csv data:

Name the dataset, ensure that "commas" is selected and the correct decimal -point character

On the tpop of the interface the active dataset is now displayed and one can view the data or edit the data directly in cells as in a spreadheet which is very useful.

One the dataset is active any of the manipulations from the above menus can be carried out. My selection: "3D Scatterplot"

The options are as follows:

I selected DBH as response variable and the x and y coordinates of the tree positions as explanatory variable and I wanted to see the smooth regression.  The result, a nice representation of the spatial distribution of tree size:

Saving the graph to file is an option in the R Commander GUI in the "Graphs" menu.

Now to get some kind of equation for this regression.... Modeeling in R using R Commander

Collecting Individual tree data Using a Surveyor's Total Station