This document is meant as an informal guide for rNMR developers. If you're not interested in writing code or distributing rNMR, this guide is not for you; stop reading and go to http://rnmr.nmrfam.wisc.edu/pages/help/manual/manual.html for more information on using rNMR. If you're still reading, you might be searching for information on customizing rNMR for your own specific needs. We encourage this, just do us a favor and check out the the terms of rNMR's open source license before continuing.
Throughout this guide I will refer to rNMR functions using text such as this(). If the function I am referring to is a user function (exported in the rNMR namespace), you may click on the function name to display the help file for that function. More information on internal functions may be obtained by looking at the comments within the code itself. Things change, this guide might not. If I have omitted anything or have forgotten to update a section, feel free to contact one of the rNMR developers listed in the Contacts section.
rNMR was designed to be customizable. It has been our hope that by creating modularized source code and extensive documentation, users and developers would not shy away from modifying rNMR to suit their needs. However, there are a few internal functions and basic concepts, undocumented elsewhere, that would-be developers should be made aware of. Please review this section thoroughly before attempting any major modifications to the source code.
Before adding or modifying rNMR code, it is essential to understand how rNMR interacts with R's global environment. If you are not already familiar with rNMR's key variables: currentSpectrum, defaultSettings, fileFolder, globalSettings, oldFolder, overlayList, pkgVar, roiSummary, and roiTable, please review the help page detailing objects used by functions in the rNMR package before continuing.
When a spectrum is opened in rNMR, a new entry for that spectrum is made in fileFolder. The "file.par" section of this entry is populated using the ucsfHead() function. This function reads the header of a Sparky (ucsf) file and returns the information stored in it, as well as the noise estimate, maximum and minimum intensities, zero offset, and upfield and downfield shifts for the spectrum. The noise estimate for the spectrum is equal to one standard deviation and is used when determining contour levels and peak picking thresholds. The upfield and downfield shifts are calculated using the sweep widths and chemical shifts at the center of the spectrum in each dimension. The upfield chemical shift is then adjusted one increment downfield to correct for the displacement of the center frequency during Fourier transform (see the comments in the code for the ucsfHead() function).
Information from fileFolder along with the data contained in the ucsf file are then used to plot the spectrum. Plotting in rNMR starts with the refresh() function. For spectra displayed in the main plot window, this calls the drawNMR() function. drawNMR() then calls upon one of the lower level plotting functions: plot1D(), plot2D(), etc. After the plotting is complete, refresh() calls pdisp() to display peak markers if a peak list has been generated for the current spectrum. If the 1D projection/slice option is set for the current spectrum, proj1D() is called to display it. Finally, ol() is called (which in turn calls the drawNMR() function) to display spectra designated for overlay in overalyList.
When refreshing the subplot or multiple file windows, refresh() calls the rvs() and rvm() functions, respectively. These functions call upon draw2D(), which is similar to plot2D(), but is more efficient for creating multiple plots. Other plotting related functions of note include the functions used to read data from ucsf files: ucsf1D(), ucsf2D(), etc. These functions are used by ed() and the lower level plotting functions (draw2D(), proj1D(), etc.) to return data from a spectrum within a specified range.
To do: describe the intricacies of peak picking.
If your already familiar with Tcl/Tk or Tk in general, this section will help you with some of the subtleties of using Tk in R. If your unfamiliar with Tk, go to http://www.tkdocs.com/index.html to get an idea of what I'm talking about. For a primer on R's tcltk package, check out Tcl/Tk section of the September, 2001 issue of R News. Some specific examples can be found here: http://bioinf.wehi.edu.au/~wettenhall/RTclTkExamples/. This section covers a few intricacies I did not feel were thoroughly explored by these sources.
R is distributed with the tcltk package, an R package designed to interface with Tcl/Tk. However coding in Tcl/Tk and writing Tk code for R are two very different things. R's Tk documentation is very sparse so you'll have to use the Tcl/Tk manual as a reference, unfortunately that won't help you at all when it comes to syntax. Here's an example of the Tk command for creating a button using Tcl:
ttk::button .top.b -text "Okay" -command "exampleFun"
The ".top.b" argument is the pathname (also referred to as a widget's ID) for the widget. This is similar to a variable name, the difference being that the pathname also describes the parent windows for a given object. In the example above, the first "." refers to the root window, "top" refers to the toplevel window that will contain the button, and "b" is the button itself. Here's the same command as it should appear in R:
b <- ttkbutton(top, text="Okay", command=exampleFun)
In this example, the button widget will be stored to variable "b" in R. The first argument, "top", specifies the parent window (or widget) for the new button, similar to the pathname argument in the Tcl example. If "exampleFun" in the previous example was a function that accepted arguments, the command would have to be modified to look like this:
b <- ttkbutton(top, text="Okay", command=function(...) exampleFun(arg))
Certain Tcl commands are not associated with any pre-defined functions in R. In these cases, the tcl() function can be used to send a command as a character string to the Tcl interpreter. For example, here is the command used to create a button using the tcl() function:
b <- tcl("ttk::button", paste(top$ID, "b", sep="."), text="Okay", command=exampleFun)
Note the argument following the "ttk::button" call. This is the pathname for the new button widget. When using the tcl() function to create a widget, it is necessary to specify a pathname for the widget. However, most of the available Tk widgets have associated R commands which will create a pathname automatically. I'll discuss pathnames in greater detail in the next section.
The tcl() function may be useful in many situations, especially when modifying an existing widget or running a widget command. However, when creating a new widget without an associated R command, it may be easier to use the tkwidget() function so you don't have to specify a pathname for the widget:
b <- tkwidget(top, "ttk::button", text="Okay", command=exampleFun)
If you haven't already read the help page on R's Tcl interface, you should do so now (type "?TclInterface" in the R console). There's a few tips I'd like to point out here but I'd like to avoid going into a lot detail on topics already covered in other sources.
Let's begin with the tclVar() function. This function creates a Tcl variable in R. This is useful when creating a widget capable of holding a value in the form of a variable, such as a text entry or listbox widget. There are two ways to assign a value to a Tcl variable. One way is to use the tclVar() function directly:
tmp <- tclVar("hello world")
This is suitable for widgets that contain only a single value. However if tmp were passed to a listbox widget, two items would be displayed in the list, "hello" and "world". Tcl uses whitespace to separate values in a vector. The Tcl value associated with tmp in this example would be "hello world", but what we're really after is "{hello world}", indicating a single value. To accomplish this, we must first create a blank Tcl variable and then assign a value to it using the tclObj() function:
tmp <- tclVar()
tclObj(tmp) <- "hello world"
This is the method which should be used whenever you wish to create a Tcl variable to be used in a list widget. The tclObj() and tclvalue() functions can be used to assign or return the values associated with a Tcl variable in R. When used on the left-hand side of an assignment (as in the example above), these functions can be used to change the value of a Tcl variable associated with an existing widget (such as a listbox, a radiobutton, etc.). Either function is capable of returning the value of such a variable, but with different results. The tclvalue() function will return the value of the variable as a character string. Vector values are returned as a single character string, with each item separated by spaces. In this instance, tclObj() may be more appropriate. The tclObj() function returns an object of class "tclVar" which may be converted using functions such as as.character(), as.integer(), etc., and is capable of handling vectors.
Variables or objects in R are slightly different concepts than pathnames in Tcl. When a Tk widget is created in R, it is given an arbitrary pathname automatically. For instance, a new toplevel window created in R may be given the pathname ".1". The first widget created within that toplevel, let's say it's a frame, would have pathname ".1.1". A button within that frame would have pathname ".1.1.1". The next widget created within that frame would have pathname ".1.1.2", and so on. In R, the pathname for a widget is stored in the "ID" field of the R object the widget is assigned to. For the most part, it is not necessary to know the pathname R assigns to a widget, as this occurs automatically and the widget may be referred to by its associated variable name within R, "b" in the example above. There are a few exceptions to this rule which you may come across in the rNMR source code, most often when a Tcl command is called which returns the pathname of a widget, or when a toplevel or widget has not been assigned to a variable in R (e.g. - the toplevel windows used to create any of the rNMR GUIs).
The functions called to display the rNMR GUIs create toplevel windows that are not assigned to variables in the global environment. Therefore, it is necessary to refer to them by their pathnames when attempting to control or modify their behavior (e.g. - the hideGui() function). The myToplevel() function was designed to make this task easier by creating a toplevel window with a user-defined pathname. myToplevel() also ensures that only one instance of each GUI may be displayed at a time by searching the "TkRoot" environment (in R) for the pathname of the toplevel window used to create the GUI.
Versions of R for all systems come with the tcltk package, which is necessary to run Tcl/Tk code in R. In order to use this package, Tcl/Tk must be installed on the user's system. This can sometimes be problematic, so I've included this section to outline some of the issues that have arisen in the past.
Windows: Windows systems do not come with Tcl installed. This is actually a blessing in disguise. Windows binary versions of R come with Tcl/Tk, usually the most recent stable release. This is not the case for Mac and Linux systems. You may occasionally run into window management issues (keeping windows on top of other windows, directing the focus to a certain window, etc.) with the Windows version of R's tcltk package, but for the most part, R and Tcl/Tk work pretty well together on Windows systems.
Mac OS X: A recent version of Tcl/Tk for Mac OS X can be obtained from the R website. The problem here is that the Mac version of R's tcltk package requires the X11 distribution of Tcl/Tk rather than the Aqua based distribution. Aqua Tcl/Tk looks much more modern than the X11 version and may run smoother on newer systems (Leopard and up). Unfortunately, the Aqua version of Tcl/Tk can not be used in conjunction with R.app, which seems to be the preferred method of running R on Mac OS. It is possible to compile a command line version of R with Aqua Tcl/Tk, which could then be distributed with rNMR, but the benefits do not outweigh the effort required for such an undertaking.
A few unexplained glitches related to R's tcltk package on Mac systems have been encountered when running rNMR. Specific sets of circumstances (some we have been able to define, others remain a mystery) cause R to crash. For example, the ROI GUI was originally designed with a menu command that called the rs() function. rs() would in turn display a selection dialog which would cause R to crash. I'll try to explain this in more general terms, but you'll have to bear with me, it's rather complex. If a toplevel window is created with a menu and the "topmost" attribute is set to true (as in the ROI GUI), invoking a command from the menu which will generate another toplevel window, also with the "topmost" attribute set to true (in this case, the rs() function), will cause R to crash. The only solution we have found for these sorts of issues is to avoid the situations that give rise to them. For this particular example, we simply removed the menu command that displayed the selection dialog from the ROI GUI.
Linux: R does not come bundled with Tcl/Tk when installed on Linux systems. The tcltk package included with R version 2.10 and up requires Tcl/Tk 8.5 and will updated when R is installed or updated using synaptic package manager in Ubuntu, and probably with other software installation/update tools on other systems as well (yum, for example). rNMR no longer supports Tcl/Tk 8.4, which we have encountered on some of the older Linux systems, as the steps required to update Tcl/Tk are usually minimal. Even on systems with updated versions of Tcl/Tk, rNMR on Linux suffers from many of the same unexplained issues encountered while running Mac OS X. Fortunately, these problems are infrequent, and the majority of rNMR users run it on Windows systems.
rNMR requires Tcl/Tk version 8.5 or higher. The tclCheck() function is used throughout the rNMR source code to ensure that Tcl/Tk 8.5 is installed on the user's system and that the tcltk package has been loaded in R. For the most part, this is not an issue, however some systems may require an update before rNMR can be installed. The rNMR package is highly dependant on Tcl/Tk and R's tcltk package, so developers must remain vigilant to ensure that new versions of either do not conflict with the rNMR source code.
This section details a set of software tools I use on a regular basis while developing rNMR. Most of the tools described here are intended for developers who plan on modifying and redistributing rNMR. Redistributing rNMR with its full set of features and documentation requires compiling the source code in to R's package format. The tools and guidelines detailed in the remaining sections of this guide will help you accomplish this task. Individuals interested in modifying rNMR solely for their own personal use needn't bother working with the rNMR package. The easiest way to make modifications to rNMR is to edit and run the source code using Eclipse (see below). If you fall in to this category, you needn't read past the next section.
Eclipse is a software development environment primarily used for creating Java applications. However, by installing the StatET plugin for the Eclipse SDK, Eclipse can be very useful for coding in R. To install StatET, you'll have to use Eclipse's software installation tool. Detailed instructions can be found at the StatET installation page. Once installed, you'll have to configure StatET to interact with R. A set of Eclipse cheat sheets are provided with StatET to guide user's through this process.
Developers may obtain the files necessary for redistributing rNMR at the rNMR subversion repository on Sourceforge. The repository contains three directories:
current release: This directory contains the uncompiled version of the rNMR package, corresponding to the most recent official release. Developers interested in modifying rNMR should start here. For rNMR administrators, this directory is provided as a resource and should not be edited, use the "development" directory instead.
development: This directory contains an uncompiled version of the rNMR package that will eventually become the next official release. rNMR administrators should use this directory to make minor modifications, bug fixes, beta testing. New rNMR releases should originate from files in this directory and should replace the files in "current release" after being released.
new code: This directory contains code not yet integrated in to the source code of the rNMR package. New features and major modifications should be contained here. Once these functions have been tested and are stable, they can be integrated in to the source code found in the "development" directory.
Non-administrators (if you are an rNMR administrator, skip to the next paragraph) should download files from the "current release" directory using the "Download GNU tarball" link on the repository page. After downloading and extracting the package directory, move the "current release" folder to your Eclipse workspace and open Eclipse. Select "Import..." from the File menu and choose the "Existing Projects in to Workspace" option. This will add the project to your workspace so you can begin editing.
rNMR administrators should use subversion to track changes made to rNMR. Only developers in charge of the official rNMR distribution will be granted permission to edit the contents of the rNMR subversion repository on Sourceforge. For these individuals, I recommend installing the Subclipse subversion plugin for Eclipse. Here are the steps you will need to perform to connect to the rNMR repository:
After making modifications to rNMR, it's a good idea to test the new package on different platforms. Past experience has made it clear that the differences between versions of R running on Windows, Mac, and Linux systems are not trivial. Developers who do not have access to all three platforms may find VMWare to be a worthy substitution. The Windows 7 release candidate and most versions of Linux may be legally downloaded free of charge as virtual appliances. Unfortunately, Mac OS X may not be legally distributed as a free virtual appliance, so you'll have to find some other way of testing your package for compatibility on Mac systems. Virtual appliances run using VMWare Player, also available for free. VMWare Player runs within your system's OS as if it were a normal application, allowing the user the freedom to switch between the OS on the virtual machine and the host operating system.
After downloading and installing VMWare Player, you'll need to download a virtual appliance containing the OS you wish to run. I recommend using one of the virtual appliances found at http://bagside.com/bagvapp/index.html, as they are very well configured and have always run smoothly for me. VMWare's appliance page is also a good resource. Before downloading an appliance, you should make sure VMWare Tools have been installed. VMWare Tools greatly enhance the performance of the guest operating system. It is possible to install VMWare Tools yourself, but it can be difficult, and sometimes impossible. Appliances downloaded from the first link I mentioned will always come with VMWare Tools pre-installed. This may not be the case when obtaining appliances from other sources, so you'll have to do some investigating before downloading appliances elsewhere.
In the next section I will discuss building the rNMR package. After installing R, Mac and Linux systems are capable of running the necessary commands from the terminal. Windows systems, however, require the installation of Rtools. This toolset contains the necessary software tools to build R packages on Windows systems. The installation and configuration of these tools can be a complicated process so follow the instructions at the above link carefully.
In this section I will often refer to commands of the form "R CMD command_name option input". Mac and Linux users may assume these commands must be typed in to the terminal, from any directory. Windows users should use the command prompt and navigate to the "bin" folder in the R installation directory (unless you've added these commands to your system's PATH variable, but I'm not going to go into that here). If I don't refer specifically to the type of input the command accepts, assume you should provide the path to the uncompiled rNMR package directory. More information on these commands is covered in the R Introduction Manual. Before continuing, Windows users should download and the R toolset if not already installed (see section 4.4).
Before going in to the specifics of building the rNMR package, it might be helpful to know a few of the basics behind R's package structure and use. In this guide I will refer to two types of package formats, the source package and the binary package. The source package is an archived (tar) and compressed (Gzip) version of the source code, manual pages, DESCRIPTION file, and other package elements. Notice the difference in language here, the source code is the text files containing the R code for a package, while the source package is the entire package formatted for installation in R. I will refer to the creation of a source package as "building" the package. This is accomplished by calling "R CMD build rNMR" from a terminal. The "rNMR" portion of this command refers to the package directory (I'll talk more about package structure later).
I'm not going to go into detail on R's required package structure but I would like to spell out the contents of the current rNMR package. For more detailed information on R package structure, refer to the Writing R Extensions manual. Here are the contents of the rNMR package:
inst: Items in this directory will be copied to the main directory for the package after it is installed (see the Writing R Extensions manual). This includes:
man: Required for R packages containing help documentation, see the Writing R Extensions manual. This directory contains help documentation for each of the functions exported in the rNMR nampespace in R documentation format (.Rd).
R: Required for R packages, see the Writing R Extensions manual. Contains the rNMR source code.
DESCRIPTION: Required for R packages, see the Writing R Extensions manual. This file contains general information about the rNMR package. Please do not alter the "Depends" field.
NAMESPACE: rNMR contains a number of internal functions not meant to be called directly by the user. As such the package contains a namespace with only a select number of exported functions, see the Writing R Extensions manual for more information on namespaces.
Before building an R package, the package must be checked for errors. "R CMD check" will do a lot of this for you, so be sure to run this command before distributing a new package. You may not be able to fix everything "R CMD check" spits back at you, so keep in mind that anything labelled "NOTE" is okay to leave unchecked, assuming the issue can not be fixed. Warnings should be dealt with, if possible, and errors can not go unchecked or the package installation will fail. Before you make any modifications to rNMR you may want to run "R CMD check" so you can distinguish between issues already present in the normal rNMR distribution, and those caused by any modifications you've made. Currently (when this guide was written), "R CMD check" spits out a few notes, but the issues either can not be fixed or don't create any problems. A few things to note, "R CMD check" calls the gui() function several times when it runs, I'm not sure why this occurs, just try to ignore it. Also, several PDF files are created during the check which can be deleted after completion.
Once your system has been configured correctly, you will also be able to install source packages using "R CMD INSTALL rNMR_version.tar.gz". The "rNMR_version.tar.gz" portion of this command refers to the path to the source package you wish to install. When R is first installed, all add-on packages are installed to the same library path as the packages that come bundled with R (Graphics, Utilities, etc. ). Users do not usually have write permission for this directory so if a path is not specified when using "R CMD INSTALL", the sudo command is also required (unless of course the user is logged in as root). Ownership of the newly created directory is then given to root, or administrator, or some level higher than the current user. However, if a path is specified, and the user has write permission for that directory, for example: "R CMD INSTALL -l user/home/R rNMR_version.tar.gz", sudo is not required and the user will have ownership over that directory. Furthermore, future calls to "R CMD INSTALL" will use this as the default library for installing packages. This is important when installing rNMR so the package can be updated later using the updater() function.
The binary version of an R package is simply a compressed version of the contents of the package's installation directory (in the R library). A binary version of a package may only be installed on the same system type as the one it was originally compiled on. All Windows systems are compatible, however different versions of Mac OS X produce different binaries (R package binaries can not be built on Linux systems). For this reason, the only binary versions of the rNMR package currently in use are Windows binaries.
Windows binaries may be built using "R CMD build" or created by compressing the rNMR package directory (the R library for the rNMR package). The final product should be a zip file named "rNMR.zip" or "rNMR_version.zip" containing the rNMR package directory (the entire directory, not just its contents) as it appears after it is installed by R. The rNMR windows binaries are still generated to support updating the application from versions prior to 1.1.8. Since then, the updater and the Windows installer use the package source to install rNMR.
This section is included as a resource for rNMR administrators. One important aspect of creating an rNMR installer are the installation\uninstallation scripts. Because these scripts contain their own documentation, I won't detail their contents here. If you have customized rNMR and want to distribute your version using the methods described below, you may require resources, such as these scripts, not included in the rNMR package distribution. If this is the case, please contact one of the rNMR developers listed in the Contacts section for more assistance.
Three programs are used to create installers for rNMR, one for each system supported. These may not be the best installation programs available, but they're the best free options I could find and they work. All of the installers perform the following functions:
How these steps are performed varies depending upon the system and installation program used. I'll explain the basic principles behind each, but for more detailed information on how to use these programs, you'll have to refer to the program's help documentation, which should be more than adequate.
Before using Inno Setup, I recommend installing ISTool, which makes compiling Inno Setup scripts a bit easier. Inno Setup is a little complex, but it is well documented and comes with a lot of features. It also allows for customized scripting using Pascal, which is extremely useful, but does require knowledge of Pascal.
Here are the steps the installer performs:
rNMR can not be installed on Windows systems if R workspaces are not associated with R (i.e. - double-clicking on a saved R workspace will open R). Pascal scripts have been included in the Inno Setup script to aid in modifying the user's registry so that the installation script contained within "rNMR.RData" can be run. However, R version updates have caused the rNMR installer to fail in the past, so it is imperative that rNMR developers keep track of changes made to R when new versions are released. Software bugs may be inevitable, but if the installer fails, users are bound to lose interest.
Updating the installer: It's not necessary to update the installer prior to a new release unless the installation process needs to be modified.
InstallJammer is the only installer I could find for Linux systems. I assume this is because software repositories are the preferred method for installing and updating software on Linux systems. This may be the way to go in the future, but for now InstallJammer works fine, and it allows developers to track rNMR downloads. InstallJammer works on Windows as well, but it isn't as slick or as straight forward as Inno Setup. According to their website, they plan to support Mac OS X soon. If that happens, and the program works well on Macs, it may be a good idea to switch over to using InstallJammer on all systems. InstallJammer supports the use of shell scripts as well as internal Tcl scripting.
Here are the steps the installer performs:
*Note - When R installs a package to a directory, everything in that directory is cleared. Because the installation files are copied to the user's home directory, it is necessary to incorporate the rNMR executable, in this case a shell script, into the package itself (see the section on package structure) so that it is present after the package is installed. The shell script runs the file "loadrNMR.R" in R, which loads the rNMR package and saves a copy of the workspace as ".RData" (whenever R is ran from a directory containing a workspace with this name, the worksapce is loaded automatically). The shell script than starts R, which loads rNMR workspace that was just created.
Updating the installer: It's not necessary to update the installer prior to a new release unless the installation process needs to be modified.
Iceberg does what we need it to do, but there may be better alternatives out there, I just haven't found any. It has a fairly limited feature set, however it does support shell scripting.
Here are the steps the installer performs:
Updating the installer: It's not necessary to update the installer prior to a new release unless the installation process needs to be modified.
These are the tasks and checks that must be completed when releasing a new version of rNMR (this section is for meant primarily for rNMR administrators).
rNMR package:
rNMR homepage:
rNMR developers:
Ian A. Lewis - ialewis@nmrfam.wisc.edu
Seth C. Schommer - schommer@nmrfam.wisc.edu