iRefIndex Output and Statistics

From irefindex
Revision as of 14:09, 21 June 2011 by PaulBoddie (talk | contribs) (→‎Creating a Mapping Wiki Page: Fixed CVS checkout.)

The production of output and statistics involves two separate programs: PSI_MI_TAB_Maker and BioPSI_Suplimenter.

PSI-MI Controlled Vocabulary Mapping

NoteNote

Sabry to help document this part.

In the data maintained by iRefIndex, various controlled vocabulary terms are used which do not match genuine terms defined in the molecular interaction ontology. As a result, a process is followed involving the extraction of such unrecognised terms, the curation of a mapping to replacement terms, and the processing of the maintained data to use the replacement terms.

Currently, the curation process is performed by assembling the unrecognised terms in a spreadsheet which is then modified, adding suggested replacements alongside the existing terms.

Creating a Mapping Wiki Page

A page summarising the mapping of unrecognised terms to known terms should be prepared such as the Mapping of terms to MI term ids - iRefIndex 8.0‎ page.

The cv2wiki.py script needs to be obtained. Get the program's source code from this location:

Using CVS with the appropriate CVSROOT setting, run the following command: 

cvs co bioscape/bioscape/modules/interaction/Sabry/cv2wiki.py

The CVSROOT environment variable should be set to the following for this to work: 

export CVSROOT=:ext:<username>@hfaistos.uio.no:/mn/hfaistos/storage/cvsroot

(The <username> should be replaced with your actual username.)

Running the Program

The program must be run on comma-separated value files exported from the curation spreadsheet. First, the "interaction type" and "interaction detection method" sheets must be individually exported using the following settings:

  • The field delimiter is defined to be the comma (,) character
  • Field quoting is done using the double-quote (") character

With exported files defined, for example, as cv_int_type.csv and cv_int_det_method.csv for the "interaction type" and "interaction detection method" sheet files respectively, the following command can then be run: 

python cv2wiki.py cv_int_type.csv cv_int_det_method.csv MediaWiki

The output in the above example will be written to standard output (the terminal/console). To write to a file, add a filename as an argument to the program. For example: 

python cv2wiki.py cv_int_type.csv cv_int_det_method.csv MediaWiki CVMapping

This file could potentially be uploaded to the Wiki using a tool suitable for this purpose. For a Wiki such as MoinMoin (also supported by the program), the file could potentially be copied into place with a certain degree of care.

Building PSI_MI_TAB_Maker

The PSI_MI_TAB_Maker.jar file needs to be obtained or built.

  1. Get the program's source code from this location:

    Using CVS with the appropriate CVSROOT setting, run the following command: 

    cvs co bioscape/bioscape/modules/interaction/Sabry/PSI_MI_TAB_Maker

    The CVSROOT environment variable should be set to the following for this to work: 

    export CVSROOT=:ext:<username>@hfaistos.uio.no:/mn/hfaistos/storage/cvsroot
    (The <username> should be replaced with your actual username.)
  2. Obtain the program's dependencies. This program uses the MySQL Connector/J library which can be found at the following location:
  3. Extract the dependencies: 
    tar zxf mysql-connector-java-5.1.6.tar.gz

    This will produce a directory called mysql-connector-java-5.1.6 containing a file called mysql-connector-java-5.1.6-bin.jar which should be placed in the lib directory in the PSI_Writer directory... 

      mkdir lib
  cp mysql-connector-java-5.1.6/mysql-connector-java-5.1.6-bin.jar lib/

    You may instead choose to copy the library from the BioPSI_Suplimenter/lib directory: 

      mkdir lib
  cp ../BioPSI_Suplimenter/lib/mysql-connector-java-5.1.6-bin.jar lib/

    The filenames in the above example will need adjusting, depending on the exact version of the library downloaded.

    The SHA.jar file needs copying from its build location: 

    cp ../SHA/dist/SHA.jar lib/

    Alternatively, the external libraries can also be found in the following location: 

    /biotek/dias/donaldson3/iRefIndex/External_libraries
  4. Edit the main method in the src/psi/no/uio/biotek/PSI_MI_Tab.java file and change the following variables:
    • Change the host, database, username and password used in the database connection. For example: 
      Connect con = new Connect("myhost", "irefindex", "irefindex", "mysecretpassword");
    • Change the output files. For example: 
      LogAppend log = new LogAppend("/home/irefindex/logs/" + date + "PSI_tab_writer.log");
String file = "/home/irefindex/output/<taxid>.mitab." + date + ".txt";  //Proprietary
String zipfile = "/home/irefindex/output/<taxid>.mitab." + date + ".txt";
    • Make sure that the output directory has been created. For example: 
      mkdir /home/irefindex/output
  5. Compile the source code. In order to build the software on a computer which does not have the NetBeans IDE installed, copy the generic build file into the PSI_MI_TAB_Maker directory: 
    cp Build_files/build.xml .

    Compile and create the .jar file as follows: 

    ant jar

Running PSI_MI_TAB_Maker

NOTE: The tables described here are prepared by the cv.no.uio.biotek.Preprocess class in a step which will be documented later.

In order to run the program, some additional database tables are required. One way of ensuring that such tables exist and are suitable is to drop any existing tables within the database being built, then to copy existing tables from a previously built database: 

use <database>;
drop table mapping_intDitection;
drop table mapping_intType;
drop table mapping_partidentification;
create table mapping_intDitection like <old_database>.mapping_intDitection;
create table mapping_intType like <old_database>.mapping_intType;
create table mapping_partidentification like <old_database>.mapping_partidentification;
insert into mapping_intDitection select * from <old_database>.mapping_intDitection;
insert into mapping_intType select * from <old_database>.mapping_intType;
insert into mapping_partidentification select * from <old_database>.mapping_partidentification;
create table taxid2name as select * from <old_database>.taxid2name;

For example: 

use irefindex;
drop table mapping_intDitection;
drop table mapping_intType;
drop table mapping_partidentification;
create table mapping_intDitection like old_db.mapping_intDitection;
create table mapping_intType like old_db.mapping_intType;
create table mapping_partidentification like old_db.mapping_partidentification;
insert into mapping_intDitection select * from old_db.mapping_intDitection;
insert into mapping_intType select * from old_db.mapping_intType;
insert into mapping_partidentification select * from old_db.mapping_partidentification;
create table taxid2name as select * from old_db.taxid2name;

Run the program as follows: 

java -jar -Xms256m -Xmx256m build/jar/PSI_MI_TAB_Maker.jar

Follow the instructions, supplying the requested arguments when running the program again.

Running BioPSI_Suplimenter to Produce Statistics

This program was already built and run in the iRefIndex Build Process. For a completed build process it should not need to be run again, but an option does exist to explicitly produce statistics for the system, should this be required. The basic details of running the program are described in Running BioPSI_Suplimenter.

Create reports

Upon selecting this option in the running program, complete the fields as previously described. The reports will be written to the designated log file directory.

Creating a Statistics Wiki Page

The reports2wiki.py file needs to be obtained. Get the program's source code from this location:

Using CVS with the appropriate CVSROOT setting, run the following command: 

cvs co bioscape/bioscape/modules/interaction/Sabry/reports2wiki.py

The CVSROOT environment variable should be set to the following for this to work: 

export CVSROOT=:ext:<username>@hfaistos.uio.no:/mn/hfaistos/storage/cvsroot

(The <username> should be replaced with your actual username.)

Running the Program

The program can be run on the report files in the log file directory as follows: 

python reports2wiki.py /home/irefindex/output/Suplimenter03052009 MediaWiki

The "prefix" of the report files should be the common part of all such files such that... 

ls /home/irefindex/output/Suplimenter03052009*

...should list the report files (and log files) produced by iRefIndex.

The output in the above example will be written to standard output (the terminal/console). To write to a file, add a filename as an argument to the program. For example: 

python reports2wiki.py /home/irefindex/output/Suplimenter03052009 MediaWiki Statistics_iRefIndex_3.0

This file could potentially be uploaded to the Wiki using a tool suitable for this purpose. For a Wiki such as MoinMoin (also supported by the program), the file could potentially be copied into place with a certain degree of care.

All iRefIndex Pages

Follow this link for a listing of all iRefIndex related pages (archived and current).

Retrieved from "http://irefindex.vib.be/wiki/index.php?title=iRefIndex_Output_and_Statistics&oldid=3508"