iRefIndex MITAB2.6 Parser

A tool has been developed to parse the MITAB files produced in the iRefIndex Build Process. Currently, the tool is capable of parsing the MITAB format described on the page README MITAB2.6 for iRefIndex 7.0.

Obtaining the MITAB Parser

The parser and associated resources can be obtained from this location:

https://hfaistos.uio.no/cgi-bin/viewvc.cgi/mitab/

Using CVS with the appropriate CVSROOT setting, run the following command:

cvs co mitab

The CVSROOT environment variable should be set to the following for this to work:

export CVSROOT=:ext:<username>@hfaistos.uio.no:/mn/hfaistos/storage/cvsroot

(The <username> should be replaced with your actual username.)

Prerequisites

The following programs are required to use the parser:

• Python (tested with 2.5.4)
• PostgreSQL (tested with 8.1.17)

Running the Parser

Given a directory for the iRefIndex output files such as...

/home/irefindex/output

...run the parser as follows:

python parse_mitab.py /home/irefindex/output/All.mitab.03042009.txt

It will be necessary to change the date details included in the above filename to match the actual name of the appropriate file found in your own output directory.

Creating the Database

A database can be created using the usual PostgreSQL tools:

createdb -E unicode mitab_irefindex

This database is initialised as follows:

psql -f init_mitab.sql mitab_irefindex

Should the database tables need to be dropped (perhaps in case of problems with the import), the following command can be used:

psql -f drop_mitab.sql mitab_irefindex

Populating the Database

The database is populated as follows:

python database_action.py mitab_irefindex import_mitab.sql

As a result, a number of tables representing the structure of the data should be available in the database. For applications built to use this data, indexes may need creating in order to make querying more efficient.

Notes on the Populated Database

The schema used by the populated database attempts to model the data as effectively as possible using a number of tables:

Entity type	Tables	Table purpose	Notable columns	Source columns (if different or converted)
Interaction	mitab_interactions	Model each interaction referencing interactors	rigid, edgetype, numParticipants, crigid
	mitab_sources	Represent sources for each interaction	rigid, sourcedb, name	sourcedb
	mitab_interaction_type_names	Represent interaction types for each interaction	rigid, code, name	interactionType
	mitab_interaction_identifiers	Represent interaction identifiers for each interaction	rigid, dbname, uid	interactionIdentifiers
	mitab_confidence	Represent confidence scores for each interaction	rigid, type, confidence	confidence
Experiment	mitab_method_names	Represent detection methods for each interaction	rigid, code, name	method
	mitab_authors	Represent publication authors for each interaction	rigid, author	author
	mitab_pubmed	Represent publication identifiers for each interaction	rigid, pmid	pmids
Interactor	mitab_interactions	Model each interaction referencing interactors	uidA, uidB, taxA, taxB, atype, btype, crogidA, crogidB
	mitab_aliases	Represent aliases for each interactor	uid, dbname, alias	uidA or uidB, aliasA or aliasB
	mitab_alternatives	Represent alternative identifiers for each interactor	uid, dbname, alt	uidA or uidB, altA or altB
	mitab_interactor_rogs	Represent alternative integer identifiers for each interactor	uid, rog	uidA or uidB, irogA or irogB
Canonical interactor	mitab_canonical_interactor_rogs	Represent alternative integer identifiers for each canonical interactor	uid, rog	crogidA or crogidB, icrogA or icrogB

Some changes in representation occur when creating the database:

• Prefixed values are generally split to expose the prefix and identifier, name or value following it.
  • The various interaction and interactor prefixes (such as irefindex:, rigid:, rogid:, crigid: and crogid:) are omitted from interaction and interactor columns.
  • Source identifiers are split with the prefix (such as intact:) used to make a dbname column with the actual identifier stored in its own column (such as alias or alt).
• The "empty value" (-) should never appear as an identifier, and where such a value is used in a list, that element should be excluded. This is pertinent in the case of vocabulary terms where MI:0000 might be used together with an empty list of identifiers or names as an "empty collection" indicator.
• Duplicate values in lists are generally discarded.

Further work may include the introduction of a separate interactor table, collecting related information for each interactor.

Canonical interactors and interactions

The mitab_interactions table incorporates the canonical interaction and interactors alongside the specific interaction and interactors. A separate mitab_canonical_interactor_rogs table is used to map canonical interactors to integer identifiers, just as mitab_interactor_rogs does so for specific interactors.

All iRefIndex Pages

Follow this link for a listing of all iRefIndex related pages (archived and current).