Difference between revisions of "iRefIndex Release Process"

From irefindex
(Added file upload information.)
(→‎iRefScape: Linked to new page.)
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
The release process consists of the following activities:
+
The release process consists of a number of activities.
  
−
# Build the database - see [[iRefIndex Build Process]] for details.
+
== Building the Database ==
−
# Generate output files and statistics - see [[iRefIndex Output and Statistics]] for details.
+
 
−
# Test the output.
+
See [[iRefIndex Build Process]] for details.
−
# Post the output files, statistics and release information.
+
 
 +
== Generating Output Files and Statistics ==
 +
 
 +
See [[iRefIndex Output and Statistics]] for details.
  
 
== Testing the Output ==
 
== Testing the Output ==
 +
 +
Testing includes the following activities:
  
 
* Checking for errors and log messages
 
* Checking for errors and log messages
Line 13: Line 18:
 
* Counting XML tags
 
* Counting XML tags
 
* Validation
 
* Validation
 +
** PSI-MI MITAB files are validated using scripts available at [ftp://ftp.no.embnet.org/irefindex/tools/validater]
 +
** PSI-MI XML files are validated using the validator at [http://www.ebi.ac.uk/intact/validator/start.xhtml]
 
* Parsing the final output files - see [[iRefIndex MITAB Parser]] for details.
 
* Parsing the final output files - see [[iRefIndex MITAB Parser]] for details.
 +
 +
See [[iRefIndex Testing 7.0]] for more information.
  
 
== Posting the Output Files ==
 
== Posting the Output Files ==
Line 19: Line 28:
 
The output files themselves are made available on the iRefIndex FTP site as described on the [[iRefIndex#Data_availability_via_download|iRefIndex information page]]. This involves making a directory in the following local location:
 
The output files themselves are made available on the iRefIndex FTP site as described on the [[iRefIndex#Data_availability_via_download|iRefIndex information page]]. This involves making a directory in the following local location:
  
−
<pre>/biotek/proteas/ftp/irefindex/data/archive</pre>
+
<pre>/mn/biotroll/ftp/proteas/irefindex/data/archive</pre>
  
−
For example, for the iRefIndex 3.0 release, the following directory hierarchy is created:
+
For example, for the iRefIndex 9.0 release, the following directory hierarchy is created:
  
−
<pre>mkdir -p /biotek/proteas/ftp/irefindex/data/archive/release_3.0/psimi_tab</pre>
+
<pre>mkdir -p /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi_tab/MITAB2.6</pre>
  
 
The data files are then copied into this new directory. For example:
 
The data files are then copied into this new directory. For example:
  
−
<pre>cp /home/irefindex/output/*.zip /biotek/proteas/ftp/irefindex/data/archive/release_3.0/psimi_tab/</pre>
+
<pre>cp /home/irefindex/output/*.zip /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi_tab/MITAB2.6/</pre>
  
 
A <tt>README</tt> file should be placed in the directory containing a link to the format description. For example:
 
A <tt>README</tt> file should be placed in the directory containing a link to the format description. For example:
Line 34: Line 43:
 
Please see
 
Please see
  
−
http://irefindex.uio.no/wiki/README_iRefIndex_MITAB_3.0
+
http://irefindex.uio.no/wiki/README_iRefIndex_MITAB_9.0
 
</pre>
 
</pre>
  
−
The symbolic link indicating the location of the current release needs to be updated. For iRefIndex 3.0 this is done as follows:
+
The symbolic link indicating the location of the current release needs to be updated. For iRefIndex 9.0 this is done as follows:
  
 
<pre>
 
<pre>
−
cd /biotek/proteas/ftp/irefindex/data
+
cd /mn/biotroll/ftp/proteas/irefindex/data
 
rm current
 
rm current
−
ln -s archive/release_3.0 current
+
ln -s archive/release_9.0 current
 
</pre>
 
</pre>
  
−
Finally, the <tt>release</tt> file must be updated to indicate the current release number. For iRefIndex 3.0 this contains the following text:
+
Finally, the <tt>release</tt> file must be updated to indicate the current release number. For iRefIndex 9.0 this contains the following text:
  
 
<pre>
 
<pre>
−
release3.0
+
release9.0
 
</pre>
 
</pre>
 +
 +
=== Mapping Files ===
 +
 +
The mapping file produced during output data preparation needs to be copied to the FTP site. First, create a directory within the release directory for the file:
 +
 +
<pre>mkdir /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/Mappingfiles</pre>
 +
 +
The <tt>mappings.txt.gz</tt> file is then copied into this new directory. For example:
 +
 +
<pre>cp mappings.txt.gz /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/Mappingfiles/</pre>
  
 
=== Optional/Experimental Output Files ===
 
=== Optional/Experimental Output Files ===
  
−
Additional PSI-MI-XML files may be made available within the release directory within a subdirectory created as follows (for iRefIndex 3.0):
+
Additional PSI-MI-XML files may be made available within the release directory within a subdirectory created as follows (for iRefIndex 9.0):
  
−
<pre>mkdir -p /biotek/proteas/ftp/irefindex/data/archive/release_3.0/psimi25xml</pre>
+
<pre>mkdir -p /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi25xml</pre>
  
 
(Instructions on copying these files will be added in future.)
 
(Instructions on copying these files will be added in future.)
Line 64: Line 83:
 
Please see
 
Please see
  
−
http://irefindex.uio.no/wiki/README_iRefIndex_PSIMI25XML_3.0
+
http://irefindex.uio.no/wiki/README_iRefIndex_PSIMI25XML_9.0
  
 
(This directory contains a sample of iRefIndex data in PSI-MI 2.5 XML format for testing purposes only)
 
(This directory contains a sample of iRefIndex data in PSI-MI 2.5 XML format for testing purposes only)
 
</pre>
 
</pre>
  
 +
== Feedback ==
 +
 +
Filter feedback files for level 6 mapping errors and send to databases.
 +
 +
The feedback is generated using BioPSI_Suplimenter as follows:
 +
 +
java -cp build/classes:lib/mysql-connector-java-5.1.6-bin.jar report.no.uio.biotek.Create_FeedBack <host> <database> <username> <password> <directory> <release>
 +
 +
For example:
 +
 +
java -cp build/classes:lib/mysql-connector-java-5.1.6-bin.jar report.no.uio.biotek.Create_FeedBack host irefindex irefindex ... /home/irefindex/output/ 9.0
 +
 +
This activity must be completed before iRefWeb data is prepared due to some dependencies on some of this activity's products.
 +
 +
=== Posting the Feedback ===
 +
 +
As with the output data, this involves making a directory in the following local location:
 +
 +
<pre>/mn/biotroll/ftp/proteas/irefindex/feedback/archive</pre>
 +
 +
For example, for the iRefIndex 9.0 release, the following directory hierarchy is created:
 +
 +
<pre>mkdir -p /mn/biotroll/ftp/proteas/irefindex/feedback/archive/release_9.0</pre>
 +
 +
The feedback files are then copied into this new directory. For example:
 +
 +
<pre>cp /home/irefindex/output/FeedBack/*.zip /mn/biotroll/ftp/proteas/irefindex/feedback/archive/release_9.0/</pre>
 +
 +
A <tt>README</tt> file should be placed in the directory containing a link to the format description. For example:
 +
 +
<pre>
 +
This directory contains feedback files for iRefIndex release 9.0.
 +
 +
Please see http://irefindex.uio.no/wiki/README_iRefIndex_Feedback_9.0
 +
</pre>
 +
 +
The symbolic link indicating the location of the current release needs to be updated. For iRefIndex 9.0 this is done as follows:
 +
 +
<pre>
 +
cd /mn/biotroll/ftp/proteas/irefindex/feedback
 +
rm current
 +
ln -s archive/release_9.0 current
 +
</pre>
 +
 +
Finally, the <tt>release</tt> file must be updated to indicate the current release number. For iRefIndex 9.0 this contains the following text:
 +
 +
<pre>
 +
release9.0
 +
</pre>
 +
 +
== Updating the Wiki ==
 +
 +
The [[iRefIndex]] page will need updating with version-specific references such as the following:
 +
 +
* <tt>README_iRefIndex_MITAB_x.y</tt>
 +
* <tt>Sources_iRefIndex_x.y</tt>
 +
* <tt>README_iRefIndex_Feedback_x.y</tt>
 +
* <tt>Statistics_iRefIndex_x.y</tt>
 +
 +
New documents will need creating for some of these references, and general redirect pages should be updated to point to the latest versions of the above pages.
 +
 +
For new data sources, the text will need editing and a citation reference should be added for the new source.
 +
 +
== iRefScape ==
 +
 +
Data for [[iRefScape]] must be prepared as described on the [[iRefIndex Data Preparation for iRefScape]] page.
 +
 +
== iRefWeb ==
 +
 +
Data for iRefWeb must be prepared as described on the [[iRefIndex Data Preparation for iRefWeb]] page.
 +
 +
=== iRefWeb LinkOut Data ===
 +
 +
The [[iRefWeb LinkOut Generator]] needs to be run and the generated data submitted to the LinkOut FTP site once iRefWeb has been updated.
 +
 +
== All iRefIndex Pages ==
 +
 +
Follow this link for a listing of all iRefIndex related pages (archived and current).
 
[[Category:iRefIndex]]
 
[[Category:iRefIndex]]

Latest revision as of 16:26, 28 October 2011

The release process consists of a number of activities.

Building the Database

See iRefIndex Build Process for details.

Generating Output Files and Statistics

See iRefIndex Output and Statistics for details.

Testing the Output

Testing includes the following activities:

  • Checking for errors and log messages
  • Web services
  • Spot checks
  • Counting XML tags
  • Validation
    • PSI-MI MITAB files are validated using scripts available at [1]
    • PSI-MI XML files are validated using the validator at [2]
  • Parsing the final output files - see iRefIndex MITAB Parser for details.

See iRefIndex Testing 7.0 for more information.

Posting the Output Files

The output files themselves are made available on the iRefIndex FTP site as described on the iRefIndex information page. This involves making a directory in the following local location: 

/mn/biotroll/ftp/proteas/irefindex/data/archive

For example, for the iRefIndex 9.0 release, the following directory hierarchy is created: 

mkdir -p /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi_tab/MITAB2.6

The data files are then copied into this new directory. For example: 

cp /home/irefindex/output/*.zip /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi_tab/MITAB2.6/

A README file should be placed in the directory containing a link to the format description. For example: 

Please see

http://irefindex.uio.no/wiki/README_iRefIndex_MITAB_9.0

The symbolic link indicating the location of the current release needs to be updated. For iRefIndex 9.0 this is done as follows: 

cd /mn/biotroll/ftp/proteas/irefindex/data
rm current
ln -s archive/release_9.0 current

Finally, the release file must be updated to indicate the current release number. For iRefIndex 9.0 this contains the following text: 

release9.0

Mapping Files

The mapping file produced during output data preparation needs to be copied to the FTP site. First, create a directory within the release directory for the file: 

mkdir /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/Mappingfiles

The mappings.txt.gz file is then copied into this new directory. For example: 

cp mappings.txt.gz /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/Mappingfiles/

Optional/Experimental Output Files

Additional PSI-MI-XML files may be made available within the release directory within a subdirectory created as follows (for iRefIndex 9.0): 

mkdir -p /mn/biotroll/ftp/proteas/irefindex/data/archive/release_9.0/psimi25xml

(Instructions on copying these files will be added in future.)

A README file should be placed in the directory with the following form: 

Please see

http://irefindex.uio.no/wiki/README_iRefIndex_PSIMI25XML_9.0

(This directory contains a sample of iRefIndex data in PSI-MI 2.5 XML format for testing purposes only)

Feedback

Filter feedback files for level 6 mapping errors and send to databases.

The feedback is generated using BioPSI_Suplimenter as follows: 

java -cp build/classes:lib/mysql-connector-java-5.1.6-bin.jar report.no.uio.biotek.Create_FeedBack <host> <database> <username> <password> <directory> <release>

For example: 

java -cp build/classes:lib/mysql-connector-java-5.1.6-bin.jar report.no.uio.biotek.Create_FeedBack host irefindex irefindex ... /home/irefindex/output/ 9.0

This activity must be completed before iRefWeb data is prepared due to some dependencies on some of this activity's products.

Posting the Feedback

As with the output data, this involves making a directory in the following local location: 

/mn/biotroll/ftp/proteas/irefindex/feedback/archive

For example, for the iRefIndex 9.0 release, the following directory hierarchy is created: 

mkdir -p /mn/biotroll/ftp/proteas/irefindex/feedback/archive/release_9.0

The feedback files are then copied into this new directory. For example: 

cp /home/irefindex/output/FeedBack/*.zip /mn/biotroll/ftp/proteas/irefindex/feedback/archive/release_9.0/

A README file should be placed in the directory containing a link to the format description. For example: 

This directory contains feedback files for iRefIndex release 9.0.

Please see http://irefindex.uio.no/wiki/README_iRefIndex_Feedback_9.0

The symbolic link indicating the location of the current release needs to be updated. For iRefIndex 9.0 this is done as follows: 

cd /mn/biotroll/ftp/proteas/irefindex/feedback
rm current
ln -s archive/release_9.0 current

Finally, the release file must be updated to indicate the current release number. For iRefIndex 9.0 this contains the following text: 

release9.0

Updating the Wiki

The iRefIndex page will need updating with version-specific references such as the following:

  • README_iRefIndex_MITAB_x.y
  • Sources_iRefIndex_x.y
  • README_iRefIndex_Feedback_x.y
  • Statistics_iRefIndex_x.y

New documents will need creating for some of these references, and general redirect pages should be updated to point to the latest versions of the above pages.

For new data sources, the text will need editing and a citation reference should be added for the new source.

iRefScape

Data for iRefScape must be prepared as described on the iRefIndex Data Preparation for iRefScape page.

iRefWeb

Data for iRefWeb must be prepared as described on the iRefIndex Data Preparation for iRefWeb page.

iRefWeb LinkOut Data

The iRefWeb LinkOut Generator needs to be run and the generated data submitted to the LinkOut FTP site once iRefWeb has been updated.

All iRefIndex Pages

Follow this link for a listing of all iRefIndex related pages (archived and current).

Retrieved from "http://irefindex.vib.be/wiki/index.php?title=iRefIndex_Release_Process&oldid=3890"