The newest model of Human-GEM gives 0% Stoichiometric Consistency when running MEMOTE

[GuilleDufortFing]

The following are the results of running the latest version of MEMOTE (0.17.0) on the latest version of the Human-GEM.xml (v1.19.0 commit "chore: new version"):

index.html.zip

Citing the paper An atlas of human metabolism:
"The quality of Human1 was evaluated using Memote, a communitymaintained framework for assessing GEMs with a standardized set of tests and metrics (22). In terms of consistency, Human1 exhibited excellent performance with 100% stoichiometric consistency, 99.4% mass-balanced reactions, and 98.2% charge-balanced reactions (fig. S3)."

I was wondering if you could give me some insight into why this discrepancy between the text and my results might be happening.

Thank you very much,
Guillermo

[JHL-452b]

Sorry to see this.

From the release of Human1 to Human-GEM 1.19, the model has changed a lot. Although we have tried to correct the Stoichiometric Consistency in #814 and #813 , it seems to be still failed from the the results you provided. The report shows we got 4410 Unconserved Metabolites in Human-GEM. It's terrible. We need some time to test what's wrong with Human-GEM. Will get back soon.

[JonathanRob]

Even though it shows a ton of unconserved metabolites and a very low stoichiometric consistency, it could be as simple as one reaction being unbalanced. I would recommend taking a look at which reactions have changed since the drop in consistency score, and trying find any that may have become mass-unbalanced.

[JHL-452b]

Yeah. I will take a look at mass-unbalanced reactions. But I remember that Human-GEM is consistency after #814 and #813 . Perhaps there have been some issues with recent PRs. Anyway, it doesn't look like it will be too hard to resolve.

[JHL-452b]

Hi. By checking the unbalanced reactions one by one, I found Human-GEM could be consistent after turn MAR12372 into balance.

ID	Reaction	Cause of imbalance
MAR12372	2 CoA + FAD + 2 H2O + 2 NAD+ + 3'-S-hydroxy-pravastatin-CoA --> 2 acetyl-CoA + 2 FADH2 + 2 H+ + 2 NADH + 3'-S-hydroxy-pravastatin-tetranor-CoA	{'charge': -2, 'C': 27, 'H': 35, 'N': 9, 'O': 13, 'P': 2}

This reaction could be balanced after remove FAD in reactants and FADH2 in products. Although this reaction was already detected in #813 , it was not successfully modified in Human-GEM due to my negligence. It will be curated in a new PR in soon.
Here is the report index.zip

[JHL-452b]

Indeed, I did not generate the memote report directly because it would take a long time to run during this check. The check_stoichiometric_consistency function in memote can be used directly to get the results faster.

import memote.support.consistency
model = read_sbml_model('Human-GEM.xml')
memote.support.consistency.check_stoichiometric_consistency(model)

If we add this code to the check tasks after every PR merge, Human-GEM will not have this problem in the future.
But the current problem with this code is that the Human-GEM.xml cannot be read by cobrapy. It reported the error as below after using validate_sbml_model function as cobrapy recommend to check model's problem.

(None,
 {'SBML_FATAL': [],
  'SBML_ERROR': ['E0 (Error): Operating system (core, L1); File unreadable; File unreadable.\n'],
  'SBML_SCHEMA_ERROR': [],
  'SBML_WARNING': [],
  'COBRA_FATAL': [],
  'COBRA_ERROR': ['No SBML model detected in file.'],
  'COBRA_WARNING': [],
  'COBRA_CHECK': []})

What did I done is to add prefix of 'M_' for mets id and 'R_' for rxns id in xml format of Human-GEM so that it could be read by cobrapy. I don't know whether this is reasonable, but if it is, we need to modify the code of the model output XML format.

[mihai-sysbio]

There is a lot more history regarding the renaming of metabolites and reactions, see SysBioChalmers/RAVEN#353

There is a way to run memote directly on the yml here, thus avoiding the xml issue above.

[edkerk]

As shown in the memote workflow referred to in the previous comment, cobrapy can read the Human-GEM.yml file directly with model = cobra.io.load_yaml_model("model/Human-GEM.yml").

At the same time, on my PC (cobrapy 0.29) there seems no problem with loading the XML file. validate_sbml_model only throws a warning:

(<Model HumanGEM at 0x7fbe64166b00>,
 {'COBRA_CHECK': [],
  'COBRA_ERROR': [],
  'COBRA_FATAL': [],
  'COBRA_WARNING': [],
  'SBML_ERROR': [],
  'SBML_FATAL': [],
  'SBML_SCHEMA_ERROR': [],
  'SBML_WARNING': ['E0 (Warning): General SBML conformance (core, L3); RDF '
                   'does not contain valid ModelHistory; LibSBML expected to '
                   'read the annotation into a ModelHistory object. '
                   'Unfortunately, some attributes were not present or correct '
                   'and the resulting ModelHistory object will not correctly '
                   'produce the annotation.  This functionality will be '
                   'improved in later versions of libSBML. \n'
                   'Reference: L3V1 Section 6.3\n'
                   ' An invalid ModelHistory element has been stored.\n',
                   'E1 (Warning): General SBML conformance (core, L3); RDF '
                   'does not contain valid ModelHistory; LibSBML expected to '
                   'read the annotation into a ModelHistory object. '
                   'Unfortunately, some attributes were not present or correct '
                   'and the resulting ModelHistory object will not correctly '
                   'produce the annotation.  This functionality will be '
                   'improved in later versions of libSBML. \n'
                   'Reference: L3V1 Section 6.3\n'
                   ' An invalid ModelHistory element has been stored.\n']})

And this warning is a result of Human-GEM still being exported with RAVEN from SysBioChalmers/RAVEN#353, as an artifact of not having fully resolved that PR yet. It is also just a warning, so does not preclude loading the model. The latest merge commit from develop to that branch in RAVEN should also already have resolved the issue of this warning.

[JHL-452b]

Yes, after I updated cobrapy to 0.29.0 in my conda environment, there is no error for loading Human-GEM.xml through read_sbml_model. Thank you very much.

Anyway, as mentioned above, MAM01802x (FAD) and MAM01803x (FADH2) in MAR12372 should be removed for Stoichiometric Consistency of Human-GEM. I will fix this right away. However, this reaction has been modified in the develop, and Human-GEM in the main will be consistent when the next version released.