4th Challenge Announcement
Introduction and Background
Researchers working in an industrial setting are commonly asked to predict a wide range of physical properties. A method that is able to predict a broad range of properties (especially properties that were not used in the original model parameterization) may be more valuable in this situation than a method that may provide more accurate results but only for one property or property type.
The primary objective of the Fourth Industrial Fluid Properties Simulation Challenge is to test the transferability of methods and force fields to a wide variety of properties for a given small molecule. There will be two categories of competition: 1) "molecular simulation" methods and 2) "other methods." A champion will be announced for each of the two categories.
A secondary objective of the molecular simulation category is to assess the variability of property predictions performed by different researchers using the same force field (a "round-robin" experiment, to be explained more fully below).
Timeline
November 17, 2006 - Problem announced
September 30, 2007 - Entries due
November 2007 - Champions announced
Molecule
Ethylene oxide (EO)
Why EO?
- It has significant industrial relevance.
- It is flammable, reactive, and toxic - so it is a good candidate for avoiding experimentation via use of modeling.
- It is a small molecule, so it may be less computationally-demanding to calculate its properties compared to larger molecules.
Typical uses and reaction conditions for EO:
| Reaction | T (°C) | P (bar) | Ref. |
| Oxidation of ethylene to form EO | 250-300 | 10-20 | 1 |
| Reaction of EO with aqueous ammonia to form ethanolamines | 60-150 | 30-150 | 1 |
| Vapor phase hydrolysis of EO to form ethylene glycol | 140-230 | 20-40 | 1 |
| Reaction of ethanol with EO to form ethylene glycol monoethyl ether | 170-190 | 10-15 | 2 |
World capacity for EO (1995, Ref. 2): 11.2 x 106 tons/year
1 H. Ulrich, Raw Materials for Industrial Polymers (Hanser, 1988).
2 K. Weissermel and H.-J. Arpe, Industrial Organic Chemistry (VCH, 1997).
Challenge
Molecular Simulation Category
Calculate any of the properties specified below for:
1 - a model (i.e., force field) for EO that you have developed
and
2 - the "round-robin model" for EO: united atom model "A" from the paper by Wielopolski and Smith, "Molecular dynamics studies of dielectric behavior and orientational correlations of liquid ethylene oxide (oxirane)." Mol. Phys., 54, 467-478 (1985).
Note: There are some typos in the force field description in the paper by Wielopolski and Smith. Please see the FAQ below for clarification on those typos.
Other Methods Category
Calculate any of the properties specified below for EO
Properties
Category 1 (100 points total)
1. saturated liquid phase density at 375 K (11 points max)
2. saturated vapor phase density at 375 K (11 points max)
3. second virial coefficient at 375 K (11 points max)
4. vapor pressure at 375 K (11 points max)
5. heat of vaporization at 375 K (11 points max)
6. normal boiling temperature at 101.325 kPa (15 points max)
7. critical density (15 points max)
8. critical temperature (15 points max)
Category 2 (100 points total)
1. saturated liquid phase heat capacity (Cp) at 375 K (15 points max)
2. saturated vapor phase heat capacity (Cp) at 375 K (15 points max)
3. saturated liquid phase isothermal compressibility at 375 K (20 points max)
4. saturated vapor phase isothermal compressibility at 375 K (20 points max)
5. surface tension at 375 K (30 points max)
Category 3 (100 points total)
1. saturated liquid phase viscosity at 375 K (25 points max)
2. saturated vapor phase viscosity at 375 K (25 points max)
3. saturated liquid phase thermal conductivity at 375 K (25 points max)
4. saturated vapor phase thermal conductivity at 375 K (25 points max)
Awarding of Points
Points will be awarded based on comparison to experimental benchmarks for each property that you predict with the model you developed. You do not have to submit predictions for all of the properties in a category. However, if you are competing in the "molecular simulation" category, then you must submit predictions for both your model and the "round-robin model" for each property you submit.
The maximum number of points available for each property is specified above in the list of property categories.
For each property, a threshold maximum % deviation from the experimental benchmark data will be specified. Predictions with % deviation greater than this threshold will receive zero points.
For each property, an uncertainty in the benchmark data will be determined. Predictions with % deviation smaller than that represented by the uncertainty in the benchmark data will receive full credit (the maximum number of points available for that property).
For predictions falling between the benchmark value and the maximum allowed deviation, points will be awarded based on a pro-rated scale inversely proportional to the percent deviation.
A bonus of 50 points will be awarded if you score greater than zero points for at least one property in all 3 categories since the purpose and spirit of this problem is to test transferability to a wide range of properties.
The entry with the highest points total is the Champion.
An Excel template for scoring an entry is available on the IFPSC web site here (.xls, 34 KB). The "round-robin" predictions will not be an explicit part of the scoring. Instead, they will be used by the judges as a gauge of the validity of the methods used.
Additional Notes/Rules/Requirements
If you are competing in the "molecular simulation" category, your calculations for the round-robin model will be used to establish the validity of your methods and also to provide data on the variability of property predictions performed by different researchers on the same model (a "round-robin" experiment).
Practitioners of other theory/modeling/simulation methods that are unable make predictions for the "round-robin model" are encouraged to submit predictions to the "other methods" category of the competition.
In the "molecular simulation" category, the same force field must be used for all of the property calculations. This means that exactly the same force field parameters must be used for calculating all property predictions that you submit.
In the "other methods" category, the same theory/method/approach/technique must be used for all of the property calculations. Admittedly, this requirement is somewhat ambiguous, but it is necessary in order to maintain the spirit of this competition: testing the transferability of a single model to a wide range of properties. Compliance will be evaluated by the judges on a case-by-case basis, and anyone who is unsure about the meaning of this requirement is encouraged to seek clarification from the competition chair.
There are no limits on what data can be used to fit your model parameters.
If constant NVT simulations are performed, the density value used must be that which is calculated for your model (i.e. is consistent with your force field's equation of state). You may not arbitrarily use a (perhaps experimental) density. You must use the density that your model would predict.
Extrapolations of simulated data are acceptable for predicting the critical properties and normal boiling point. For example, the Clausius-Clapeyron equation is commonly used to calculate normal boiling points from simulation data at higher temperatures and the saturated density scaling law and the law of rectilinear diameters is commonly used to estimate critical properties from simulation data at sub-critical state points.
Entry Submission
Entries should be submitted to contest@ifpsc.org no later than September 30, 2007.
A submission for this challenge problem is to be in the form of a manuscript suitable for submission to a refereed, archival, scientific journal. The manuscript must contain sufficient detail about the prediction method (and about the force field, if applicable) so that an experienced practitioner could reproduce the results without requiring access to proprietary information. In particular, all potential parameters and molecule geometry parameters must be explicitly specified in the manuscript. The results are to be reported in SI units. A randomly selected subset of the submitted predictions may be validated by the judges by reproducing the reported calculations.
An analysis of the uncertainty in the calculated results is required and must be included in the manuscript. Entries are expected to present results that are statistically significant and to present sufficient supporting evidence to establish this quality. Also, the scientific reasoning behind any new (unpublished) force field parameterizations must be clearly spelled out in the entry. If there is a consensus among the judges that an entry is of poor quality (uses a method commonly accepted to be fundamentally flawed, presents results that are not statistically significant, fails to provide sufficient supporting data and details, violates the various rules and guidelines established for the competition, or for any other reason would be unlikely to be accepted by any peer-reviewed scientific journal in the field), that entry will be rejected and will not be considered in the judging.
Entries that represent collaborations between multiple research groups are welcomed.
FAQ
Q: There are some typos in the force field description of the round-robin model in the paper by Wielopolski and Smith. Can you please clarify?
A: Yes, you are right. On page 471, the oxygen charge should be -0.3216. On page 472, the Lennard-Jones size paramter σ(O-O) is incorrectly labeled as a mixed interaction σ(O-C). It is actually the oxygen size parameter.
Q: How do I know in which category ("molecular simulation" or "other methods") I should compete?
A: An "other method" is any method which does not neatly fit into the definition of molecular simulation: Molecular simulation refers to any method that generates a trajectory through phase space for the system of interest by executing a series of deterministic and/or stochastic steps that obey the thermodynamic constraints. (Ilja Siepmann, 2006). If your method is capable of predicting properties of the round-robin model (a specific model for EO from the literature with three Lennard-Jones sites plus point charges), then you should compete in the "molecular simulation" category. If your method is not applicable to the round-robin model, then you should compete in the "other methods" category.
Q: Will you post the experimental benchmark data for the 12 properties ahead of time? Otherwise, there is a danger that we spend significant effort fitting to inaccurate data.
A: We considered providing the data, but we decided not to do so for a couple of reasons. 1) If the experimental data were provided ahead of time, some potential contestants might lose interest because they would consider it to be purely an uninteresting exercise in fitting. 2) It's doubtful that the benchmark team will have all the data evaluated, any required new measurements done, etc. in time to be able to publish the data soon enough to be useful to you. It's true that there is danger of fitting to inaccurate data, but this is always a danger in real life too.
Q: For the molecular simulation category, it would be nice to specify allowable routes for the calculation of the heat capacity, i.e. can we compute the heat capacity from classical sampling of a condensed phase and correct for the difference between classical and quantum mechanical sampling obtained in the ideal gas phase?
A: Yes, that approach would be acceptable.
Q: How are you using the reported uncertainties in the calculated results? When determining the champion, do you make sure that the top entries are really distinguishable from one another given the uncertainties in the calculations?
A: We plan to continue to only use the uncertainty data as a requirement for submission to make sure that an error analysis has been done and that the expert judges believe the results are robust based on that error analysis. In any contest (such as athletic events, for example), both luck and skill have a role in determining the winners. When the skills are nearly evenly matched, the role of luck can be decisive. This is not ideal, but is probably something that we have to live with if we want to have a contest that has a winner.
Existing Literature for Molecular Simulations of EO
1. P. A. Wielopolski and E. R. Smith. "Molecular dynamics studies of dielectric behavior and orientational correlations of liquid ethylene oxide (oxirane)." Mol. Phys., 54, 467-478 (1985).
2. R. D. Mountain. "A Polarizable Model for Ethylene Oxide." J. Phys. Chem. B, 109, 13352-13355 (2005).
3. M. Krishnamurthy, S. Murad and J. D. Olson. "Molecular dynamics simulation of Henry's constant of argon, nitrogen, methane and oxygen in ethylene oxide." Mol. Simul., 32, 11-16 (2006).
