Second Challenge FAQ
1. Why is the Challenge focused on molecular simulation?
In short, the focus of the Challenge is currently molecular simulation (in terms of the competition for prizes), both this time and in the first Challenge, because a majority of the members of the organizing committee believed this to be the path most likely to achieve the goals of the Challenge. Though there is a diversity of opinion within the committee concerning what requirements should be placed on allowed methods, the current format represents the consensus of discussions among the committee. It is likely that the scope and format of future Challenges will continue to evolve as individuals join or leave the organizing committee, as the organizing community receives feedback from the scientific community, and as the organizing committee revises its collective opinion about the best path to achieving its goals (and even exactly what its goals should be).
What follows is further amplification and explanation, including a description of the history and goals of the Challenge. We ask that interested individuals read this answer to understand where the committee is coming from and why the current Challenge format was chosen.
Definition of Terms
Before beginning further discussion, let us first define some terms. “Computational chemistry” (CC), “molecular modeling” (MolMod), “molecular mechanics” (MolMech), and “molecular simulation” (MS) are four subjects/terms that are not mutually exclusive. As these terms are commonly employed, CC, MolMech, and MS fall within the category of MolMod. In some circles, CC is a synonym for ab initio-type methods that explicitly treat the electronic degrees of freedom of molecules (e.g., ab initio quantum mechanics and density functional theory). At other times, CC is used as a more general term that encompasses both ab initio methods as well as methods that rely on the use of model potentials or force fields (typically partly empirical and partly ab initio in origin) to describe the interactions between atoms. As the term is commonly used, MS is rather well-defined. Typically, MS refers to any method that involves an ensemble of many molecules whose coordinates are "evolved in accordance with a rigorous calculation of intermolecular energies or forces" (R.J. Sadus, Molecular Simulation of Fluids, Elsevier, 2002). MS is typically divided into two subsets, molecular dynamics (MD) and Monte Carlo (MC), though of course hybrid methods do exist. This definition for MS would include some ab initio methods (e.g., ab initio molecular dynamics). MolMech is used in some contexts to describe a set of methods that typically use force fields to determine, for example, the minimum energy configuration of a single molecule. In that context, MolMech is similar to MS in that they both rely on force fields (potential models). They are distinct in that MolMech typically only seeks to determine a minimum energy configuration while MS attempts to sample the properties of many molecules by rigorously following trajectories in phase space and computing time (MD) or ensemble (MC) averages. In other contexts, the term MolMech is used more broadly to include all force-field-based methods.
History of the IFPSC
The Industrial Fluid Properties Simulation Challenge was first suggested to scientists from The Dow Chemical Company by an employee of NIST during a discussion concerning possible opportunities for collaboration in the area of molecular simulation. Subsequently, a workshop (the “Workshop on Predicting the Thermophysical Properties of Fluids by Molecular Simulation”) was organized by scientists from a variety of companies, national labs, and universities and held at NIST in Gaithersburg, MD, in May of 2001. As is clear from the workshop web site, its focus was primarily molecular simulation and “…to initiate a programmatic effort directed at stimulating further research in the development and validation of force fields and methods for molecular simulation” (quote taken from the “Introduction” on the workshop web site). The initial detailed plans for the Simulation Challenge were created during a break-out session at the workshop. Subsequently, an organizing committee formed and met periodically to define, administer, and judge the first Simulation Challenge. During the course of the first Challenge, inquiries were made by several practitioners of non-MS methods about whether or not they were eligible to enter. The organizing committee used its best judgment and made decisions on a case-by-case basis. Some types of non-MS methods were allowed (e.g., a continuum solvation model method) and others were not (e.g., an equation of state with a rigorous basis in statistical mechanics). In fact, a non-MS method was named champion of one of the challenge categories. As the committee began to make plans for the second Simulation Challenge, the decision was made to explicitly declare the requirements for allowable methods in order to make it clear which methods are allowed and to remove the ambiguity inherent in making the decision on a case-by-case basis (e.g., allowing continuum solvation but not equation of state methods). There was significant discussion and even disagreement among the organizing committee members about where and how to draw the line “cleanly,” so to speak. In the end, the committee arrived at the consensus represented by the format of the Second Industrial Fluid Properties Simulation Challenge.
The goal is not to evaluate the full gamut of methods
The current primary goal of the Second Simulation Challenge IS NOT to compare and evaluate the full gamut of available computational methods for physical property prediction (including MS, CC, group contribution, equation of state, correlation, back-of-the-envelope, etc.). Such a comprehensive evaluation is, by default, one of the primary jobs of an industrial scientist who is responsible for providing physical property data to his colleagues. It is impossible to meet all of the needs by experimentation. A wide variety of computational methods are used to supplement experiment, and the industrial scientist must maintain a fairly high level of awareness about the full range of methods available within the scientific community, including the strengths and weaknesses of each (such as expected accuracy, level of reliance on experimental data, level of expertise required, computational expense, whether or not it is implemented in a user-friendly tool, etc.). The industrial scientist chooses whatever method is available that can provide an acceptable level of accuracy while requiring an acceptable level of time, effort, and expertise. Not coincidentally, industrial scientists rarely make major advances in the development of algorithms, methods, or force fields for molecular simulation. In general, such contributions are not possible because, within the constraints of his/her job role, the industrial scientist is unable to devote the focused effort that would be required. Instead, industrial scientists rely on academics to make advances in the development of algorithms, methods, or force fields.
The goal is to compare molecular simulation methods
The current primary goal of the Second Simulation Challenge IS to compare and evaluate various methods in the field of molecular simulation. This is not because the organizing committee believes MS currently has more practical relevance than the other available methods. Each tool has its place in the tool belt of the industrial scientist. In fact, it is quite likely that each company represented on the organizing committee has one or more employees that are practitioners of each of the following methods including correlative, group contribution, equation of state, quantum mechanics, COSMO, etc. It is also likely that most if not all of the previously-mentioned methods are used more often and with more practical success than classical molecular simulations at each of the companies represented on the organizing committee. That, to a certain extent, is the point.
The industrial members of the organizing committee view MS as a very promising field. With MS, essentially all thermodynamic properties of fluids, including phase equilibria and derived properties, can be calculated at all fluid densities in a consistent manner, without additional simplifying assumptions and models. Bridging the gap between experimentation and analytical theory, MS enables a unique molecular-level analysis of an enormous range of properties of both gaseous and condensed phases ranging from the bulk properties of the small molecules that have provided the foundation for the chemical industry for many decades to the molecular nanotechnology believed by many to represent the next industrial revolution. However, the potential benefits of MS have not been realized in the chemical industry despite several decades of development in academia. There are many circumstances that contribute to this situation. MS is still very demanding in terms of the required expertise and computational resources. There is a general lack of knowledge regarding the level of accuracy that can be routinely expected from MS for the range of properties and chemistries that are relevant in an industrial setting. Most current industrial problems of interest involve multiple components, multiple phases, multiple relevant length and time scales, and/or multiple properties that must be considered simultaneously. It is not yet clear that the field of MS has advanced sufficiently to meet these challenges in a practical industrial setting. Though numerous force fields are available in the literature, it is not uncommon for an industrial scientist to find that he or she is unable to find the necessary force field parameters for the system of interest, even if one removes the requirement that parameters provide sufficient quantitative accuracy for the problem at hand. For most academics, force field development for its own sake is not an exciting use of their resources, in part because it does not motivate the various funding agencies to provide additional resources in the future. Though a multitude of different MS methods and techniques are known within the scientific community that can (in theory) predict the types of properties of interest to the industrial scientist, it is usually not clear how well these methods will perform in practice on real industrial problems (especially if the answer is not known ahead of time), nor is it clear how to make sure that new methods are transferred to industry in a timely manner without relying on commercial enterprises that are motivated primarily by profit (as they should be) and are typically only able to make methods available some 5 to 10 years or more after they are first introduced.
Therefore, the current goal of the Second Industrial Fluid Properties Simulation Challenge IS to evaluate and benchmark the available MS methods and force fields on problems that have significant industrial relevance and, in the process, encourage the continued development of better algorithms, methods, and force fields while improving the alignment of academic efforts with industrial needs. The consensus of the committee is that the best way to achieve this goal is to make MS the primary focus of the simulation challenge. On the other hand, we don’t want to cover our eyes and plug our ears to the possibility of new or especially promising methods that do not fit the common definition of a MS. Therefore, we invite practitioners of any method to tackle the contest problems and submit an entry to the organizing committee. It is likely that practitioners of non-MS methods will be included in the various reports, conference symposia, and publications that are associated with the Simulation Challenge. However, non-MS methods will not be considered in the selection of Champions. This is believed by the organizing committee to be a reasonable compromise that maintains primary focus on MS while leaving the door open for potentially-valuable input by practitioners of other methods.
Conclusion
For the various reasons outlined in this document, molecular simulation continues to be the primary focus of the simulation challenge. The organizing committee hopes that this rather lengthy description of the rational for the decisions we have made is clear and informative. The decisions were not arbitrary nor formulated with the goal of excluding any particular individuals or methods. Rather, the decisions were made in a good-faith effort to achieve the overall goals of the Challenge. As mentioned earlier in this discussion, the committee lacked unanimity on these issues but was able to achieve consensus. When plans are made for the third and subsequent Challenges, these issues will be considered again. Therefore, we solicit the support and participation of the community in the current Challenge and welcome feedback and/or criticism, as long as it is constructive, with the goal of continued improvement in the future.
2a. Problem 2 – Determination of Henry’s Law Constant (HLC)
Q: The normal boiling point (NBP) of ethanol is 351 K. Thus, it’s somewhat strange to ask for a solubility in LIQUID ethanol at a temperature of 373 K…?
A: The NBP temperature is an arbitrary quantity related to environmental conditions on the planet Earth. The contest problem is to determine the Henry’s Law constant (HLC) which is the limit of the slope of the gas partial pressure (actually fugacity) with respect to liquid phase composition of the dissolved gas. The reference pressures for the HLCs in Problem 2 are the saturation pressures of the solvent at the two defined temperatures. HLCs can be realized at any temperature below the critical temperature of the solvent, albeit in a metal apparatus above the NBP. See, for example, HLCs defined above the NBP in Journal of Chemical and Engineering Data, 22, 326-329 (1977).
2b. Problem 2 – Determination of Henry’s Law Constant (HLC)
Q: At what pressure should the Henry’s Law Constants (HLC) be reported for Problem 2?
A: The reference pressures for the HLCs in Problem 2 are the saturation pressures of the solvent at the two defined temperatures. The full designation for the HLC is,
H2,1(T, Pref) where Pref = Psat, the saturation pressure of ethanol at T
The reference pressures for the two temperatures in Problem 2 are:
T = 323 K, Psat = 29.4 kPa
T = 373 K, Psat = 223.4 kPa
3. Who can enter?
The challenge is open to researchers from academia, government laboratories, and industry not affiliated with the organizing committee. Participants must register at the web site to ensure that their proposed methodology is eligible. There are no geographical restrictions regarding eligibility, and attendance at the AIChE Annual Meeting (where the champions will be announced) is not a requirement.
Are there any limitations on non-US entries?
No, there are none except that to be eligible for prizes, the participants may not be employed by any of the organizations represented on the organizing committee.
4. Can I use force field parameters that have not yet been published?
Yes. For a given problem, any force field parameters may be used if the origin of the parameters does not violate the restrictions regarding what experimental may be used for that given problem. Additionally, if the force-field parameters were published in a peer-reviewed journal by the date of the Contest announcement, they may be used even if their origin violates the other contest restrictions. All force field parameters that are used must be reported in the manuscript submitted to the contest committee.
5. Are the problems easier this time? If so, why?
Maybe. Whether or not the contest is easier this time is a matter of opinion. The committee is eager to increase the number of entries while maintaining problems that are challenging. When the general problem descriptions (without the specific molecular identities) were evaluated by academic experts, the experts who responded indicated that the problems were challenging to very difficult to do accurately and would require about the right amount of effort. Therefore, the committee concluded that they were a reasonable set of problems for the second challenge.
6. Why not include transferability in the judging criteria?
Transferable force fields are certainly highly desirable, including transferability between molecules, properties, and state conditions. For the current challenge, we chose to focus on the aspect that is most straightforward to accomplish and to judge, transferability among state conditions. By restricting what experimental data can be used as input to the force field parameterization process, we are able to include this aspect in the problems without including it explicitly in the judging (scoring) criteria. This was desirable in order to facilitate an unambiguous and straightforward process for scoring the entries. Future challenges will likely address transferability between molecules and/or between properties. It is also likely that they will be incorporated into the definition of the problems and their restrictions rather than including them explicitly in the judging (scoring) criteria.
7. What are the prizes?
The details of the current plan for awarding monetary prizes are as follows: For each of the three challenge problems, $2500 will be awarded for first place, $1500 will be awarded for second, and $1000 will be divided equally among all of the other entries that meet the minimum requirements of the Challenge. A special award of $1000 will be given to "best-in-show," the entry that performs the best among those that enter all three Challenge problems. These plans are subject to change at the discretion of the organizing committee. For example, prizes may not be awarded if the level of participation is extremely small (e.g., if there are fewer valid entries than prizes for a particular problem) or if the quality of competition is deemed to be especially poor (e.g., if all of the entries for a problem are deemed to be of poor quality and not worthy of special recognition).
8. What are the benefits of participating?
Participation in the Challenge gives you an opportunity to test your models and methods on problems that have significant industrial relevance and for which you don't know the answer ahead of time. A special session at the 2004 AIChE annual meeting will be devoted to the Challenge and offers a prime opportunity for you to highlight your work, as does the special journal issue devoted to the reporting the results of the Challenge. As mentioned in the previous section, cash prizes will be awarded to all entries that meet the minimum requirements of the Challenge. Though the organizing committee intends the Simulation Challenge to be a vehicle for engagement between industry and the top practitioners in the field of molecular simulation, it also represents a good opportunity for graduate students to gain experience and exposure (and a cash prize). If you are a graduate student and are interested in any of these benefits, talk to your advisor about entering the Challenge. If you are a professor or group leader, you might consider using a problem or problems from the Simulation Challenge to "initiate" new students to the world of molecular simulation. If you are teaching a graduate course in statistical mechanics, molecular modeling, molecular simulation, or another closely related field, you might consider using the Simulation Challenge as a group project for your students.
9. Who organizes the Challenge?
The competition is organized by scientists from 3M, BP, Case Scientific, The Dow Chemical Company, DuPont, ExxonMobil, Mitsubishi Chemical, and the National Institute of Standards and Technology.
10. Who will judge the Challenge?
The judging will be performed by a panel of experts in the field.
11. For Problem 3, is it permissible to perform simulations at a pressure other than 101.325 kPa?
Yes. The pressure of 101.325 kPa specified in parts 3-1 and 3-2 precisely defines the state point of interest. However, the actual pressures used in the experimental measurements are likely to be slightly different, and it may be necessary to make small corrections to obtain the value at 101.325 kPa. Likewise, it is permissible for simulators to use other values of pressure in their calculations for this problem. Since the heat of mixing has no significant pressure-dependence within a reasonable range of pressure around 101.325 kPa, small changes in the pressure will not affect the results but may have practical experimental or computational advantages and, therefore, are permissible.
12. Determination of Benchmark Data
The answers to the contest problems will be determined by a combination of new, unpublished experimental measurements (conducted for the Challenge) and critically evaluated literature data (if such data exist). The experimental data will be measured by techniques consistent with the determination of reliable benchmark property data. Members of the Benchmark Data Committee for the contest are highly experienced in these issues, including members from NIST and industrial labs.
Examples of how these data were determined for the first Challenge may be found in the recently published special volume of Fluid Phase Equilibria:
Volume 217, Issue 1, Pages 11-51 (March 10, 2004)
