To assess how reliable predictions and
designs made based on models turn out to be, in
2009 we initiated a blinded study where nine unpublished high-resolution x-ray
Fab crystal structures covering a wide range of antigen-binding site
conformations were used as benchmark to compare Fv models generated by four
structure prediction methodologies. The methodologies included two homology
modeling strategies independently developed by The Chemical Computer Group
(CCG) and Accerlys Inc (ACC), and two fully automated antibody modeling
servers: PIGS (Prediction of ImmunoGlobulin Structure), based on the canonical
structure model, and Rosetta Antibody Modeling (ROS), based on homology
modeling and Rosetta structure prediction methodology.
structure sequences were submitted to Accelrys and CCG and a set of models for
each of the nine antibody structures were generated. PIGS and Rosetta models
were obtained using the default parameters of the servers. In most cases, we
found good agreement between the models and x-ray structures. The average rmsd
(root mean square deviation) values calculated over the backbone atoms between
the models and structures were fairly consistent, around 1.2 Å. Average rmsd
values of the framework and hypervariable loops with canonical structures (L1,
L2, L3, H1, and H2) were close to 1.0 Å. H3 prediction yielded rmsd values
around 3.0 Å for most of the models.
Antibody 3D modeling methods continued to evolve since AMA-I and the number of antibody structures used as template for modeling increased significantly since the first assessment. For instance, between July 2009 and July 2012, 256 new antibody structures including Fvs, scFvs and Fabs were deposited in the PDB, representing 22% of the total number of structures (1,202) available in the PDB. Hence, to monitor the evolution of the antibody 3D modeling methodologies we launched AMA-II in 2012.
Eleven unpublished high-resolution x-ray Fab crystal structures from diverse species and covering a wide range of antigen-binding site conformations were used as benchmark to compare Fv models generated by seven structure prediction methodologies. The participants included: Accerlys Inc (ACC), Chemical Computer Group (CCG), Schrodinger (SCH), Jeff Gray’s lab at John Hopkins University (JEF), Macromoltek (MMT), Astellas Pharma/Osaka University (JOA) and Prediction of ImmunoGlobulin Structure (PIGS).
The rabbit structure (see Ab01 in Structures) represented a remarkable challenge for the modelers. The rmsd of the Fv models for this antibody ranged from 1.7 to 2.8 Å, with rmsd values above 3.0 Å for three hypervariable loops: L1, H1 and H3. For the other benchmark structures, the average rmsd value for the Fv generated by all the seven methods and all models is 1.1 Å. The average rmsd value for the framework was 0.9 Å. Average rmsd values of the framework and hypervariable loops with canonical structures were 1.1 Å. H3 yielded an average rmsd value of 2.8 Å, which represents an improvement with respect to AMA-I.
Other Achievements and Future Plans
The main outcome of AMA-II has been the
expansion of the discussion forum for antibody engineers and antibody modeling
method developers. Information has been shared among the groups through:
- Teleconferences and a
face-to-face meeting at the conclusion of the modeling studies.
- A workshop at the 2013 IBC
Antibody Engineering and Therapeutics conference was organized in
conjunction with The Antibody Society - a summary of the Workshop can be found here.
- Preparation of a special
issue of Proteins, which provides a detailed description of several 3D
modeling protocols available to the antibody engineering community - see
We plan to continue this initiative of antibody modeling
assessment with yet more challenging structures, including antigen-antibody
complexes, and refined modeling methods in the near future.