Variant effect predictors: AVE's resource expands into non-coding territory

AVE's Analysis, Modelling and Prediction (AMP) workstream has recently delivered a major expansion of one of its most-used tools: a resource cataloguing computational models for predicting variant effects. Previously covering coding variants, the resource now also covers non-coding predictors, an area of interest which is growing rapidly. We spoke to Prof. Joe Marsh (Institute of Genetics and Cancer, University of Edinburgh) about how the resource was developed, what makes non-coding predictors so different from their coding counterparts, and whether VEPs will ever replace deep mutational scanning.

Where did this resource come from originally, and what need was it filling?

“My group is really interested in computational models for predicting variant effects, and this field is heterogeneous and scattered. There are lots of new methods coming out and called different things by different people. It's hard to keep track of everything that's available.

Ben Livesey, who was a PhD student in my lab at the time, had been compiling scores from a variety of different methods as part of his research, and we decided to extend that into a proper resource. When we first released it, two or three years ago, it was focused on coding predictors; that is, things that predict the effects of sequence changes in proteins, since that's where most of the field's work has been. We put together a page with some background on the methods, how to use and interpret them, and a big table of well over 100 different methods with links, classifications and details.

It's turned out to be genuinely useful for the field [Editor’s note: it is one of the most accessed parts of the AVE web site], and if you search ‘variant effect predictors’, the page ranks very highly on Google. When we later published our guidelines on releasing variant effect predictors, we described the resource there too, partly to give it a proper citable reference.”

And now you've expanded into non-coding predictors. What was the driver behind that?

“The original resource was very focused on protein missense, coding variants. But there's been increasing interest in non-coding variant interpretation, so we decided to do the same thing for that area. A second-year PhD student in my lab, Tesni Walsh, has been compiling a list of non-coding methods as part of her project, classifying them by what information they use and what they output.

Non-coding predictors are quite different from coding ones. Coding predictors have mostly been applied to one question: “is this pathogenic or not?”. That's harder to ask of non-coding variants, so some non-coding methods aim to give a general measure of variant deleteriousness, while others predict specific molecular consequences, like splicing, transcription factor binding or chromatin accessibility. Then you get something like AlphaGenome, Google DeepMind's model, which predicts a whole range of different properties at once. So, it took a bit more work to classify and organize these methods sensibly. We've also updated the resource page with more general information on coding versus non-coding predictors and how they differ.

The community can now start using it and also send us their own methods to add, in the same way some groups have already done for the coding resource.”

Is it possible that variant effect predictors might one day replace deep mutational scanning (DMS) experiments?

"VEPs and DMS experiments are absolutely complementary, and I don't see that changing any time soon. On one hand, high-throughput DMS datasets are increasingly important for training and validating the next generation of computational models; if we want to predict well, we need that experimental data first. On the other hand, these assays can measure a lot more than most computational models currently capture. Many predictors are quite generic, a lot of them don't account for things like cell-type specificity. Often, they're leaning on evolutionary information, which tells you something has been selected for or against, but nothing about the mechanism behind it. Experimental approaches will continue to have scope to give us that mechanistic detail that predictors often can't currently provide.

People sometimes compare this moment to AlphaFold and protein structure prediction, but it's a very different situation. With structure prediction, you have a clear three-dimensional target, so there is a relatively well-defined target against which you can assess whether a model got it right. Predicting variant effects is much vaguer; the effects are often subtle, and we don't have the same kind of ground truth that structural biology has. So, I'm not worried, at least in the short term, about AI solving this problem outright.”

What's next for the VEP resource?

“One thing we've been talking about is getting better at making predictor scores actually available, not just linking out to methods. As it stands, we link to a method and, where a team provides a way to run it or download scores, we link to that too — but there's a lot of variability in how people share their data, predictions and methods. Our guidelines paper tried to address some of that.

What we're working towards now is a more standardized format for sharing variant effect predictions, and a more centralized resource where people can deposit their scores — partly to encourage people to make predictions available when they release new methods, and partly as a resource for people who just want to have the scores and use them.”

---------------------------------------------------------------

Find the Variant Effect Predictors, developed by the Marsh lab as part of AVE’s AMP workstream at: https://www.varianteffect.org/veps/ 

Click here to submit a new variant effect predictor for the resource.

---------------------------------------------------------------

Image: Ben Livesey (left) and Tesni Walsh (right), Marsh lab, Institute of Genetics and Cancer, University of Edinburgh.