We introduced ordered and unordered beam search algorithms, staples from computer science, to test how fixing the order of sequence edits compares to a more flexible, random-order approach. We also created Gradient Evo, a novel hybrid that enhances the directed evolution algorithm by using model gradients to guide its mutations to independently evaluate how important gradients were for edit location selection versus selecting a specific edit.
We also developed AdaBeam, a hybrid adaptive beam search algorithm that combines the most effective elements of unordered beam search with AdaLead, a top-performing, non-gradient design algorithm. Adaptive search algorithms don’t typically explore randomly; instead, their behavior changes as a result of the search to focus their efforts on the most promising areas of the sequence space. AdaBeam’s hybrid approach maintains a “beam”, or a collection of the best candidate sequences found so far, and greedily expands on particularly promising candidates until they’ve been sufficiently explored.
In practice, AdaBeam begins with a population of candidate sequences and their scores. In each round, it first selects a small group of the highest-scoring sequences to act as “parents”. For each parent, AdaBeam generates a new set of “child” sequences by making a random number of random-but-guided mutations. It then follows a short, greedy exploration path, allowing the algorithm to quickly “walk uphill” in the fitness landscape. After sufficient exploration, all the newly generated children are pooled together, and the algorithm selects the absolute best ones to form the starting population for the next round, repeating the cycle. This process of adaptive selection and targeted mutation allows AdaBeam to efficiently focus on high-performing sequences.
Computer-assisted design tasks pose difficult engineering problems, owing to the incredibly large search space. These difficulties become more acute as we attempt to design longer sequences, such as mRNA sequences, and use modern, large neural networks to guide the design. AdaBeam is particularly efficient on long sequences by using fixed-compute probabilistic sampling instead of computations that scale with sequence length. To enable AdaBeam to work with large models, we reduce peak memory consumption during design by introducing a trick we call “gradient concatenation.” However, existing design algorithms that don’t have these features have difficulty scaling to long sequences and large models. Gradient-based algorithms are particularly affected. To facilitate a fair comparison, we limit the length of the designed sequences, even though AdaBeam can scale longer and larger. For example, even though the DNA expression prediction model Enformer runs on ~200K nucleotide sequences, we limit design to just 256 nucleotides.
