Google DeepMind, the UK-based research company, recently unveiled a new study exploring a technique for better-exploiting inference time compute.
Google DeepMind’s technique, ‘Mind Evolution’, uses a language model to generate a diverse population of candidate solutions and then recombine and refine these based on feedback from an evaluator.
Unlike sequential reasoning approaches, such as self-refinement or tree search, which require evaluation of individual reasoning steps, Mind Evolution performs global refinement of complete solutions, the authors claimed.
The TravelPlanner benchmark evaluates a model’s ability to organise a trip plan ‘for a user who expresses preferences and constraints’. Across multiple levels of travel planning difficulties, the Mind Evolution technique outperformed the others.
In the Meeting Planning task, a model is evaluated for its ability to schedule meetings based on constraints like number of people to meet, availability, location, and travel time.
The authors also suggested that this task differs from TravelPlanner tasks as not all meetings can be scheduled due to conflicts in constraints like availability or location.
The results shown demonstrate significant performance for Mind Evolution over baseline strategies, achieving an 85.0% success rate on the validation set and 83.8% on the test set. Notably, the two-stage approach using Gemini 1.5 Pro achieves success rates of 98.4% and 98.2% on validation and test, respectively.
That said, the authors also acknowledged that the ‘main limitation’ of Mind Evolution is that it focuses mostly on natural language planning problems, where ‘proposed solutions can be programmatically evaluated and critiqued.’
Inference time compute is a concept that has been widely used in large language models, especially in OpenAI’s o1 reasoning models. This technique has been regarded as an effective method to solve the scaling problem in large language models.
A few days ago, Google DeepMind also published a study that introduces inference time scaling for diffusion models.
The research titled ‘Inference-Time Scaling for Diffusion Models Beyond Scaling Denoising Steps’ explores the impact of providing additional computing resources to image generation models while they generate results.
In December last year, Google unveiled the Gemini 2.0 Flash Thinking model. The model offers advanced reasoning capabilities and showcases its thoughts. Logan Kilpatrick, senior product manager at Google, said the model “unlocks stronger reasoning capabilities and shows its thoughts.”
