Large Language models (LLMs) are increasingly being use in clinical research, offering potential applications ranging from evidence synthesis and protocol development to data extraction. Benjamin Ineichen explores the opportunity and limitations of these tools, with a particular focus on the challenges of evaluating their performance. He also highlights the gap between the potential capabilities of LLMs and their demonstrated clinical value, underscoring the need for rigorous evaluation before their broader adoption in clinical research.