Testing innovations in real-world environments; Testing innovations in real-world environments;

Real-World Testbeds, London

Testing innovation in the real world – how do you design better, more effective testbeds?

Testing how new ideas and technologies work in practice is a critical part of ensuring that they are both fit for purpose and safe before they reach a wider audience. For example, a new medical product, an automated vehicle or a new method of construction all need to be thoroughly tested close to real-world conditions before they can be commercialised.

Real-world testbeds (hereafter called testbeds), are bounded environments tailored to testing specific innovations in close-to real-world conditions similar to the environment in which they would be used. They explore how new ideas and technologies can be applied to solve the world's biggest challenges, but existing evidence is limited.

Nesta commissioned Arup and LSE to carry out a comprehensive review and analysis of existing testbeds around the world to broaden the understanding of their effectiveness as a tool for innovation policy. The aim was to equip public and private sector stakeholders with insights to help them design and implement successful testbeds to facilitate the safe introduction of new technologies. The work resulted in a report including in-depth case studies, recommendations on policy and what to consider when designing effective testbeds.

Project Summary

95 testbeds reviewed globally

6ways of using real-world testbeds established

<10%of reviewed testbeds in the UK deal directly with climate change

Researchers from Arup’s City Economics and Planning team, in collaboration with LSE, were selected to create a report mapping real-world testbeds around the world, how they differed from other innovation tools, and what made them successful in design and policy environment.

Arup reviewed hundreds of innovation tools branded as testbeds and sifted out 95 that operated in real-world environments. As the research was globally-oriented across many sectors, the challenge was identifying the most interesting examples in the sectors and countries where information was less readily available. Our aim was to showcase the traditional uses of testbeds but also the wider potential of using testbeds in fossil-free districts, blockchain technology, tourism and their use in the defence industry. 

The comprehensive research was made significantly less challenging with the support of the global Arup network. Arup's previous experience developing one of the UK's largest testbeds for the NHS and work on UK Autodrive, enabled us to dive deeper into the aims and successes of testbeds and better understand the design process. Through our wide array of teams, we were able to swiftly identify testbeds across different sectors such as aviation, building technology, transport, public policy and digital technology. Arup’s experts assisted the research team in understanding what made a testbed successful and how to design it. 

Real-world testbeds examples by geography Real-world testbeds examples by geography
Geographic distribution of real-world testbed examples considered for this study

We also simultaneously delivered the design of a ‘Grand Challenge’ for Nesta, focusing on testing drones in cities. The theoretical insights obtained from the testbed report were used to directly inform the design of the drone testbed, and vice versa with the practical experiences, allowing us to test the validity of some of our recommendations in practice.

Our research shows that developing real-world testbeds can help people and places in important ways by:

  • Strengthening collaboration in a clear and structured framework between the public sector, businesses, universities and other research-intensive organisations;

  • Focusing and attracting investment and resources in innovation where the local area is seeking to develop and strengthen a competitive advantage;

  • Reducing risk in the process of developing new products and processes for firms, providing a safe space for them to iterate, fail, influence regulatory and policy change.

The final report contains a new definition of real-world testbeds, a framework of various use-cases, or motivations, for using testbeds, recommendations on how to design effective testbeds and policies incentivising them.

Read the report here.