Protocol for a systematic review of living labs in agricultural-related systems

Abstract

Background

Living labs are innovative platforms that bring together stakeholders (academic and non-academic actors from diverse disciplines), to collaboratively co-create, develop and test new technologies, products, services in real-life environments. As living labs become increasingly popular in addressing sustainability challenges in agriculture, it is essential to understand the potential of living labs to support innovation in the agricultural context. However, the existing knowledge is dispersed, and uncertainties remained regarding their approaches, methods, and outcomes. To address this gap, this study outlines a systematic review protocol of the existing literature on living labs in agricultural-related contexts. This will be done through questions focusing on the contributions of living labs to agricultural sustainability and innovation, their effectiveness, and strategies for development and implementation. Additionally, it will identify areas that require further research and development to advance our understanding of these initiatives.

Method

This study will be conducted according to the RepOrting Standards for Systematic Evidence Syntheses (ROSES) review protocol. We will search databases, including Scopus, Web of Science (core collection) and Google Scholar, which will be limited to titles, abstracts, and keywords in English to eliminate irrelevant literature. The quality of the method used for each selected study will be evaluated using the Critical Appraisal Skills Program, Qualitative Research Checklist. Coding and data extraction will include for example bibliographic information, types of agricultural practices tested, dominant framework (e.g. open innovation, user innovation, and participatory design), the contribution of living lab in agriculture, stakeholder roles, etc. We will use thematic analysis to synthesise our findings. We will compare descriptive and narrative sub-groups to take into account differences in living lab concepts in relation to the context, outcomes, and limitations.

Conclusions

The results of this review will provide a foundation for informing user-centred innovations in agriculture using living lab methodology. Researchers and practitioners working in areas such as co-design which incorporate user involvement, collaboration, and knowledge exchange, may also benefit from these findings.

Background

Agricultural systems have changed over the past half-century as a result of intensification, simplification and agro technologies such as biotechnology, robotics, data-driven technologies, and remote sensing [1, 2]. Moreover, the evidence of the role of agriculture for greenhouse gas emissions, water contamination and biodiversity loss, which occur at the landscape scale, demonstrates that the current agricultural systems are not sustainable [3,4,5,6]. Due to the lack of a systematic connection to sustainable agriculture, the majority of sustainable agriculture innovations at present do not find their way to sustainable agriculture [7]. It prevents their widespread adoption and reduces the potential for their positive effects on the agricultural sustainability transition, which should contribute to the required transformation of the human and nature nexus [8,9,10,11]. However, conventional top-down and internally driven solutions to agricultural innovation and development have been criticized [12]. These approaches have been found to have limited ability to address complex problems that require the involvement of multiple stakeholders. Additionally, they often fail to take into account the needs and knowledge of local farmers and communities, resulting in solutions that are not well-suited to their context [13]. Moreover, they neglect the social and institutional aspects of innovation [12].

To address these limitations, there has been a growing interest in open innovation, co-creation, and collaboration approaches such as the living lab approach [14]. The living lab approach is believed to close this gap by directly involving all end users in the development of agriculture [15, 16]. Living labs are a mechanism or approach that brings various stakeholders together to produce user-centric ideas and solutions,as a result, they might offer a practical approach to resolving complicated problems [17]. Living labs aim to address complex societal challenges by bringing stakeholders together from different backgrounds and disciplines to jointly develop (co-develop) solutions through an iterative process that includes prototyping, testing, and validation of new solutions [18]. In the context of agriculture, living labs have been used to address various sustainability challenges, including food security, water management, and climate change adaptation. Collaborative efforts are leading a new generation of sustainability initiatives that involve a combination of policy instruments, state regulations, and private sector tools. These multi-stakeholder platforms emphasize the importance of hybrid and multi-partner forms of sustainable governance [19]. Policymakers can play an enabling role in modifying regulations, developing tools, or promoting capacity building and training that will assist in scaleing-up and implementing the innovations that have been produced in living labs [20].

Living labs offer researchers the opportunity to work directly with end-users, such as farmers, in real-world environment. This experiential collaboration enhances researchers' understanding of practical challenges, enabling them to develop contextually relevant solutions. It also foster collaboration between researchers from diverse disciplines, such as agriculture, technology, and social sciences. This multidisciplinary approach encourages holistic problem-solving and innovative thinking. Also the Iterative feature of living labs engage researchers in iterative design cycles to refin their solutions based on real-time feedback from stakeholders.

Farmers actively participate in the co-creation of solutions within living labs, ensuring that innovations directly address their needs [21]. This involvement enhances the relevance and acceptance of technological advancements. Moreover, living labs provide a safe space for farmers to experiment with new technologies and innovations, mitigating the risk of adopting unproven solutions. Innovative practices developed through living labs often lead to improved efficiency, reduced costs, and increased yield for farmers [22]. These outcomes contribute to the overall sustainability of farming operations. Farmers also benefit from capacity-building activities conducted in living labs, such as training workshops and knowledge-sharing sessions. Iiving lab provides spase to rexchange experiences, challenges, and best practices, strengthening the farming community as a whole.

There is a growing attempt to consider the concept of living labs in agroecosystem and local agri-food systems [10, 15]. Literature reveals that in the agricultural sector researchers have focused on several aspects of living labs, such as their potential and limits [10], defining characteristics [15], actors’ roles [23], strengthening agricultural knowledge and innovation systems [20], evaluation [24, 25], and establishment [14]. Despite such research examples, knowledge is still scattered across different aspects of living labs and multiple uncertainties regarding living lab approaches have been highlighted [23]. In light of the present challenges, there is a great demand to understand how the living lab methodology can be used for fostering agricultural innovations supporting transformation towards resilience and sustainability [15, 20]. Developing an in-depth understanding of the theoretical foundations and methods by which living labs have been established and utilized within agricultural-related systems, is an essential initial phase towards optimizing the potential of living labs in agriculture. To the best of our knowledge, there is no literature that focuses on a systematic review of agricultural-related system living labs. In this research, agricultural-related systems refer to various aspects of agriculture, including but not limited to crop production and food processing. In the context of living labs, agricultural-related systems may also include the development and testing of innovative technologies, practices, and business models aimed at improving the efficiency, productivity, and sustainability of agriculture. Examples of specific agricultural-related systems that may be studied in living labs include precision agriculture, agroforestry, organic farming, ecosystem services, and climate-smart agriculture.

Living labs, traditional systems and real-world laboratory approach

The living lab concept started outside of academia in the 2000s, revolving around testing new technologies in home-like constructed environments to promote entrepreneurial innovation [26] and evolving into transdisciplinary scientific methods. The idea of Real-World Laboratories (RWL) (also known as "Reallabore") developed into a scientific field from the beginning. It is especially common in German-speaking academic groups that prioritize sustainability and transdisciplinary transformative research [27]. Stakeholder involvement and real-world experimentation are both aspects of RWLs and living labs, but the emphasis of RWLs is more on the process of learning, exploring, and testing novel structures and transformative processes than the actual implementation of a particular invention [28] which receives more attention in the living lab approach.

There are some differences between a living lab and a traditional system. The first difference between living Labs and traditional systems is related to the concept of realism. In the living lab approach, a real user is involved in real-world contexts and situations. While in the traditional system, the user is asked to use the innovation in a context that the researcher defines [29]. Openness is another dimension of difference between the two systems. A living lab is defined as an open innovation approach (Beaudoin et al., 2022) [24], to integrate internal and external ideas into the development process. However, a traditional system limits the number of inflows into the development process for financial reasons.

In living labs, all stakeholders are considered equal partners in the innovation process. But in systems development, end users are seldom seen as partners but rather as actors, even though their organizations might be viewed as partners [29]. Bergvall-Kareborn et al., [29] identified that in living labs, most activities are accomplished in close relation to academia, while this is often not the case within more traditional systems. Leminen & Westerlund [30] investigated the difference between a living lab and a traditional project model in the context of objectives (which is not defined a priori in the living lab but defined in the traditional model), the role of the project manager, and user communities, resources and capabilities.

While the characteristics of agroecosystem living labs were defined at the G20 Meeting of Chief Agricultural Scientists (G20 MACS) in 2018, practical implementation of the agricultural living lab may be hindered by a lack of literature on agroecosystem living labs and their unique characteristics. Therefore, there is an opportunity for future research to identify and describe the distinctive features of agroecosystem living labs, which would be beneficial for living lab managers and those seeking to establish a living lab to improve the sustainability and resilience of agriculture and agri-food systems [15].

Objective of the review

By synthesising the existing knowledge of living labs in agricultural-related systems, we aim to provide a comprehensive and up-to-date understanding of the potential of living labs to support innovation and transformation in the agricultural landscape context, as well as identify areas for future research and development in this field. This synthesis will involve a systematic review of the relevant academic and grey literature, including peer-reviewed articles, reports and other publications related to living labs in agricultural-related contexts. Specifically, the review will aim to identify the aims, theoretical frameworks, processes, methods, and geographical domains of the application of living labs in agricultural related systems. Additionally, the study will document any existing strengths and shortcomings in current applications if possible. To achieve these objectives, a series of research questions are formulated, which will guide the analysis of the literature as follows:

• 1. What are the contributions of living labs to agricultural sustainability and innovation reflecting the outcomes of living labs across agricultural contexts?

• 2. What are the factors that influence the effectiveness of living labs in agricultural-related systems related to the theoretical foundations, activities and role of stakeholders?

• 3. What are the current knowledge gaps in the literature on living labs in agriculture, and what areas require further research and development to advance our understanding of these initiatives?

These research questions are not exhaustive and may be refined during the coding and critical reading of the studies, but they will serve as the basis for the review.

Method

This study will be conducted according to the RepOrting Standards for Systematic Evidence Syntheses (ROSES) review protocol, [31] which is a recognized tool in environmental management. The systematic search process involves identifying relevant studies, screening based on inclusion and exclusion criteria, and assessing their eligibility. The study also involves quality appraisal, data acquisition, abstraction, and analysis.

Search for articles

Search terms and strings

The process of identifying search terms for a systematic review involves selecting relevant keywords and terms that describe the topic of interest. We identified the search terms through a combination of methods, including reviewing relevant articles and brainstorming with experts and members of the research team and searching in synonym databases.

Search sources

The primary information sources were chosen to search grey literature, search engines, online databases, and electronic journals. For this research, the literature was found by searching the Google Scholar, Scopus and Web of Science (core collection) databases, which were limited to titles, abstracts, and keywords to eliminate irrelevant papers. Google Scholar was chosen to identify grey literature and support quick searches through snowballing. To minimize algorithm biases associated with previous internet searches, we will disable browser history and cookies during the internet search process. Additionally, we will use the "private" navigation mode to ensure that our search results are not influenced by previous search activity. According to Haddaway et al. [32], we will select the first 200 relevant results of the search to avoid non-relevant literature in Google Scholar. Searches in databases will not be limited by year, however, access would be restricted to specific years by institutional subscriptions.

Supplementary searches

In case of an insufficient number of papers, the reference lists of included studies will be snowballed for identification of other relevant studies.

Estimating the comprehensiveness of the search

We initially identified benchmark articles according to the eligibility criteria (Additional file 4) in order to improve and examine the search string's comprehensiveness. We aimed to include all relevant studies related to living labs in agricultural-related systems. For our systematic review, we have developed search terms that include two main components related to living labs and agricultural systems, and we will not be limiting the results to specific criteria, in order to comprehensively identify relevant studies. Based on the search plan outlined in Additional file 4, it was found that all search steps would have successfully captured all the articles listed in the benchmark list.

Search result

The full-text publications will be obtained by organizational subscriptions and the search results will be combined and de-duplicated in the EndNote 20 desktop version.

Article screening and study eligibility criteria

Screening process

To minimize the likelihood of bias, a blinded dual-reviewer, with a two-step selection procedure based on pre-established inclusion criteria will be implemented. The titles and abstracts of all studies through database searches will be independently checked for relevance by two qualified reviewers during the primary screening. Research technicians and qualified research assistants will participate as reviewers; they will be blind to each other's selection for inclusion. Each study will be classified as ‘include’, ‘exclude’ or ‘maybe’; studies designated as ‘maybe’ decisions will be re-examined through secondary screening. The full texts of articles that had been previously classified as "exclude" or "maybe" will be independently reviewed by two trained reviewers following pre-established inclusion criteria for the secondary screening. Inconsistencies will be resolved by discussion with a third reviewer (main author), who will be blinded to which reviewer did not vote 'exclude' to prevent bias. If the full texts of possibly included studies are unavailable, we will contact the stated associated author, twice with a one-week interval between each attempt. If this method is ineffective, the article will be excluded from the review. Multiple reports from a single study will be collated as a single resource in the SR.

Eligibility criteria

Given the number of descriptive and case study-related papers on living labs identified, the following exclusion criteria were considered:

Eligible participants and study design

Studies whose participants include farmers, decision-makers, or users of knowledge who are relevant to the living lab in the primary research will be included. If the full-text document only mentions ‘living lab’ as a phrase in passing and no description of the concept or approach is provided, the document will be excluded. When living labs are discussed outside of agriculture, forestry, and the food system, the article will be excluded.

Intervention

Studies that examine or discuss the development, implementation, and/or evaluation of living labs for any agricultural-related system purpose in any population or context will be included. The empirical publications that may suggest overlap in terms of innovative and user-centered concepts but do not mention or identify themselves as the living lab will be excluded.

Outcomes

We will include studies that report outcomes related to how living labs have been implemented and within the context of agricultural-related systems in relation to the development, assessment, and impact on agricultural innovation or research of living lab principles. Studies written in languages other than English will be excluded.

Study validity assessment

The quality of the method used for each selected study will be evaluated using the Critical Appraisal Skills Programme [33]. The evaluation of the research evidence will include an assessment of the quality of the method employed by each study. The checklist comprises 10 questions regarding research aims and objectives, design, recruitment strategy, data analysis and synthesis, findings, and overall research value. Each question will require a response of "yes," "no," "unclear," or "not applicable," as presented in Additional file 3.

Data extraction

A data extraction framework facilitates the extraction of bibliographic and methodological information about each study. In our research, the data extraction framework will be developed based on three main sources: literature review, expert consultation, and a pilot test on a selection of studies chosen at random. Firstly, the literature review will provide a comprehensive overview of the components of living labs. This will involve identifying relevant databases and conducting a thorough search using relevant keywords to include essential components of living labs in the in-depth analysis. Secondly, expert consultation will be used to gather insights and perspectives from individuals who have expertise in the field of living labs in agriculture. These experts will be identified through networking and referrals, and their opinions will be sought through interviews or online surveys. Finally, a pilot test and detailed examination of studies chosen at random will help to modify the data extraction framework if necessary. By using a conceptual framework that integrates these three sources, the review will be able to provide a robust analysis of living labs in agriculture that is grounded in both empirical evidence and expert opinion. We will probably extract from each article the following information: title; authors; year of publication; journal, publication title; country, the research aims; types of agricultural practices tested, design, dominant framework (e.g. open innovation, user innovation, participatory design), the contribution of the living lab to agricultural related context, multimethod approaches (e.g. surveys, interviews, observations, participatory scenarios [34], duration of the living lab (short term and long term), role of stakeholders, main findings relevant to our review questions,limitation and strength and reviewer comments.

Data coding and extraction strategy

We highlight the importance of using rigorous and systematic methods to analyze data in a systematic review, to ensure that the findings are reliable and relevant to the research question. We will use thematic analysis to synthesize our findings [35]. Thematic analysis is a widely used qualitative research method for analyzing data that involves identifying, analyzing, and reporting patterns (themes) within the data. The steps involved in conducting thematic analysis are as follows [35, 36]: 1) familiarization with the data, reading and re-reading the data to gain an overall understanding of its content, 2) generating initial codes, which can be descriptive or interpretive, 3) searching for themes to identify potential themes by grouping related codes and exploring the relationships between them, 4) reviewing themes, refining and naming the themes based on their coherence, consistency, and relevance to the research question 5) defining and naming themes to define and describe each theme in detail, 6) writing and summarizing the findings and presenting them clearly and coherently. In case of obtaining and confirming missing or unclear information or data from authors, we will attempt to contact them through email or other available means of communication. If we do not receive a response, we will make a note of the missing information or data and proceed with the available information.

To test the repeatability of the meta-data and data extraction process in our systematic review, we will perform a pilot study using a small subset of articles. First, we randomly will select a sample of articles from full set of articles. Then, we will assign two independent reviewers to extract data and meta-data from the articles in the sample. After the extraction is complete, we will compare the results of the two reviewers to determine any discrepancies. If the reliability is high, we will proceed with the full review process. If it is not satisfactory, we may need to refine the data extraction form.

Data synthesis and presentation

We will use the Rayyan platform for the organization and screening of the studies. Rayyan https://www.rayyan.ai/) is a free intelligent research collaboration platform that facilitates collaborative citation screening and improves adherence to SR methodology [37]. We do not anticipate performing a meta-analysis or risk-of-bias assessment, as the previous reviews on living labs suggest that most research in this field is qualitative or descriptive, and there is expected to be significant diversity in the data. We plan to create evidence tables that will help us categorize and analyse domain information across core extraction variables. We will compare descriptive and narrative subgroups to take into account differences in innovation tools, researcher roles, and other living lab concepts concerning context, outcomes, and limitations. The lead author will oversee the analytical process, in collaboration with research assistants and technicians. Through discussion with other collaborating researchers, they will ensure that evidence tables are comprehensively and accurately interpreted, allowing for the construction of narrative summaries and recommendations. We will report detailed information on the articles that have been included or excluded at each stage of the screening process, along with any modifications made to the present protocol.

Given the potential multidisciplinary nature of living lab research and the the complexity of the data, we anticipate that the findings of this systematic review may yield a substantial amount of valuable information across various aspects of the topic. To ensure comprehensive reporting and dissemination of the results, we plan to publish multiple papers based on the outcomes of this review. These papers will be structured to address specific research questions or themes identified during the review process.

Stakeholder involvement

A team at the Leibniz-Centre for Agricultural Landscape Research (ZALF) is leading a living lab project in agriculture. The team specializes in various fields related to agriculture and aims to work with internal and external experts in the field. The project aims to open calls through social media, and team networks to undertake the project's review. The team plans to disseminate the results in a one-page summary with infographics for academics and practitioners. Through this research, the team aims to promote sustainable agriculture and bridge the gap between research and practice.

Implications for policy/management

The findings from this systematic review could provide valuable insights into the potential of living labs to support innovation in the agricultural landscape context. The review could identify best practices, strengths, and shortcomings of living labs in agricultural-related systems. Policymakers and managers could use this information to design and implement living lab initiatives in the agriculture sector that are more likely to be effective and achieve their intended goals.

Additionally, the review could help identify areas for further research and development to advance our understanding of these initiatives, potentially leading to more effective policies and management strategies.

Implications for research

The implications for research from the planned systematic review will flag the need but also provide guidance for further empirical studies on the development, implementation, and evaluation of living labs in agricultural-related systems. Additionally, it will help to plan further research about the factors that influence the effectiveness of living labs in agriculture, as well as the theoretical foundations, processes, methods, and tools used in these initiatives. Researchers and practitioners working in areas such as co-design and integrated knowledge translation, which incorporate user involvement, collaboration, and knowledge exchange, may also benefit from these findings.

Conclusions

The existing literature on living labs in an agricultural context show design limitations in reporting and comprehensiveness issues that affect the reliability and usefulness of the living lab findings. This upcoming review on living labs in agriculture will address these gaps in knowledge by focusing on agriculture-related concepts and the potential for collaborative, multi-stakeholder, and innovative initiatives. The results of this review will provide a foundation for informing subsequent user-centred innovations in agriculture using living lab methodology. By placing emphasis on these aspects, the review aspires to contribute significantly to the existing understanding of living labs in agriculture. The insights garnered from this review are expected to lay the groundwork for informed decision-making and strategic planning in subsequent user-centered innovations within the realm of agriculture, all underpinned by the robust methodology of living labs. This proactive approach seeks to not only rectify the observed design limitations and reporting issues but also to pave the way for the practical implementation of sustainable, impactful, and transformative initiatives that directly address the complexities of agricultural challenges.