Table of Contents
Year : 2019  |  Volume : 4  |  Issue : 3  |  Page : 81-86

A web-based, real-time quality control and progress monitoring tool for multicenter environmental health surveys

1 National Institute of Environmental Health, Chinese Center for Disease Control and Prevention, Beijing, China
2 Fielding School of Public Health, University of California, Los Angeles, California, USA

Date of Submission04-Jul-2019
Date of Acceptance18-Sep-2019
Date of Web Publication30-Sep-2019

Correspondence Address:
Tiantian Li
Chinese Center for Disease Control and Prevention, National Institute of Environmental Health, Beijing
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/cp.cp_18_19

Get Permissions


Background: Large multicenter surveys are increasingly popular in the environmental health field. The quality of the data and management of surveys is crucial to the success of the project. However, none of the existing survey tools include remote, multirole, and real-time quality control of the survey. We have therefore developed an electronic investigation system (EIS), with multirole real-time quality control and progress monitoring modules. Methods and Results: Our investigation system has three levels of quality control officers. Questionnaires verified by the lower-level quality control officers can also be verified at a higher level. The strictly hierarchical verification roles improve quality control even more than retaining quality control officers at each study center. Project investigators can also monitor the progress of the project and each center daily and annually, for a real-time understanding of the performance of each investigator and quality control officer. Conclusion: Our EIS has met the requirements for multicenter studies in the environmental health field. Researchers may use our design to conduct multicenter surveys more efficiently in future.

Keywords: Multicenter survey tool, progress monitoring, real-time quality control, web-based

How to cite this article:
Zhang Y, Fang J, Du P, Wang J, Ma R, Pei H, Li T. A web-based, real-time quality control and progress monitoring tool for multicenter environmental health surveys. Cardiol Plus 2019;4:81-6

How to cite this URL:
Zhang Y, Fang J, Du P, Wang J, Ma R, Pei H, Li T. A web-based, real-time quality control and progress monitoring tool for multicenter environmental health surveys. Cardiol Plus [serial online] 2019 [cited 2021 Oct 25];4:81-6. Available from:

  Introduction Top

The environmental health field has seen an increase in the number of multicenter surveys conducted nationwide and worldwide.[1],[2] Standardization of data collection and ensuring data quality is vital to maintain the objectivity of multicenter surveys. We were involved in planning a face-to-face nationwide survey on health effects related to air pollution in China, from July 2016 to June 2020. China has network of centers for disease control and prevention (CDC) from county level to country level. This advantage gives us the opportunity to easily conduct nationwide epidemiology surveys. This multicenter survey was organized and delivered by the National Institute of Environmental Health (NIEH), China CDC, Provincial CDC, and County-level CDC. We selected 57 communities in China as our study areas, and the sample size was about 10,500 participants [Figure 1]. The project collected five types of survey data: (1) basic information on counties and communities; (2) family income, housing situation, cooking frequency, ventilation, use of heating, air conditioner and air cleaner; (3) socioeconomic status and health conditions; (4) a 24-h retrospective questionnaire covering smoking and drinking status and use of medicine; and (5) the results of physical examinations, which were uploaded to an online system. These included the results of routine blood and urine tests, electrocardiography, and pulmonary function tests.
Figure 1: Map of survey sites used electronic investigation system

Click here to view

Traditional paper-based questionnaires have many limitations for data management and usability. These include difficulties inputting data into an electronic database and performing quality control. Quality control for a multicenter study in environmental health is difficult because it must be performed simultaneously at each center, due to the effects of seasonal and climate changes. Traditional electronic questionnaires also have numerous limitations. For example, Epidata's ( open-access software can embed logic checks, but cannot provide real-time data sharing or central data manual control when used for multicenter studies. The EpiData database is also unable to manage photographs and signatures. Walther et al. compared time required and error rate between an electronic data capture system and a paper-based method for a survey.[3] They concluded that the electronic data capture system was more efficient and had a lower error rate than the paper-based method. However, no electronic data capture or management system has been developed to capture investigation information in the environmental health field.[4],[5],[6] To assure the quality of the data and the efficient progress of multicenter surveys, a new electronic system needed to be developed.

Our project investigators (PIs) seized the opportunity to develop an electronic investigation system (EIS). Our aims were to: (1) develop an EIS to capture digital information directly from the survey field, including numbers, characteristics, signatures, and photographs; (2) establish a real-time electronic quality control method to improve the efficiency of quality control in multicenter surveys; and (3) demonstrate the real-time progress of each center, to make project management easier for the PIs. This is the first time that an EIS has been developed for a nationwide multicenter survey in the environmental health field. The design and outcomes from our study are crucial for future projects so other researchers can avoid our limitations and improve the quality of their surveys.

  Software Implementation Top

Basic foundation

Our EIS is based on a Browser/Server structure (a web-based system) that can be deployed on terminal equipment, such as mobile phones, tablet computers, and other equipment. We used tablet computers most often. The website for the system is The system has four basic modules: data entry, real-time quality control, real-time progress control, and data management. The basic workflow of the EIS is shown in [Figure 2]. Our EIS has obtained software copyright from the National Copyright Administration of the People's Republic of China (ID: 2017SR603994).
Figure 2: The basic workflow of electronic investigation system

Click here to view

The main project staff roles are PIs, investigators, quality control officers, and system managers. Their responsibilities are shown in [Table 1]. Each person with an account on the system has to be assigned one of these system roles. The system character is the combination of system authorities, and each authority corresponds to an operation or set of operations in the system. Data safety is assured by rigorously assigning the characters. All project staff members need to submit their E-mail address and organization information for the system manager to assign an account to them. In the hierarchical user management system, there are three levels of PIs and quality control officers (i.e., national, provincial, and county levels). The national-level quality control officers can see all the entries, provincial quality control officers can only see entries from their provinces, and county-level quality control officers can only see their own survey fields. The county CDCs of the study areas must collect and submit basic information about all the residents in the filtered community, including name, date of birth, sex, ID, phone number, length of residence and address to NIEH, China CDC. After random sampling using R software (version: 3.4.1), each individual in the sample was allocated a random serial number. The system managers embedded this basic information into the EIS.
Table 1: The main staff involved in the project and their responsibilities

Click here to view

Real-time quality control module

Our study used three levels of real-time quality control. The rates of quality control sampling for county-level CDCs, provincial CDCs and NIEH, China CDC were 100%, 30%, and 20%. Once an investigator submits a questionnaire, it can be viewed and queried by the quality control officers. Once submitted, a questionnaire cannot be modified unless the quality control officer creates a query about it. From that point, all amendments can be seen by quality control officers. When the questionnaire has been finished and verified by the lower-level quality control officer, it can still be queried by a higher-level quality control officer. The basic workflow for the quality control process is shown in [Figure 3]. The real-time quality control module has a filter above the subject list, including three options, “National level,” “Provincial level,” and “County-level”. Using the filter, the PIs and quality control officers can see how many questionnaires have been checked by the quality control officers at different levels. This means they can all monitor the real-time progress of quality control and check the activity of project staff at the same or lower levels.
Figure 3: The basic workflow of the quality control module

Click here to view

Real-time progress monitoring module

The system can provide real-time statistics and graphs for the number of participants surveyed daily, weekly, monthly, and overall, so the progress of each center can be monitored. The system can also provide a summary table of the number of questionnaires completed by each investigator, the rate of queries and the error rate. The system also generates real-time tables of the progress of quality control [Figure 4]. The PIs and the quality control officers can view the number of questionnaires queried or verified by each quality control officer. These data can be used to determine how many questionnaires still need to be investigated or verified. The system manager can send notices to all the users of the system, and users can send questions and responses to the manager.
Figure 4: Report bar chart and tables generated automatically by the real-time progress monitoring module. (a) Number of the enrolled subjects each day in one center; (b) Work performance of each staff. Fake names and real data were used

Click here to view

Special design for follow-up surveys

This study required an option for a follow-up survey. The system is, therefore, able to automatically connect basic information from the participants in the original and follow-up surveys. When the follow-up survey was launched, the investigator could enter the follow-up survey entrance for the particular county, and search for the name or serial number of a participant, and the system would show the questionnaire for that person. The questionnaire was simplified in the follow-up survey because all the follow-up surveys were conducted within 1 year of the initial survey, so most of the basic information was unchanged. If data were censored, we added information on the participant's status, including survived, loss to death, or loss to follow-up. If the participants were lost to death or follow-up, the investigator could also choose to include the cause of death and the reason for the loss to follow-up. To ensure the data from the follow-up survey were consistent with the original survey, the participants completed their ID again in the follow-up survey. Date of birth, sex, and other basic information were not included in the follow-up survey, so the basic personal information from the first survey was automatically added for each participant in the follow-up survey, for the convenience of investigators and quality control officers.

  Discussion Top

This is the first time of which we are aware that a real-time quality control and progress monitoring tool for a multicenter survey has been reported. Our EIS contains both a data entry module and a real-time quality control module, for the remote management of the survey. The hierarchical levels of the quality control structure were embedded in the EIS to streamline the quality control process. The real-time monitoring module means that progress report charts and tables can be generated automatically in real-time. The automatic location function and the signature and photograph upload function were developed specifically to solve the difficulty of collecting and storing this type of information. The aim was to develop an EIS that could be used for a nationwide multicenter population-based study in the environmental health field.

The strengths of our electronic investigation system

A few studies have used electronic systems in population-based surveys. Kumar et al. developed a system based on Epidata, Dropbox (, and Team Viewer ( for a multicenter study.[4] The three tools were used for data entry, data-sharing, and problem-solving. Our EIS contains all three functions, and is web-based, with no need to install any software. This makes it much easier to use. As well as real-time data sharing, our EIS has a real-time quality control and monitoring function. Meyer et al. established a central data management system to store, refine, and transfer data from multiple studies.[5] This system collected unified data, and also managed data from different studies. It, therefore, focused on data cleaning and reorganization, whereas our EIS focused on data collection, real-time quality control, and program management of the multicenter project. Tao et al. introduced a data capture framework, which was specifically designed for intervention studies.[7] The system had the advantage of defining specific workflows in a clinical trial or experimental study. However, it was not adapted for use with a multicenter cohort. The workflow for the cohort study was simple, and the system can be embedded as a built-in feature. However, a multicenter survey like ours needs multirole real-time quality control, because of the different subteams operating in the different centers.

Real-time online multirole interaction quality control

The first reason we chose a web browser-based system was that this project was a nationwide multicenter study, and we knew that we needed standard real-time quality control. Previously, investigators using paper-based questionnaires or the Epidata software often did not upload the data until the last day of the survey, which sometimes resulted in delays in quality control. If quality control is delayed, detailed information may be forgotten by the investigators and the participants, which will influence the quality of the data. A delay in quality control can also occur in multicenter surveys using applications (apps) instead of an online website. With the online EIS, the survey data can be uploaded in real-time. Quality control officers can check the uploaded data and send feedback to investigators immediately. This means that investigators will receive the feedback the next time they log into the website and they can amend the data as necessary. The fourth-generation wireless network covers all the cities and main villages/towns in China,[8] so investigators can access the system online using the public network, improving availability. Unlike previous systems, our EIS has three levels of real-time quality control: County-level CDCs, provincial CDCs and NIEH, China CDC. This made it more applicable for our purposes than other systems that are usually used for randomized controlled trials.[6],[9]

Real-time multicenter survey progress management

We developed a real-time progress management module and other functions to help the PIs capture progress at each center. Previously, if we wanted to determine the progress at each center, we needed to contact each center to acquire this information. The real-time progress management module provides immediate access to the numbers of participants enrolled and the percentage and progress of enrollment each day. The system also contains information on the number of questionnaires that completed quality control and the total number of queries. The PIs can simply log in to the website and obtain this information quickly and easily. We are also able to generate reports on the workload and quality of each investigator and quality control officer. This design makes it convenient for the PIs and quality control officers to focus attention on particular investigators.

Other benefits

In the environmental health field, addresses are very important, so we introduced an automatic location function to our EIS. This can save time and provide a more exact location. Our EIS has also solved the photograph and signature uploading issues common in paper- and EpiData-based survey tools. This means different researchers can remotely check this information, conveniently and simultaneously. To help participants complete the questionnaire, we established our EIS as a face-to-face interview system. This means we assigned the role of investigators and ensured that only they can enter answers, based on responses from participants.[10]

Limitations and future development

We have used the EIS to conduct a survey in 57 counties and enrolled more than 10500 participants across China. However, our EIS still has some limitations. Each week, we summarized the stages of the system and investigation status of all study centers, including the information on whether the sample has been embedded and quality control status. We had to collect the information separately for each center on the system. This is time-consuming, and in future, we hope the system will be able to summarize the excel report form and send the E-mail automatically to every staff member. We also have numerous other documents (such as the survey notes and supervision reports) related to each study area, which needed to be filed. We currently file them manually and store them in the PIs' computers. We are still developing the upload entrance and the filing function. Finally, we have currently separated the EIS and Biobank systems. Information on the storage of bio-samples of each participant is only available from the Biobank system. To improve the workflow, the two systems need to be brought together to establish an interface.

  Conclusion Top

We have developed a unique and efficient EIS for use in a particular multicenter study. Its structure and workflow can be used in other similar studies. The EIS has many special functions, such as real-time quality control and monitoring, and being able to facilitate follow-up studies. Our EIS improved data quality, and minimized time and travel expenses of the project. However, it still has limitations. We will continue to improve existing functions and develop new functions to improve its efficiency and workflow.


We would like to thank Mr. Guang Zhao, director of Solution (Shanghai) Science and Technology CO., Ltd., for his contribution in on-line survey system implementation.

Financial support and sponsorship

This work was funded by grants from the National Natural Science Foundation of China (Grant: 91543111, 21277135), Beijing Municipal Natural Science Foundation (Grant: 7172145), and National High-level Talents Special Support Plan of China for Young Talents and Environmental Health Development Project of National Institute of Environmental Health, China CDC.

Conflicts of interest

There are no conflicts of interest.

  References Top

Cohen MA, Adar SD, Allen RW, Avol E, Curl CL, Gould T, et al. Approach to estimating participant pollutant exposures in the multi-ethnic study of atherosclerosis and air pollution (MESA air). Environ Sci Technol 2009;43:4687-93.  Back to cited text no. 1
Andersen ZJ, Stafoggia M, Weinmayr G, Pedersen M, Galassi C, Jørgensen JT, et al. Long-term exposure to ambient air pollution and incidence of postmenopausal breast cancer in 15 european cohorts within the ESCAPE project. Environ Health Perspect 2017;125:107005.  Back to cited text no. 2
Walther B, Hossin S, Townend J, Abernethy N, Parker D, Jeffries D. Comparison of electronic data capture (EDC) with the standard data capture method for clinical trial data. PLoS One 2011;6:e25348.  Back to cited text no. 3
Kumar AM, Naik B, Guddemane DK, Bhat P, Wilson N, Sreenivas AN, et al. Efficient, quality-assured data capture in operational research through innovative use of open-access technology. Public Health Action 2013;3:60-2.  Back to cited text no. 4
Meyer J, Ostrzinski S, Fredrich D, Havemann C, Krafczyk J, Hoffmann W. Efficient data management in a large-scale epidemiology research project. Comput Methods Programs Biomed 2012;107:425-35.  Back to cited text no. 5
Zhang J, Sun L, Liu Y, Wang H, Sun N, Zhang P. Mobile device-based electronic data capture system used in a clinical randomized controlled trial: Advantages and challenges. J Med Internet Res 2017;19:e66.  Back to cited text no. 6
Tao S, Zeng N, Wu X, Li X, Zhu W, Cui L, et al. A data capture framework for large-scale interventional studies with survey workflow management. AMIA Jt Summits Transl Sci Proc 2017;2017:278-86.  Back to cited text no. 7
The Central People's Government of the People's Republic of China; 2017. Available from: [Last accessed on 2019 Jun 20].  Back to cited text no. 8
Rorie DA, Flynn RWV, Grieve K, Doney A, Mackenzie I, MacDonald TM, et al. Electronic case report forms and electronic data capture within clinical trials and pharmacoepidemiology. Br J Clin Pharmacol 2017;83:1880-95.  Back to cited text no. 9
Wong D, Cao S, Ford H, Richardson C, Belenko D, Tang E, et al. Exploring the use of tablet computer-based electronic data capture system to assess patient reported measures among patients with chronic kidney disease: A pilot study. BMC Nephrol 2017;18:356.  Back to cited text no. 10


  [Figure 1], [Figure 2], [Figure 3], [Figure 4]

  [Table 1]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
Software Impleme...
Article Figures
Article Tables

 Article Access Statistics
    PDF Downloaded231    
    Comments [Add]    

Recommend this journal